With the explosive growth of unstructured data across virtually all industries, from digital media in the creation, management and delivery of entertainment content, to genome sequence data used in scientific research, fast and reliable movement of massive digital data over global distances is becoming vital to business success. Individuals and teams need to collaborate daily across global distances, distributing, sharing, and exchange multi-terabyte data sets, demanding high-speed movement of big data around the world. However, changes to TCP - the Internet’s underlying transfer protocol developed in the 1970's, have not kept up with these trends. Unable to take full advantage of modern high-bandwidth networks, TCP is thoroughly unsuitable to the demands of today's big-data applications. Attempts at TCP “acceleration” have delivered only incremental improvements that cannot overcome the inherent design flaws in the core protocol.
Aspera's patented fasp™ transfer technology is an innovative software that eliminates the fundamental shortcomings of conventional, TCP-based file transfer technologies such as FTP and HTTP. As a result, fasp transfers achieve speeds that are hundreds of times faster than FTP/HTTP and provide a guaranteed delivery time regardless of file size, transfer distance or network conditions, including transfers over satellite, wireless, and inherently long distance and unreliable international links. fasp also provides complete visibility into bandwidth utilization and extraordinary control over transfer rates and bandwidth sharing with other network traffic. Complete security is built-in, including secure endpoint authentication, on-the-fly data encryption, and integrity verification.
The Transmission Control Protocol (TCP) provides reliable data delivery when deployed on low latency and low packet loss networks, but it quickly becomes unreliable and slow with increased packet loss and latency that are typical of long-distance WANs. Adding more bandwidth does not change the effective throughput. File transfer speeds do not improve and expensive bandwidth is underutilized.
The source of the throughput bottleneck is TCP's flow rate control mechanism in which the sender requires an acknowledgement for each data packet sent to the receiver. If an acknowledgement does not arrive back on time due to high network latency or packet loss, the sender assumes that its sending rate is too fast for the receiver to handle, severely reduces it and recovers it slowly. As a result, transfer rate slows to a crawl and the modern high-bandwidth WAN pipes are significantly underutilized.
In contrast to TCP, fasp™ throughput is independent of network latency and robust to extreme packet loss that can be found on intercontinental WANs and on satellite, Wi-Fi or cellular connections. The result is transfer times that are hundreds of times faster than standard FTP and highly predictable, regardless of network conditions.
To achieve this performance, fasp does away with TCP's coupling of rate control and reliability, with a unique and patented approach that enables maximum transfer speeds, without compromising congestion avoidance, and achieves ideal efficiency and full bandwidth utilization.
Maximum Transfer Speed
fasp enables large data set transfers over any network at maximum speed, source disk to destination disk, regardless of network conditions or distance. For example, over gigabit WANs with 1 second RTT and 5% packet loss, fasp achieves 700-800 Mbps file transfers on high-end PCs with RAID-0 and 400-500 Mbps transfers on commodity PCs. Large data sets of small files are transferred with the same efficiency as large single files. The implementation is very lightweight, and thus does not require specialized or powerful hardware in order to maintain high speeds or high concurrency.
Full utilization of available bandwidth
In addition to significant transfer rate gains, fasp is able to fully utilize the available bandwidth, maximize use of the existing infrastructure and eliminate costly upgrades that may not even benefit TCP-based protocols.
Configurable “fair play” to standard TCP traffic
While fasp can fill any available bandwidth, it also includes an intelligent adaptive transmission rate control mechanism that throttles down for precision fairness to standard TCP traffic, and automatically ramps back up to fully utilize the unused bandwidth. This ensures that business-critical TCP traffic such as email, web, and business applications can function normally while allowing fasp to utilize unused bandwidth.
EXTRAORDINARY BANDWIDTH CONTROL
The patented rate control allows extraordinary dynamic control over transfer speeds and bandwidth sharing: applications and end users can pre-set and change on the fly individual bandwidth priorities, transfer rates and finish times, without the support of network QoS.
The fasp protocol also provides comprehensive built-in security model that does not compromise transfer speed. Based on open standards cryptography, it provides SSH end-point authentication, on-the-fly data encryption (with the option to store files encrypted on the receiver’s end), and data integrity verification that protects against man-in-the-middle, re-play, and UDP denial-of-service attacks.
THE TCP BOTTLENECK
The Transmission Control Protocol (TCP) provides reliable data delivery under ideal conditions, but has an inherent throughput bottleneck that becomes obvious, and severe, with increased packet loss and latency found on long-distance WANs (see Figure 1 below). Adding more bandwidth does not change the effective throughput. File transfer speeds do not improve and expensive bandwidth is underutilized.
TCP performance over increasing network latency and
packet loss conditions normally present on long-distance WANs
The source of the throughput bottleneck is the mechanism TCP uses to regulate its data flow rate. The TCP sender requires an acknowledgment of every packet from the TCP receiver in order to send more data. When an acknowledgment is missed, the sender assumes that it is overdriving the network capacity and enters an aggressive congestion avoidance mode, severely reducing the data flow rate and recovering the rate too slowly to keep modern high-bandwidth pipes full. Even small variation in latency or network errors can cause TCP to enter congestion avoidance. Standard TCP is not equipped to distinguish between the reasons behind packet loss with dramatic consequences to file transfer speeds.
The result of TCP's self-induced bottleneck leads to disappointing and unpredictable transfer times, particularly over long distance networks, and severe under-utilization of available and expensive bandwidth. Large data transfers are slow and unreliable, if not impractical. In local area networks with only a fraction of a percent packet loss, even on gigabit Ethernet, TCP’s maximum throughput is only 50 Mbps (5% bandwidth utilization). An FTP transfer across the United States over a link with 90ms latency and 1% packet loss has a maximum theoretical limit of 1.7 Mbps, independent of available bandwidth. On high-latency, high-loss intercontinental links or satellite networks, the effective throughput may be as low as 0.1% to 10% of available bandwidth. On a typical global link (3%/150ms), maximum TCP throughput degrades to 500-600 Kbps, utilizing only 5% of a 10 Mbps link.
Bandwidth underutilization and erratic transfer rate swings result in unpredictable delivery times and failed transfers that lead to missed business-critical deadlines and require costly transfer oversight and redundancy. TCP acceleration devices may improve throughput under ideal conditions by tweaking the deceleration and recovery rates, but do not address the fundamental deficiencies of TCP-based file transport.
TCP’s security and monitoring are also insufficient for today’s business concerns about security of their digital assets. FTP often requires external security mechanisms to prevent content piracy or tampering, and network performance details and transfer statistics are not available for monitoring or billing.
Unlike TCP, fasp™ throughput is perfectly independent of network delay and robust to extreme packet loss. As shown in Figure 2 below, fasp™ transfer times are as fast as possible (up to 1,000x standard FTP) and highly predictable, regardless of network conditions. The maximum transfer speed is limited only by the resources of the endpoint computers (typically disk throughput).
fasp versus TCP on a 1Gbps WAN link
over increasing network latency and packet loss
THE ASPERA ADVANTAGE
fasp is a new large data transport protocol that achieves reliability in the application layer in a novel approach that eliminates TCP's inefficient loss and error handling, and the resulting erratic transfer rate swings. To guarantee 100% reliability, fasp implements its own, theoretically proven optimal mechanism that identifies and retransmits precisely the real packet loss on the channel. At 10% packet loss, fasp achieves 90% bandwidth utilization with less than 1% redundant data overhead.
In contrast to brute-force multi-stream TCP approaches, fasp™ achieves ideal efficiency with a single stream, and does not exhaust system resources or flood the network. Unlike simplistic UDP data blasters, fasp has ideal bandwidth efficiency, and does not flood the network with redundant data, or deny access to standard TCP applications.
A novel file streamlining technique in fasp helps it achieves the same ideal efficiency for transfers of large numbers of small files. For example, one thousand 2MB files can be transmitted from the US to New Zealand with an effective transfer speed of 155 Mbps, filling an entire OC-3.
AUTOMATIC ADAPTIVE RATE CONTROL
Whereas TCP self-limits its transfer rate even on an uncongested link, fasp™ detects the unused bandwidth and ramps up to fill it (Figure 1). When congestion builds up, fasp™ reduces its rate to accommodate TCP traffic and equally shares the link with multiple TCP flows (Figure 2). This approach has fundamental benefits over the flow control algorithm used by standard TCP or the “accelerated” or "high-speed" TCP's:
- Loss tolerant. Reacts only to true congestion, while remaining immune to inherent channel loss.
- TCP fair. Quickly stabilizes at a TCP-friendly rate when links are congested without squeezing other traffic.
- Perfectly efficient. Ramps up to fill unclaimed bandwidth, regardless of latency and packet loss.
- Stable. Zeroes in on the available bandwidth, and runs "flat", without oscillation.
The fasp™ protocol provides complete built-in security without compromising transfer speed. The security model, based solely on open standards cryptography, consists of SSH authentication, on-the-fly data encryption using strong cryptography (AES-128) for privacy of the transferred data, and an integrity verification per data block, to safeguard against man-in-the-middle and anonymous UDP attacks. The transfer preserves the native file system access control attributes between all supported operating systems, and is highly efficient: With encryption enabled, fasp™ achieves WAN file transfers of 100-200 Mbps on a single processor machine or laptop; and 200-500 Mbps+ on dual-processor or duo-core workstations.
The fasp™ transfer technology serves as the core of Aspera’s suite of software products for global exchange of files of any size, over any distances and network conditions at maximum speeds. It can also be embedded into any desktop, web or mobile application via Aspera’s comprehensive set of SDKs and APIs. It provides a number of key benefits that make it superior to competing transfer technologies.
MAXIMUM SPEED & RELIABILITY
Leveraging existing WAN infrastructure and commodity hardware, fasp achieves speeds that are hundreds of times faster than FTP and HTTP. Data set of unlimited sizes can be transferred over any network at maximum speed, regardless of network conditions or distance. Large sets of small files achieve the same transfer efficiency as large single files. Without the erratic, unpredictable behavior of TCP, fasp delivers guaranteed transfer times and 100% reliability.
fasp also enables perfect progressive-style transfers, such as streaming media. Transfer speeds do not degrade with congestion or distance, ensuring smooth, immediate processing of the incoming data.
EXTRAORDINARY BANDWIDTH CONTROL
The bandwidth utilization algorithm in fasp enables fast, automatic discovery of the bandwidth capacity and its full utilization while remaining fair to other traffic. fasp supports on-the-fly configurable bandwidth sharing policies so that users may pre-set and change individual transfer rates and finish times.
Using open standards cryptography for user authentication, data encryption and data integrity verification, fasp assures complete security of business-critical digital assets.
fasp is a software-only application protocol that runs on commodity hardware and over standard, unmodified IP. It runs on a variety of operating systems (server, desktop, mobile, and cloud platforms) and requires no changes to the operating system or driver installation on the file transfer endpoints, no new appliances, and no network changes.
fasp provides end-to-end transfer progress reporting and detailed performance statistics for real-time monitoring and reporting, and for custom pre- and post-transfer processing. It automatically resumes partial transfers and retries failed transfers, ensuring 100% reliability.
FLEXIBLE & OPEN ARCHITECTURE
fasp supports interoperable file and directory transfers between all major operating systems and cloud platforms, and provides a complete, modern software API.
Customer Success Stories
“Aspera’s technology not only provides significantly faster transmission speeds, which reduce the margin for error but also includes comprehensive security and data verification schemes to ensure that the possibility of corruption is reduced significantly. Plus, adaptive bandwidth control allows concurrent transfers to run with high efficiency, enabling us to meet strict deadlines.”Jonathan Humphrey, IT Director at Codemasters