Accelerating the network aspect of bulk data transfer is a popular problem. Companies and research entities alike are pursuing a number of competing approaches for accelerated data transport. Most approaches fall roughly into two categories:

1)  Modified or Accelerated TCP/IP and
2)  Forward Error Correction applied to unreliable datagram (UDP) transport.

Modified/Accelerated TCP/IP
In this approach, the standard behavior of TCP that causes FTP and HTTP transfers to reduce throughput dramatically below link capacity is modified to allow TCP to maintain larger "data windows”, and in turn higher average throughputs. This may involve one or more techniques including using large TCP window sizes, selectively acknowledging missed packets, modifying TCP's back-off response to packet loss, and modifying TCP's slow startup behavior. This approach is typically applied as a transparent proxy appliance installed in a data center on either end of a WAN link, or as a non-standard network protocol stack installed on every host where accelerated transport is desired.

Forward error correction applied to unreliable data stream (UDP) transport
Forward error correction is a technique whereby raw file data is encoded with redundant information so that the original data may be reconstructed at the receiver, even if some data is lost in transit. Applying FEC to unreliable (UDP) data streams allows file data to be transmitted reliably without the time overhead of TCP acknowledgments. However, FEC has its costs. The higher the loss protection required, generally the more redundant data that must be sent. Also encoding and decoding add to the end-to-end file transfer time, and may require specialized hardware for computationally-intensive encoding/decoding operations. The most efficient FEC codes still must work block-by-block, rather than file level, and require some acknowledgment messages from receiver to sender to advance through the file.

Comparison

While both modified TCP/IP and FEC-based approaches are valuable for particular WAN transport applications, we believe they lead to poorer file transfer performance and deployment limitations, and so we have taken a different path in developing fasp file transport. Compared to the alternatives above, fasp provides:

Predictable transfer times.
Aspera's fasp file transfer precisely maintains the user-configured target rate for guaranteed file transfer times using equation-based rate control. The modified TCP approaches maintain high throughput when loss is negligible but their window-based flow control does not distinguish network congestion from channel packet loss and behaves like standard TCP under high packet loss, leading to unpredictable transfer times.

Faster file transfers.
fasp file transfers are faster than FTP over TCP/IP acceleration technologies, and file transfers using state-of-the-art FEC. fasp file transfer rates on gigabit ethernet are limited only by the disk speeds of the endpoints, and have no theoretical limit due to window-based flow control. TCP/IP acceleration approaches typically quote theoretical memory-to-memory data transfer rates over very clean networks. Over realistic networks, long haul FTP file transfers over TCP/IP acceleration top out at less than 100 Mbps. Published file transfer benchmarks for one IP acceleration appliance indicate that FTP over IP acceleration achieves 69 Mbps in a cross-country transfer (90 ms latency, loss numbers not published). fasp achieves 200-300 Mbps end-to-end file transfer rates over gigabit ethernet on average commodity PCs independent of latency and loss.

Fairness to standard TCP traffic plus full bandwidth utilization.
The window-based flow control of accelerated TCP approaches must rely on feedback from the network to determine the optimal transfer rate. The flow-control algorithm increases rate until the network is actually oversaturated, and packets are dropped by the intervening routers. This may unfairly squeeze other traffic and destabilize the network. In contrast, fasp identifies network congestion, and chooses its optimal rate under the congestion threshold, using an equation-based algorithm. fasp fully utilizes available bandwidth and is precisely fair to standard TCP under congestion.

Minimum overhead.
A 100% efficient file transport protocol sends an amount of data equal to the original file size plus precisely the amount of data lost in transit over the network, and no more. All real protocols are imperfect and send some “overhead” — redundant data that has already been received. The most efficient FEC implementations have 3-5% overhead, depending on the reliability requirements of the application. fasp has extremely low overhead, usually 0.1-1%.

Built-in transfer reporting for monitoring and billing.
fasp has complete reporting of end-to-end file transfer performance. fasp reports the forward packet loss, protocol overhead, network latency, and cumulative transfer statistics including bytes transferred to disk, transfer rate, and transfer time on both sending and receiving endpoints. Generic TCP/IP acceleration applies only to the network portion of file transfer, and does not have visibility into disk-to-disk file transfer statistics needed for application monitoring and billing.

Built-in data security.
fasp has built-in security. Data is encrypted on-the-fly, using standard block ciphers, and each received block is verified for integrity using standard one-way hash functions. Other approaches usually rely on external security systems to encrypt and verify the data.

Software only.
fasp is implemented as an application-level protocol in software. The software installs as an application on the file transfer endpoints, downloadable from Aspera, and requires no hardware appliances, new network drivers, or protocol stacks. A major break from network "acceleration" strategies, the approach is much simpler, much less expensive, and better performing. The fasp transfer applications provide 100% interoperable transfers between all major operating systems with a single transport stream, and the security model supports trusted collaboration between partners within an enterprise and between organizations, across corporate firewalls. The SDK is a rich, open interface that can be embedded in virtually any transfer workflow.