The emergence of cloud computing and web-based applications such as video hosting, scientific collaboration and online storage, combined with the explosive growth in the size of the data being moved around the world (think gigabytes of scientific and enterprise data or terabytes of video content) has resulted in the emergence of lower cost 10 Gbps network infrastructure. As organizations deploy 10 Gbps network technologies over campus- and wide-area networks (WANs), they are turning to Aspera fasp™ as the underlying data transport.
A major inhibitor to the adoption of 10 Gbps WANs is the TCP’s inability to take advantage of the extra bandwidth, leading to inefficient utilization, poor performance (relative to bandwidth), and lack of scalability. Conventional TCP-based data replication and file transfer tools do not scale or perform reliably on any IP network over anything more than trivial distances.
Less known is the fact that more bandwidth does not solve the problem. Conversely, on higher speed networks more bandwidth goes unused or is wasted by transferring redundant data through brute force approaches such as multi-stream TCP or UDP data blasting. In the best-case, low-latency and low-loss campus scenarios, the scalability bottleneck shifts from the network to the endpoint (i.e. servers and storage systems).
Aspera has developed a scalable, location-agnostic, storage-integrated file transfer and replication solution for 10 Gpbs WANs based upon its high-speed fasp™ transport protocol. In partnership with industry-leading storage system vendors, it offers solutions that break the storage-to-storage data transfer bottlenecks using any 10 Gbps IP WAN.
In addition to optimal performance over the network, fasp™ also provides a highly adaptive and scalable storage rate control optimized to maximize disk I/O and end-to-end throughput for large data transfers between remote sites with DAS, NAS, and SAN. This means that customers can fully utilize their deployed storage, of any type, and scale up and out for transfer performance over any distance and network. Using modern storage systems, Aspera fasp™ delivers ultra-high-speed throughput over any distance.
Global organizations now have the opportunity to move large data sets long distances over multi-gigabit WANs in lieu of digital shipping, but to realize the promise, transfer performance must scale with available bandwidth. Aspera faspTM has no theoretical limit on its speed, and has been engineered to take advantage of multi-gigabit pipes to their fullest capacity. As a software-only solution, it takes advantage of commodity hardware to deliver transfer speeds that exceed those achieved by purpose-built hardware appliances.
Digital cinema frameset transfers
Modern digital cinema 2K or 4K feature films consist of individual frame files (up to a million), each encoded as a 12 MB (2K) or 50 MB (4K) file. Aspera provides an ultra simple, ultra fast, software only application called faspframes™, designed for 10 Gbps global transfers of such framesets using fasp™ as a network transport protocol and optimized storage I/O.
In tests, faspframes™ ran over a 10 Gbps network at 8 Gbps with 200 millisecond round-trip times and 2% packet loss. The result: 60 minutes of footage (1 TB) transferred globally in under 20 minutes – more than twice as fast as results published by purpose-built appliance vendors, on WANs with modest latency and packet loss.
Global scientific collaboration
Modern genome research applications generate terabytes of sequencing data. With research collaboration going global, research centers have had to resort to shipping hard drives to customers and partners around the world. This method has proven expensive, prone to human errors and overly time consuming.
Aspera’s fasp addresses the shortcomings of the above approach, providing superior transfer speeds and security, enabling researchers around the globe to collaborate more efficiently. In pilot studies at BGI, the world’s largest genomics research institute, Aspera’s patented fasp transport dramatically improved delivery time, decreasing cost for partners and removing the need for shipping hard drives for the delivery of sequencing data. Easy to integrate with on-premise or cloud-based systems, fasp has become the gold standard for research centers around the world.
Enterprise data analysis
More than ever before, global enterprises are relying on geographically dispersed workforce to gather and analyze information. Oftentimes, massive amounts of digital content reside in data centers scattered around the world. In order to implement effective follow-the-sun workflows, businesses are faced with moving and synchronizing terabytes of data. Having invested in multi-Gbps intercontinental links, they find their performance underwhelming when using traditional transfer tools like FTP or rsync.
With Aspera fasp running over high capacity networks, businesses can get the most out of their IT infrastructure investments, cut data transfer times from days to hours, eliminate shipping costs, and improve reliability and data security.
The limiting factor in transfer speed over most WANs — from low-bandwidth up to 1 Gbps — is usually the transport protocol on the network (such as TCP), which typically underperforms significantly on high-loss, high-latency network. In some cases, the servers that initiate transfers can also be a limiting factor and the transfer speed can thus be bound by CPU, RAM, and/or Disk I/O on server hosts. At speeds over 1 Gbps, the bottleneck may further shift from the server host to the storage disk controller or NAS head, depending on the storage system and configuration.
Aspera has teamed up with industry-leading storage companies to enable ultra-fast transfers on multi-Gbps WANs and has provided joint file transfer and data replication solutions.
Test results from typical scenarios using popular NAS and SAN storage from Aspera’s storage partners are documented in our white paper, "Ultra High-speed Transport: The Future of Wide Area Data Movement." A brief summary of the tests and the results is presented below.
The results summarized here are intended to demonstrate the capability for breakthrough transfer speeds and the best practices for achieving these with each storage system, and should not be used to conclude any specific performance comparisons among the storage system architectures.
Throughput versus latency and packet loss over a 10 Gbps WAN
fasp™ as compared to TCP-based transfer tools
As the test results demonstrate, given proper infrastructure, fasp™ is the only protocol that is capable of moving a petabyte data on a daily basis over commodity global IP networks (1PB/24hr=104.2 Gbps which requires approximate 50 fasp™ transfers at 2.1 Gbps globally).
The complete results are documented in the white paper, "Ultra High-speed Transport: The Future of Wide Area Data Movement. "