The explosive growth of unstructured data across virtually all industries and the speed with which enterprises need to access that data have created a “big data” challenge that cannot be solved with traditional network and storage technologies. The emergence of faster wide-area networks (WANs), faster CPUs, and larger storage arrays has further highlighted the problem - reliable movement and storage of massive bulk data over global distances cannot be achieved with solutions that are based on traditional software architectures and technologies with inherent bottlenecks in performance and scalability.
Transfer tools that use traditional network protocols fail to utilize the available bandwidth, while traditional server and storage operating systems experience performance bottlenecks from CPUs, disk IO, and file system overhead. To meet the extreme business requirements of big data access, transfer, and storage, solutions must overcome the limiting factors associated with conventional network protocols, servers, and storage systems.
Solution: Aspera Enterprise and Connect Server for Isilon OneFS
Aspera’s high-speed transfer software shipping with Isilon OneFS operating system and Isilon scale-out NAS delivers an integrated solution uniquely enabled to transfer and store file assets of any size, with maximum speed, over any IP network, with complete bandwidth control and security. This cost-effective and breakthrough solution delivers extreme performance and reliability of scale-out NAS across high- latency and low bandwidth networks.
Transfers occur at maximum speed, fully utilizing the allocated bandwidth while being fair to other critical network traffic. Transfers are fast, stable, robust and predictable, even for the largest files, most challenging networks, longest distances, and highly concurrent and loaded transfer scenarios.
Single Integrated System
Aspera Enterprise and Connect server are bundled and ship with Isilon storage. The Isilon server and storage tier are collapsed into a single, common storage and transfer management pool, simplifying the management of multiple fasp™ processes running on multiple cluster nodes. As file transfer needs expand, simply add performance or capacity to the Isilon cluster, without having to manage servers and storage separately and with no downtime.
Easily Scale-Out Performance
The joint solution enables linear scale-out transfer performance from lower bandwidth rates up to 10 Gbps cluster to cluster for both single and concurrent transfers. Aspera fasp™ transfers can be easily scaled out across nodes in the Isilon cluster to achieve maximum aggregate throughput end-to-end, with full predictability - transfer times can be calculated with certainty based on data set size and bandwidth rates. As bandwidth increases to higher speeds, simply add nodes to create additional performance. Performance and capacity can scale independently by adding storage-only nodes, or performance accelerator nodes.
Complete Control Over Performance
Aspera’s unique adaptive rate control virtualizes performance management cluster-to-cluster. A single virtual link spanning multiple nodes allocates bandwidth and priorities to transfers. Combined with Isilon scale-out performance, this capability provides unmatched transfer management and bandwidth prioritization—enabling precision fairness to other network traffic. Bandwidth priorities can be assigned to individual transfers, down to the user, nodes, cluster, DNS name or IP address.
All-Active High Availability and Load Balancing
Isilon SmartConnect provides intelligent client connection load balancing and failover support for Aspera fasp™ clients. Through a single host name, SmartConnect enables client connection load balancing and dynamic Aspera fasp™ failover of client connections across storage nodes to provide optimal utilization of the cluster resources. SmartConnect is compatible with any Aspera fasp™ client or server.
No single point of failure
If one node becomes unavailable, the transfer resumes on the remaining nodes. By using the Isilon SmartConnect application software zones, transfer connections can failover to other nodes and resume transfer from the point of interruption.
Single point of management
With this solution, it is as simple to manage one Aspera Enterprise or Connect Server as it is to manage ten servers since they are all hosted on the same cluster. All Aspera Enterprise and Connect Servers share the same configuration: user authentication, transfer rate control, ports, security, destination name, and other settings.
Web Services and Industry-standard API support
Aspera software supports a fully extensible set of web services and common SOAP APIs to provide easy integration with 3rd-party data management tools and web-based management applications.
The solution works with any IP network, and can be configured to take advantage of high-speed networks and endpoints that use large packet sizes by increasing the MTU setting.
TRANSFERRING FILES FROM NON-ISILON CLIENTS TO AN ISILON CLUSTER
In this basic use case, an Aspera Client connects to an Aspera server, hosted on a remote Isilon cluster, and initiates a transfer. The link can be any IP network link such as a DSL line, satellite link, or another LAN in a remote branch. In addition, the transfer rate can be monitored by the Client and modified on the fly by adjusting the minimum and maximum rate control bar. Different transfers can then be run using different transfer rates or rate control modes, based on priority and quality of service requirements. Data transfers can also be configured to adjust the transfer rate automatically if other network activity is detected on the network with maximum and minimum limits applied.
WEB-BASED FILE TRANSFERS
When using the Aspera Connect Server on the Isilon cluster, the server directory listings can be exposed via a simple, customizable web portal. Once web transfers are enabled on the cluster, remote users simply point their browsers to the Isilon cluster and transfer files over the web with the Aspera Connect high-performance browser plug-in. The small footprint plug-in is compatible with most standard browsers, installs on demand, and powers high-speed uploads and downloads with the Aspera Connect Server. The Aspera Connect Server web application can be customized to provide specific workflows and a custom look and feel.
DATA TRANSFER FROM REMOTE FILERS TO AN ISILON CLUSTER
For filers (file servers) that cannot host Aspera applications, a client machine can be connected to the local filer using any standard file sharing protocol such as a CIFS share, or an NFS mount, and separately connect to a remote Isilon cluster using Aspera Client. Using this scheme, a transfer can be initiated from the local filer to the remote Isilon cluster through the Aspera Client.
In other cases, file servers such as Windows Storage Server or Mac Xserver can host the Aspera Client, in which case transfers can take place directly from the file server to the Isilon cluster. To scale performance, multiple client machines with Aspera Client can create similar transfer connections working concurrently to transfer different sub-directories of the filer. All clients can use the same Isilon cluster SmartConnect zone. The SmartConnect software module on the Isilon cluster will balance the Client connections across all Isilon cluster nodes in that zone, each connection contributing linearly to the overall transfer throughput.
PARALLELIZED DATA TRANSFERS BETWEEN TWO REMOTE ISILON CLUSTERS
When data is transferred between two Isilon clusters, identical transfer streams can be launched from each node on the source cluster working together on the same dataset. In this case, the dataset on the source cluster does not need to be manually carved out between the transfer streams. Individual large files are broken off into smaller chunks and sent across multiple streams. This enables sending in parallel multi-gigabyte and multi-terabyte files as well as large datasets of small files.
AUTOMATED TRANSFERS USING CLIENT ‘HOT FOLDERS’
Whether pushing content from a central location to remote locations, or pulling fresh content from the central location, Aspera Client provides a mechanism to automate transfers using the ‘hot folders’ feature. This feature allows a user to designate certain folders to be monitored for changes. Any new or changed file data in these folders is automatically transferred to the predefined remote location. This feature can be used for pushing content from local folders or pulling content from remote folders.
<Insert some deployment diagrams>