Aspera Transfer CLUSTER Manager

The Aspera Transfer Cluster Manager (ATCM) provides Autoscale capabilities that allow the Aspera transfer nodes to self-scale in today's dynamic cloud environments. The technology is used by the Aspera Transfer Service and is available for customers using Aspera On Demand. The core capabilities are an autoscale cluster manager service for elastic autoscaling of transfer hosts and client load balancing and automatic cluster configuration, the new node/v4 multi-tenant secure access key system, and a new cluster management UI for managing policies and reporting on transfers across the cluster. ATCM supports Aspera On Demand clusters on SoftLayer, AWS and Azure.

The Autoscale functionality is complemented by ScaleKV, the scale out data store for distributed, high throughput collection, aggregation and reporting of all transfer performance and file details across the Autoscale cluster.

The service provides automatic start/stop of transfer server instances, automatic balancing of client requests across available instances, monitoring and restart of all critical services on the server instances, and configurable service levels to manage maximum transfer load per instance, available idle instances for "burst", and automatic decommissioning of unused instances.

USE CASES

FAST TRANSFERS OF VERY LARGE FILES AND DATA SETS TO/FROM THE CLOUD

Moving huge files and data sets or massive file repositories globally to and from your cloud infrastructure can be very challenging. Traditional transfer software technologies are slow and unreliable, and shipping physical disk storage is time consuming, expensive and exposes your data to unnecessary security risks. Using the web interface, the Aspera Transfer Cluster Manager enables you to set user-defined policies that automatically manage the provisioning and deprovisioning of servers in a cluster to quickly accommodate large spikes and drops in transfer loads.

Features & Benefits

OPTIMIZED TO MEET SLAS AND CUT COSTS   

The ATCM manages transfer throughput SLAs and compute/bandwidth costs with elastic scaling. The service is part of the Aspera transfer server software stack, and automatically manages the number of server instances needed to support client transfer demands based on user-defined policies and automatically manages the number of nodes in use and booted up in reserve but idle.

HIGH AVAILABILITY AND LOAD BALANCING

As transfer loads increase and decrease, nodes are moved from idle to available for client requests, and from available to highly utilized and back again based on user-defined load metrics such as tolerances for low and high transfer throughput and online burst capacity. If the minimum number of available nodes drops below the user-defined threshold, the cluster manager boots up new nodes automatically, and then brings them back down when they are no longer needed. Similarly, unavailable/down nodes are automatically detected and restarted, and client requests are re-pointed to healthy nodes.

COMPATIBLE ON ALL MAJOR CLOUDS

Works in conjunction with Aspera Direct-to-Cloud storage - infrastructure independent. All of the Autoscale capabilities are implemented in the Aspera software and thus are portable across cloud providers. Includes support for AWS, SoftLayer, and Azure autoscale clusters.

SCALEABLE, ROBUST TRANSFER MANAGEMENT & REPORTING

All transfer initiation and management scales across the cluster with ScaleKV technology, a new Aspera created scale out data store for distributed, high throughput collection, aggregation and reporting of all transfer performance and file details across the autoscale cluster, and transfer initiation across the cluster.

CLUSTER API CAPABILITIES

Key API capabilities provided by the Cluster:

  • Query transfer status across the cluster - The new transfer API (node/v4) returns transfer progress and details across the entire cluster. Transfer statistics are automatically sharded across the memory of all nodes in the cluster, keeping up with very high transfer rates and number of sessions.
  • Transfer using all nodes in the Cluster - The new transfer API (node/v4) allows third party applications to initiate transfers that use multiple / all nodes in the cluster for throughput beyond the capability of a single node, and automatic failover and fault tolerance.
  • Securely Support Multiple Tenants - New mulit-tenant secure access key system (node/v4) allows Aspera administrators of applications such as Files, faspex and Shares to securely support multiple tenants on the same  cluster with private access to separate cloud storage and transfer reporting.
  • Report transfer status and usage by Tenant - The transfer API (node/v4) allows queries of transfer history by access key to securely report history and usage by access key tenant. (Aspera On Demand usage reporting is being enhanced with usage by access key.)

CLUSTER MANAGEMENT UI

A New web-based cluster management UI manages access keys, cluster configuration including Autoscale policy and in memory data store for transfer statistics.

Key features include:

  • Configuring auto-scale policies and transfer node (aspera.conf) templates and automatically implementing these on new and running clusters
  • Configuring and installing SSL certificates for Aspera nodes
  • Enhanced service monitoring and management to monitor and restart key Aspera services and automatically decommission failed nodes
  • Live cluster upgrade allowing a cluster manager to migrate a live running cluster to a new machine image version without disrupting running transfers.
  • Enhanced resilience to failure including ability to backup and restore the cluster state, options to back the main state store on a clustered database (AWS RDS, Azure ClearDB), and in failure recovery, the ability to launch new node images with configuration from the backup.
  • Easy duplication of cluster template configurations through the UI
  • Create access keys for all supported Aspera transfer cluster platforms (AWS, Softlayer, Azure in 3.7)
  • Health monitoring and restart of all critical Aspera services on individual instances. 
  • Updating of aspera.conf file across all nodes in the cluster.
  • View all active and completed transfers with filters by access key

Purchase & Deployment

The Aspera Transfer Cluster Manager is available as licensed software with pricing based on a committed amount of data transferred over a monthly, annual or multi-annual term. The ATCM is packaged on a virtual machine and can provision clusters of Aspera servers leveraging a deployment template. The clusters are deployed in a virtual private cloud (VPC).

 LICENSING AND Pricing

 Transfer volume-based subscription pricing is based on:

  • Data volume tiers 
  • Monthly, annual and multi-year terms available
  • Additional fees for optional add-ons

ATCM COMPONENTS

The standard ATCM offering includes the following components:

  • A desktop and command line client for use with the service
  • Pre-packaged virtual machine images built for the cloud platform or your choosing
  • ATCM web-based UI 
  • Can be enabled to support your separate Aspera Files, Faspex and Shares applications

SUPPORTED INFRASTRUCTURE

  • SoftLayer 
  • AWS
  • Azure & Google (coming soon)

Cloud Requirements

  • Amazon
    • S3 Object Storage
    • EC2 Server with an Amazon Machine Image (AMI) from Aspera for both the Aspera Transfer Node and Cluster Manager
    • Security Group, VPC
  • SoftLayer
    • Swift Object Storage
    • Server instance with CCI Image IDs from Aspera for both the Aspera Transfer Node Image ID and Cluster Manager Image ID
    • DNS service with an assigned Domain to the Route 53 service

Click here for more ATCM requirements and support detail on the Aspera support site. Download information is available here.

RELATED PRODUCTS

ASPERA FILES

A full-featured Software as a Service offering for sharing large files and data sets directly from cloud and on premises storage – located anywhere, to anywhere, with anyone.

ASPERA SHARES

An easy to use web application for sharing files and directories of any size with employees across your organization or with external customers and partners.

ASPERA FASPEX

Global person-to-person file delivery and collaboration platform for file-based collection, distribution, and collaboration among geographically dispersed teams. 

ASPERA CONSOLE

Web application for transfer management, monitoring and control across your entire Aspera network.

Aspera eNTERPRISE SERVER

A versatile server application which enables high-speed movement of files across global enterprises, high-volume content ingest and distribution, and replaces FTP/SFTP servers for transfers of large, business-critical data.

ASPERA TRANSFER SERVICE

A hosted multi-tenant software service running in the Cloud that enables high-speed upload and download of large files and large data sets directly into Cloud object storage.

Use Cases

Performance calculator

“Aspera offers a mature, fast, reliable and secure solution capable of handling the high volume of video content we need to deliver directly to cloud storage."  Simon Christie Broadcast Systems Engineer, Distribution & Broadcast Technology, Channel 4