July 17, 2024
Introduction
The commercial Aspera data transfer protocol offers high speed data transfer over large distances. Free alternatives for moving large amounts of data quickly over the Internet include Fast Data Transfer and UDR.
When to use Aspera?
If you transfer large volumes of data to or from institutions that offer the Aspera protocol, for example the Broad Institute and the NCBI. Contact Scientific Computing with specific inquiries regarding large-scale data transfer.
The Aspera Connect Client
This client application runs on Windows, Mac OS X and Linux, and is able to send and receive data to remote Aspera Connect Servers like those at the Broad and NCBI.
Installation
From a web-browser
Visit http://downloads.asperasoft.com/connect2/ and follow the installation instructions on that page
From the command line on Linux
Obtain a direct download link to the v3.3.3 (or newer) Linux x86_64 plugin from this page, using a web-browser on your laptop/workstation. The direct download link will look like this with a different random key string: http://d3gcli72yxqn2z.cloudfront.net/connect/bin/aspera-connect-3.3.3.81344-linux-64.tar.gz
Note: To copy the link, use the R mouse button (not L) and select "Copy Shortcut".
On the Linux server, download the installation file, replacing the URL with the one you obtained earlier:
wget http://d3gcli72yxqn2z.cloudfront.net/connect/bin/aspera-connect-3.3.3.81344-linux-64.tar.gzUnpack the installation file, and run the installer, replacing the version number below to match the file downloaded
tar xvfz aspera-connect-3.3.3.81344-linux-64.tar.gz./aspera-connect-3.3.3.81344-linux-64.shTesting
From a web-browser
See the Aspera Documentation
From the Command Line
See also the Aspera ascp documentation
Test upload and download using a connection to the Aspera demo server, using password: demoaspera
env ASPERA_SCP_PASS=demoaspera ~/.aspera/connect/bin/ascp asperaweb@demo.asperasoft.com:aspera-test-dir-large/100MB 100MB
Password:
100MB 100% 100MB 43.7Mb/s 00:20
Completed: 102400K bytes transferred in 21 seconds
(38923K bits/sec), in 1 file.
env ASPERA_SCP_PASS=demoaspera ~/.aspera/connect/bin/ascp 100MB asperaweb@demo.asperasoft.com:Upload/
Password:
100MB 100% 100MB 37.2Mb/s 00:22
Completed: 102400K bytes transferred in 23 seconds
Use with the Scientific Computing (SciC) Linux Clusters
Aspera is installed on ErisTwo and it is available with the command
module load acsp
You will need to provide path to a private key file, which is located at
/apps/lib-osver/aspera/connect/etc/aspera_tokenauth_id_rsa
Regular data transfer jobs using "ascp" should be submitted with SLURM to the "filemove" queue
NCBI Documentation
http://www.ncbi.nlm.nih.gov/books/NBK47527/
Notes
To use the graphical interface (GUI) for the Aspera Connect Client a web-browser is required, however the "ascp" tool can be installed and used from Linux servers at the command line with no graphical interface or web-browser installed
For access to a web-browser in the SciC Linux Clusters environment, see the FAQ SciC Linux Clusters Virtual Desktop
The Aspera Connect Client does not require administrative level access to a computer for installation, each user installs their own copy of the software.
The Aspera Connect Client is only compatible with Aspera Connect servers, a different client is required for Aspera Enterprise servers.