November 21, 2023
Introduction
For performance and reliability we recommend copying all files to the HPC cluster prior to analysis. This article addresses copying files to and from other network file shares directly, a related article covers ways to transfer data to the ERISOne cluster from your workstation or laptop.
Windows (SMB/CIFS) Shares
SFA, RFA, MAD3 and IDEAFS storage
These network file servers are available on the remote desktop, login nodes and filemove nodes only - they cannot be accessed directly from compute nodes. Please use the "filemove" nodes for any large or long-running file transfers between the ERISOne cluster and other network file servers, since heavy network traffic may cause the login nodes to be slow for other researchers.. Using the filemove nodes avoids slowing down the login nodes. Smaller file transfers and browsing remote network shares can be done on the login nodes and remote desktop nodes directly.
Obtain an authentication token
- First get a session on one of the filemove nodes, by typing in an ERISOne command line terminal:
bsub -Is -q filemove bash
- Obtain an authentication token. This command will prompt you to enter your Mass General Brigham password
kinit
- Check the expiry time of the authentication token
klist
- TIP: if you login to the cluster with your Mass General Brigham password over SSH, an authentication token is created automatically
- TIP: for small, short-running tasks, ssh directly to the filemove nodes with "ssh <username>@erisonexf.partners.org"
Navigate to the network storage server location
Service | Description | Path on the ERISOne cluster filemove nodes |
SFA | Mass General Brigham Shared File Area storage | /external/SFA |
RFA | ERIS Research Interactive storage | /external/RFA |
MAD3 § | ERIS MAD3 archive server | /external |
IDEAFS | Storage for the IDEA analytics platform | /external/IDEAFS-GROUP and /external/IDEAFS-USER |
MAD § ¤ | Original MAD archive servers | /partners.org/mad-store1 and /partners.org/mad-replicated1 |
§ If you get an error message "cannot access /external/MAD3" when connecting to the MAD server, please wait 5 minutes and try again
¤ Many accounts on the original MAD servers did not use Partners passwords. This method will not work in that case
cd /external/SFA
- Now the "ls" command will list all available network shares. Change directory to your share
ls
PHS-ABC PHS-DEF
- TIP: Once you know the direct path to your network share, it is faster to go directly to that location.
- TIP: Create a symbolic link to the share from your home folder
Other storage servers using smbclient command-line tool
The command line program "smbclient" is like a command line "ftp" client and allows copying files between the remote file share and the cluster. Note that this program uses forward slash "/"
- Example 1, using a Mass General Brigham DFA:
smbclient -U PARTNERS/username //dfa6.partners.org/prcdfa$
Domain=[PARTNERS] OS=[Windows Server 2003 R2 3790 Service Pack 2] Server=[Windows Server 2003 R2 5.2]
smb: \> help
- Example 2: Copying files to/from HPCWIN
smbclient -U PARTNERS/username //hpcwin.partners.org/username
- Example 3: Non-partners authentication on the MAD Research Archive store:
smbclient -U madusername //mad3.partners.org/madusername
Graphically from an ERISOne HPC Virtual Desktop server
Use the file browser mount a remote Windows share to transfer files inside an ERISOne Remote Desktop:
- Select the Places menu in the top-left of the screen
- Select Connect to Server
- Service type: Windows share
- Server: the name of the network share server
- Tip: Include the full DNS name of the server, e.g. SFA9.PARTNERS.ORG, instead of the short name (e.g. SFA9)
- Share: the name of the network share on this server
- Folder: leave this blank
- User Name: your username for the network share
- Domain Name: the domain your username belongs to, if any. Set this to PARTNERS if using your Partners ID
Once mounted, the share is available in the file browser, inside applications that are part of the "Gnome desktop" and via command line under "${HOME}/.gvfs"
Note that many applications in the remote desktop are not part of the Gnome Desktop suite, and will not show the mounted network share in their file browser windows. To access the mounted share in these applications it is recommended to create a link (shortcut) to the mount location. This example creates a shortcut to a mounted network share "ERIS" on server name "rfanfs.partners.org". Not that when mounted, the share name is all lowercase. In the example, the ERISOne username is abc123
- Find the file path to your mounted network share. Nothing will appear if no shares are currently mounted. Type the text that appears in bold in a Terminal window
[abc123@rgs00 ~]$ ls -d $HOME/.gvfs/* /PHShome/abc123/.gvfs/eris on rfanfs.partners.org
- Change folder to your home directory and create a link to the path provided by the previous command, using " " to protect the spaces in the path name. Replace MY_NETWORK_SHARE with a name for your share. The ls command tests that it works
[abc123@rgs00 ~]$ ln -s "/PHShome/abc123/.gvfs/eris on rfanfs.partners.org" MY_NETWORK_SHARE [abc123@rgs00 ~]$ ls MY_NETWORK_SHARE ShareFolder1 ShareFolder2
- Now you will be able to access this folder in any application via the folder MY_NETWORK_SHARE in your home folder. Creating the link only needs to be done once, but you will need to connect to the share each time you open a new remote desktop, or if the the share connection is idle for an extended period and expires
Request an entry added to the ERISOne automount
If access to the remote files is frequent or you would like to submit batch jobs to transfer files to and from your remote storage, request an automount entry on the "filemove" nodes. This method is not suitable for shares that are accessed via your personal Partners usernames/password. A local account or service account must be used to access the remote file share. The fileshare will appear under the path "/external" on a group of nodes dedicated to file transfer; you can use the "filemove" cluster queue or an interactive session or remote desktop to move files from remote storage to the cluster and back. It is not possible to run batch computational jobs on files that are stored outside the cluster.
NFS Shares
Request an entry added to the ERISOne automount
If the remote fileshare uses UID's from Mass General Brigham LDAP authentication, and you are satisfied with the security provided by the NFSv3 protocol for your data, request an automount entry on the "filemove" nodes. The fileshare will appear under the path "/external" on a group of nodes dedicated to file transfer; you can use the "filemove" cluster queue or an interactive session or remote desktop to move files from remote storage to the cluster and back. It is not possible to run batch computational jobs on files that are stored outside the cluster.