Accessing Fileservers (network shares) from the ERISOne Linux Cluster

Introduction

For performance and reliability we recommend copying all files to the HPC cluster prior to analysis. This article addresses copying files to and from other network file shares directly, a related article covers ways to transfer data to the ERISOne cluster from your workstation or laptop.

Windows (SMB/CIFS) Shares

SFA, RFA, MAD3 and IDEAFS storage

These network file servers are available on the remote desktop, login nodes and filemove nodes only - they cannot be accessed directly from compute nodes.  Please use the "filemove" nodes for any large or long-running file transfers between the ERISOne cluster and other network file servers, since heavy network traffic may cause the login nodes to be slow for other researchers.. Using the filemove nodes avoids slowing down the login nodes.  Smaller file transfers and browsing remote network shares can be done on the login nodes and remote desktop nodes directly.

Obtain an authentication token

  • First get a session on one of the filemove nodes, by typing in an ERISOne command line terminal:
bsub -Is -q filemove bash 
  • Obtain an authentication token. This command will prompt you to enter your Partners password
kinit
  • Check the expiry time of the authentication token
klist
  • TIP: if you login to the cluster with your Partners password over SSH, an authentication token is created automatically
  • TIP: for small, short-running tasks, ssh directly to the filemove nodes with "ssh <username>@erisonexf.partners.org"

Navigate to the network storage server location

Service  Description  Path on the ERISOne cluster filemove nodes
 SFA  Partners Shared File Area storage  /external/SFA
 RFA  ERIS Research Interactive storage  /external/RFA
 MAD3 §  ERIS MAD3 archive server  /partners.org/MAD3
IDEAFS  Storage for the IDEA analytics platform  /external/IDEAFS-GROUP and /external/IDEAFS-USER
MAD § ¤ Original MAD archive servers  /partners.org/mad-store1 and /partners.org/mad-replicated1

§ If you get an error message "cannot access /external/MAD3" when connecting to the MAD server, please wait 5 minutes and try again

¤ Many accounts on the original MAD servers did not use Partners passwords. This method will not work in that case

cd /external/SFA
  • Now the "ls" command will list all available network shares. Change directory to your share
ls 
  PHS-ABC   PHS-DEF
  • TIP: Once you know the direct path to your network share, it is faster to go directly to that location.
  • TIP: Create a symbolic link to the share from your home folder

Other storage servers using smbclient command-line tool

The command line program "smbclient" is like a command line "ftp" client and allows copying files between the remote file share and the cluster.  Note that this program uses forward slash "/"

  • Example 1, using a Partners DFA:
smbclient -U PARTNERS/username //dfa6.partners.org/prcdfa$

Domain=[PARTNERS] OS=[Windows Server 2003 R2 3790 Service Pack 2] Server=[Windows Server 2003 R2 5.2]

smb: \> help

  • Example 2: Copying files to/from HPCWIN
smbclient -U PARTNERS/username //hpcwin.partners.org/username
  • Example 3: Non-partners authentication on the MAD Research Archive store:
smbclient -U madusername //mad.research.partners.org/madusername

Graphically from an ERISOne HPC Virtual Desktop server

Use the file browser mount a remote Windows share to transfer files inside an ERISOne Remote Desktop:

  • Select the Places menu in the top-left of the screen
  • Select Connect to Server
  • Service type: Windows share
    • Server: the name of the network share server
    • Tip: Include the full DNS name of the server, e.g. SFA9.PARTNERS.ORG, instead of the short name (e.g. SFA9)
    • Share: the name of the network share on this server
    • Folder: leave this blank
    • User Name: your username for the network share
    • Domain Name: the domain your username belongs to, if any.  Set this to PARTNERS if using your Partners ID

Once mounted, the share is available in the file browser, inside applications that are part of the "Gnome desktop" and via command line under "${HOME}/.gvfs"

Note that many applications in the remote desktop are not part of the Gnome Desktop suite, and will not show the mounted network share in their file browser windows. To access the mounted share in these applications it is recommended to create a link (shortcut) to the mount location. This example creates a shortcut to a mounted network share "ERIS" on server name "rfanfs.partners.org".  Not that when mounted, the share name is all lowercase.  In the example, the ERISOne username is abc123

  • Find the file path to your mounted network share. Nothing will appear if no shares are currently mounted. Type the text that appears in bold in a Terminal window
[abc123@rgs00 ~]$ ls -d $HOME/.gvfs/* /PHShome/abc123/.gvfs/eris on rfanfs.partners.org
  • Change folder to your home directory and create a link to the path provided by the previous command, using " " to protect the spaces in the path name.  Replace MY_NETWORK_SHARE with a name for your share.  The ls command tests that it works
[abc123@rgs00 ~]$ ln -s "/PHShome/abc123/.gvfs/eris on rfanfs.partners.org" MY_NETWORK_SHARE [abc123@rgs00 ~]$ ls MY_NETWORK_SHARE ShareFolder1 ShareFolder2
  •  Now you will be able to access this folder in any application via the folder MY_NETWORK_SHARE in your home folder. Creating the link only needs to be done once, but you will need to connect to the share each time you open a new remote desktop, or if the the share connection is idle for an extended period and expires

 

Request an entry added to the ERISOne automount

If access to the remote files is frequent or you would like to submit batch jobs to transfer files to and from your remote storage, request an automount entry on the "filemove" nodes. This method is not suitable for shares that are accessed via your personal Partners usernames/password. A local account or service account must be used to access the remote file share. The fileshare will appear under the path "/external" on a group of nodes dedicated to file transfer; you can use the "filemove" cluster queue or an interactive session or remote desktop to move files from remote storage to the cluster and back.  It is not possible to run batch computational jobs on files that are stored outside the cluster.

NFS Shares

Request an entry added to the ERISOne automount

If the remote fileshare uses UID's from Partners LDAP authentication, and you are satisfied with the security provided by the NFSv3 protocol for your data, request an automount entry on the "filemove" nodes. The fileshare will appear under the path "/external" on a group of nodes dedicated to file transfer; you can use the "filemove" cluster queue or an interactive session or remote desktop to move files from remote storage to the cluster and back.  It is not possible to run batch computational jobs on files that are stored outside the cluster.

Files on an SSH enabled Server

sshfs at the command line

The 'sshfs' command can be used from ERISOne HPC Virtual Desktop servers to mount folders on remote SSH servers

Graphically from an ERISOne Virtual Desktop server

Use the file browser mount a remote folder on an SSH server

  • Select 'Places' from the Menu in the top-left of the screen
  • Select 'Connect to Server'
  • Service type: SSH
  • Enter details of share

Once mounted, the share is available in the file browser and via command line under "${HOME}/.gvfs" - see earlier notes in the Windows Share section on accessing this from graphical applications


Related articles