Public folders Shared to ERISOne and Partners Network

Overview

This FAQ covers two methods to share files publicly from the ERISOne Cluster. 

  • A folder named "public" in your home directory visible to all cluster users under "/pub/home/your_username"
  • A folder named "public" in your lab group data folder visible to all cluster users under "/pub/lab/your_group"
  • A folder "public/www" in your home directory, accessible over the Partners intranet via HTTPS to all Partners users

This functionality is disabled by default for data protection, see the section below on requesting a public folder.

Use Cases

  • Create a "$HOME/public" folder containing scripts and data files to share with other ERISOne users
  • Place BigWig files under "$HOME/public/www" to analyze with the ERIS UCSC Genome Browser mirror
  • Share pipeline reports, output files and graphics with colleagues who do not have cluster accounts
  • Copy scripts, log files or data to "$HOME/public" for facilitate support from HPC support or peers

Alternatives

  • For sharing large files with other cluster users, this method may cause your home folder to exceed the storage quota.  Consider using "/pub/dropbox" instead.  Note that files shared via /pub/dropbox are removed after 1 month, while files in "$HOME/public" are not.
  • For sharing files with collaborators outside Partners hospitals, contact Scientific Computing to discuss available options.
  • A more structured way to share applications or pipelines with other ERISOne users is via a lab applications folder integrated with the environment modules system on the cluster.
  • "public" folders within lab group data folders may be available in future

Accessing Public Data

To access public data shared by another user, first identify their Partners ID.  The "/pub/dropbox FAQ" includes notes on how to do this. Replace "<user_id>" in the examples below with their Partners ID.

Sharing Public Data

Requesting public folder activation

For security and data protection, ERISOne data is not exposed publicly without the knowledge of the Principal Investigator or data owner. PI's should contact Scientific Computing to request enabling public folders for

  1. Members of your lab group 
  2. Your lab group data folder.

Setting up $HOME/public to be shared via "/pub/home/<your_user_id>"

Enter the following series of commands in an ERISOne command line terminal:

cd $HOME
mkdir public
chmod go+rx public

At this point, the public folder will be accessible to other cluster users under "/pub/home/<your_user_id>".  Files copied to this folder also need permissions set appropriately for other users to access.  This step does not enable HTTPS (web) access to your files.

Setting up "$HOME/public/www"

Enter the following series of commands in an ERISOne command line terminal:

cd $HOME
mkdir public/www
chmod go+rx public/www

At this point, the public/www folder will be accessible under "https://hpcweb2.partners.org/~<your_user_id>".  Files copied to this folder need world-readable permissions or read permission for the apache group (GID 48) to be shared by HTTPS. 

Setting up "/data/<your_lab>/public" to be shared via "/pub/home/<your_lab>"

Enter the following series of commands in an ERISOne command line terminal:

cd /data/<your_lab>
mkdir public
chmod go+rx public

At this point, the public folder will be accessible under "/pub/home/<your_lab>".  Files copied to this folder also need permissions set appropriately for other users to access.  This step does not enable HTTPS (web) access to your files. 

Setting up "/data/<your_lab>/public/www"

Enter the following series of commands in an ERISOne command line terminal:

cd /data/<your_lab>
mkdir public/www
chmod go+rx public/www

At this point, the "public/www" folder will be accessible under "https://hpcweb2.partners.org/~<your_lab>".  Files copied to this folder need world-readable permissions or read permission for the apache group (GID 48) to be shared by HTTPS.

Changing access restrictions to HTTPS accessible files

By default files shared over HTTPS require a Partners ID logon outside the ERISOne HPC environment, but no authentication is needed within the cluster.

Using a ".htaccess" file you may

  • Change the access restriction so that a Partners ID is NOT required to access the public folder
    • Type this command, exactly as below, into a command terminal on ERISOne:
echo 'Allow from all' >> $HOME/public/www/.htaccess
  • Restrict access to specific Partners user IDs or groups, or IP address ranges, for data protection and security. See the Apache htaccess tutorial for implementation details.

To restrict a folder to only be accessible via HTTPS and not publicly accessible within the cluster, email the request to Scientific Computing. The folder's group will be set to "apache" and world-read permission will be removed by the cluster administrator.

Notes

  • "$HOME/public" folders are read-only to other users
  • Creating a public folder does not change access permissions to other locations in your home directory, those areas remain restricted to your user and group
  • Access to public folders via HTTPS (web) requires a Partners login, except for the ERISOne network where no additional credentials are required

Related articles