Using Singularity on the ERIS Cluster

What is Singularity

Singularity is a container platform similar to Docker in which each container comprises its own local OS customized with software that is compatible with that OS. Crucially, this OS is not required to be the same as that of the host machine running Singularity. Using Singularity containers (derived from Singularity or Docker images) running old OS versions therefore represents a very convenient approach to support legacy workflows from platforms such as ERISOne for example.

While Docker and Singularity share many key features, the key difference is that running Singularity containers does not require sudo privileges. Consequently, users can run computations using Singularity containers as they would other types of jobs. Furthermore and for convenience, users should be able to pull a suitable Docker image from one of the many Docker repositories available on the web.

 

2024/03/13 Please note this page will be extensively updated in the run up to the transition from ERISOne to ERISTwo.

 

Getting Started

Follow these steps to get the singularity module running locally on ERIS.

1. login to ERISTwo

2. Use this command to check which versions of singularity are available on the cluster

$ module avail singularity

3. Once you know which version you will be using, use this command to load the module

$ module load singularity/<version>

 

Building Singularity Images

There are many different ways a user can build a singularity image.  However all singularity build options require sudo privileges which most ERIS users will not have access to due to security policy requirements of the cluster.   Only system administrators with root privileges will be able to execute sudo singularity build commands. 

Therefore, regular ERIS users must use a remote pc/mac or a Virtual Box where they have admin/root privileges and singularity installed to build their own image.  The processes of  image "building" with singularity can be accomplished with one of the following commands (depending on where they want to host the image and if they need writable permission on the image, etc) : 

$ sudo singularity build --sandbox tmpdir/mybioapp            
$ sudo singularity build --writable container.img mybioapp
$ sudo singularity build container.img mybioapp
$ sudo singularity build container.img shub://myaccount/mybioapp
$ sudo singularity build container.img docker://mysourcedockerimage

Once the singularity image has been built the user can store that singularity image in his/her singularity hub account in the cloud and access it "pull it" with his/her ERISTwo account.  The following command illustrates the steps: 

$ singularity pull shub://myaccount/mybioapp.img

 

Converting Docker to Singularity

These steps will allow you to run a docker image as a singularity container on the ERIS Cluster.

1. Create your docker image with docker and push it to your own docker hub account in the cloud

2. Login to your ERISTwo account and pull the docker image with the following singularity command: 

$ singularity pull docker://mybioapp

3. This will create a temporary singularity image of your docker container in ERISTwo (under .singularity) and dump a copy in your current directory.  You can then run the following commands to shell into or remotely execute processes in your singularity enabled docker container: 

$ singularity shell mybioapp.img cat /etc/os-release $ singularity exec mybioapp.img echo "Hello bio app!"

 

 

 

 

Go to KB0039307 in the IS Service Desk