May 25, 2024
Introduction
In an earlier article Using Singularity on the ERIS Cluster we introduced the image management tool called Singularity and which is made available on the ERIS cluster from a software module. By using Docker images with this tool, we noted that the user is able to run legacy software from ERISone, for example, on the newer ERIStwo platform.
Docker images hosted at the repository www.docker.com should be sufficient to cover the needs of most users. For those that wish to customize and create their own Singularity or sif images, however, we outline one approach below that makes use of small ascii text files called Dockerfiles. Although the procedure is involved it has the advantage of avoiding having to permanently store large (approximately 10GB as noted previously) sif files on the ERIS cluster.
Finally, several approaches were investigated and at this point in time it was concluded that using the Docker application on a user's local PC represented the most convenient approach to ultimately creating customized sif images for use on the ERIS cluster.
Before proceeding we note that creating/customizing Docker images requires sudo privileges so that the steps described below can *only* be performed on a user's local PC (windows, linux or macintosh) i.e. they cannot be implemented on the ERIS cluster.
Installation of Docker Desktop on your local PC
Windows
- Installation instructions are available at Install Docker Desktop on Windows
- Login to the Privileged Local Administrator Service (PLAS) to enable the installation of Docker Desktop.
- As Administrator run the installer Docker Desktop Installer.exe. By default, Docker Desktop will be installed at
C:\Program Files\Docker\Docker
- After the installation of Docker Desktop is successfully completed you should be able to see the Docker icon as shown
- Reboot your Laptop / PC (according to WSL Installation recommendations).
- Double click the Docker icon to start the Docker Desktop app.
Macintosh
- Installation instructions are available at Install Docker Desktop on Mac
Linux
- Installation instructions are available at Install Docker Desktop on Linux
Creating a Sample Dockerfile
Here we create a sample Dockerfile consisting of commands to build an image containing python and other utilities and which subsequently runs python3 on a demo code (primeNumbers.py) at start up.
1. Create a Dockerfile:
- Start by creating a text file named Dockerfile. This file defines the instructions for building your Docker image.
- Open the Dockerfile in a text editor.
2. Specify the Base Image:
- Choose a base image that suits your requirements. For this example, lets use Ubuntu 20.04.
- In your Dockerfile, add the following line to specify the base image:
FROM ubuntu:20.04
3. Install Python and Libraries:
- Install Python and any necessary libraries. For instance, lets install numpy and matplotlib:
RUN apt-get update && apt-get install -y python3 python3-pip
RUN pip3 install numpy matplotlib
4. Copy Your Python Script:
- Place your Python script (e.g., for computing prime numbers) in the same directory as your Dockerfile.
- Add the following line to copy your script into the image:
COPY primeNumbers.py /app/
5. Set Working Directory:
- Specify the working directory inside the image:
WORKDIR /app
6. Define the Command to Run:
- Finally, specify the command to run when a container is generated from the docker image:
CMD ["python3", "primeNumbers.py"]
and where the contents of primeNumbers.py are the following:
# Python program to print all
# prime number in an interval
def prime(x, y):
prime_list = []
for i in range(x, y):
if i == 0 or i == 1:
continue
else:
for j in range(2, int(i/2)+1):
if i % j == 0:
break
else:
prime_list.append(i)
return prime_list
# Driver program
starting_range = 1000
ending_range = 2000
lst = prime(starting_range, ending_range)
if len(lst) == 0:
print("There are no prime numbers in this range")
else:
print("The prime numbers in this range are: ", lst)
Note: Windows/Notepad will append .txt to the ascii DockerFile e.g. <myDockerFile>.txt. You will need to remove the suffix .txt to to enable <myDockerFile> to be used in VSCode.
The full Dockerfile reference of commands etc. can be found here.
Building the Docker Image from the sample Dockerfile
1. Open a Terminal:
On your local machine, open a terminal or command prompt.
2. Navigate to Your Project Directory:
Change directory to where your Dockerfile and Python script are located.
3. Build the Docker Image:
Run the following command to build the image (replace <your_image_name> with an appropriate name, e.g. test_case_1):
docker build -t <your_image_name> .
or
docker build -t test_case_1 .
It will show the docker images was created.
Running a Container from the New Image
Once the image is built, you can generate and run a container from it:
docker run -it --rm <my-image> /bin/bash
e.g.
docker run -it --rm test_case_1 /bin/bash
which will invoke an interactive terminal session using the bash shell and where option "--rm" will automatically
remove the container after it has been exited. More details about docker run can be found here.
In fact at this stage it is quite possible to continue customizing the image from an interactive shell. To do so we would invoke
docker run -it <my-image> /bin/bash
and then perhaps install more packages e.g.
apt update
apt-get install vim
After exiting the container with CTL-D (or simply exit) we would then identify the associated containerID with
docker ps -a
and subsequently commit the modifed container to a new image using
docker tag <containerID> <my-new-image>
following which it could be pushed to Docker Hub as described below.
Pushing the new image to Docker Hub
To push an image to Docker Hub you will first need to create an account at www.docker.com
Sign up or Sign into Docker Hub.
Select the Create Repository button.
In this case the account name is hlinma and the repository is called test_case_1, subsequently we ensure the visibility is Public.
Select Create.
You can push the image to your account's repository using either the Docker Desktop
or the command line i.e.
docker login
then tag the local image with that of your docker hub account i.e.
docker tag <local_image_name> <your_dockerhub_account_name>/<your_image_name>
finally, push the image to dockerhub
docker push <your_dockerhub_account_name>/<your_image_name>
The Docker image is now available to be pulled into ERIStwo/Singularity as described previously.