Caffe is an open-source deep learning library with a focus on image recognition. It's really modular and provides support for different third party libraries and programming interfaces (C++, Python, and Command line). It's extremely efficient, especially when it's used with GPUs.
The ERISOne Caffe install has been compiled to support:
- Pycaffe (Programming interface for Python 2.7.3 )
- Matcaffe (Module for Matlab 2015b)
- OpenCV 3.0.1
- IO libraries, lmdb, leveldb
Almost all the optional modules are available except cuDNN.
You can connect to ERISOne using a Remote Desktop session or a Command Line session via SSH. Follow these instructions to connect to ERISOne.
If you decided to connect to the Remote Desktop, you will have to open a Terminal window from your desktop. From the command line, you can use Caffe with CPUs or with GPUs depending on the configuration of your script
The ERISOne login nodes don't have GPUs installed. So, in order to use a GPU you'll have to request an interactive session in a node with GPUs.
On the Terminal, type:
$ bsub -Is -q gpu /bin/bash
Please, remember that you'll need to get access to that queue. Contact us if you get an error.
Once you get the shell on the gpu0XX node, type:
$ module use /apps/modulefiles/test $ module load caffe/1.0rc3
Now, you can run some tests to be sure that everything is working fine. This is a feature extraction example using Pycaffe, and a trained BVLC GoogLeNet model
$ cd /apps/testsuite/caffe $ python caffe_feature_extractor_GPU.py -i img_list.txt -o ~/vectors.output
If you want to customize that example, you just need to provide a different input file with a full-path list of your images.
You can run Caffe with CPUs on the remote desktops servers. This can be very convenient if you want to use a graphical tool in conjunction with Caffe. But, remember that the remote desktop servers don't have GPUs.
On the Terminal, type:
$ module use /apps/modulefiles/test $ module load caffe/1.0rc3-CPU
And you can also run the same example, but please note that the name of the python script is different (CPU vs. GPU)
cd /apps/testsuite/caffe python caffe_feature_extractor_CPU.py -i img_list.txt -o ~/output
Tips & Warnings
- The remote desktops don't have a GPU installed, use the gpu nodes instead if you want to use GPUs.
- If you want to know exactly what library versions and modules are supported<
$ module help caffe/1.0rc3
- You will need an ERISOne Linux Cluster account. Please, follow these instructions if you need to request an account.
- If you want to use Caffe with GPUs (Graphics Processing Units), you will also need to get access to the GPU nodes. Please, contact us with a description of your project to be able to grant access to the GPU queue.