Caffe on ERISOne Cluster

Caffe

Caffe is an open-source deep learning library with a focus on image recognition. It's really modular and provides support for different third party libraries and programming interfaces (C++, Python, and Command line). It's extremely efficient, especially when it's used with GPUs

The ERISOne Caffe install has been compiled to support:

  • Pycaffe (Programming interface for Python 2.7.3 )
  • Matcaffe (Module for Matlab 2015b)
  • OpenCV 3.0.1
  • IO libraries, lmdb, leveldb

Almost all the optional modules are available except cuDNN.

Instructions

You can connect to ERISOne using a Remote Desktop session or a Command Line session via SSH. Follow these instructions to connect to ERISOne.

If you decided to connect to the Remote Desktop, you will have to open a Terminal window from your desktop. From the command line, you can use Caffe with CPUs or with GPUs depending on the configuration of your script

Using GPUs

The ERISOne login nodes don't have GPUs installed. So, in order to use a GPU you'll have to request an interactive session in a node with GPUs.

On the Terminal, type:

$ bsub -Is -q gpu /bin/bash

Please, remember that you'll need to get access to that queue. Contact us if you get an error.

Once you get the shell on the gpu0XX node, type:

$ module use /apps/modulefiles/test $ module load caffe/1.0rc3

Now, you can run some tests to be sure that everything is working fine. This is a feature extraction example using Pycaffe, and a trained BVLC GoogLeNet   model

$ cd /apps/testsuite/caffe $ python caffe_feature_extractor_GPU.py -i img_list.txt -o ~/vectors.output

If you want to customize that example, you just need to provide a different input file with a full-path list of your images. 

Using CPUs

You can run Caffe with CPUs on the remote desktops servers. This can be very convenient if you want to use a graphical tool in conjunction with Caffe. But, remember that the remote desktop servers don't have GPUs.

On the Terminal, type:

$ module use /apps/modulefiles/test $ module load caffe/1.0rc3-CPU 

And you can also run the same example, but please note that the name of the python script is different (CPU vs. GPU)

cd /apps/testsuite/caffe python caffe_feature_extractor_CPU.py -i img_list.txt -o ~/output

 

Tips & Warnings

  • The remote desktops don't have a GPU installed, use the gpu nodes instead if you want to use GPUs.
  • If you want to know exactly what library versions and modules are supported<  
 $ module help caffe/1.0rc3 

Requirements

Related

 


Related articles