January 24, 2024
Loading Available Modules
Software provided on the Scientific Computing (SciC) Linux Clusters through the Modules system (aka environment modules) are installed in one place but available on all servers and compute nodes. Loading one of these modules sets the correct paths and environment variables for your account to use software, and they can be "unloaded" at a later time. There are many advantages to using environment modules for providing access to software, over manually setting paths and variables.
Create New Modules
If a lab requires a change to a module, a new module should be created under "modulefiles/lab/their_lab". Essential changes to modules under "modulefiles/lab" can only be made during the Sunday maintenance window following notification to SciC Linux Clusters Support. The new version is created under "modulefiles/test" and moved to "modulefiles/base" during maintenance.
Where a package may be available, either on the compute node or through environment modules, use the module version if you require version stability or a specific release version.
Module file organization
Module files are located under "/apps/modulefiles", there are several repositories described below. Within each repository, the structure is (all in lower case)
package_name/package_version
eg. the module for "SCons" version 2.3.0 is in a text file named "2.3.0" in a folder named "scons" and appears when "module avail" is run as
scons/2.3.0
Modules can have a default version, for researchers who wish to always have the current version and do not require or desire notification when the current version changes. A default version is created with a symbolic link pointing to the current version. This can be changed without affecting running jobs
cd scons/
ln -s 2.3.0 default
/apps/modulefiles/base
Available to all users by default. Finalized modules go here after being tested and accepted (under "/apps/modulefiles/test") by SciC Linux Clusters and the requesting researcher
/apps/modulefiles/test
Modules in development go here until finalized. Ask the requesting researcher to issue this command to make the module available for testing:
module use /apps/modulefiles/test
Modules in this location may be unfinished or broken
/apps/modulefiles/licensed
Modules for software requiring a proprietary license tied to a specific group goes here, to prevent confusion if users without access to the license see it in the main repository and attempt to run the software.
/apps/modulefiles/lab
Lab groups wishing to keep software centrally for high-availability will be given a modules folder under "/apps/modulefiles/lab". This is also the location for individuals or lab groups requiring the same software as under "base" but with different compile options. By issuing
module use /apps/modulefiles/lab/mylab
the custom version or lab-specific software will be loaded in preference over "base" modules
Module template
Refer to the Documentation.
All modules should create an environment variable with their name followed by "VERSION" containing the version number
setenv SCONSVERSION "2.3.0"
Modules depending on other modules should use the "prereq" statement (eg. to depend on any version of python)
prereq "python"
Python Interpreter Environment Module Template
In addition to the standard environment module template, a python interpreter module requires this variable set
setenv PYTHONLIBVERSION "python2.7"
Python Egg/Application Environment Module Template
Python "eggs" are compatible with the major version of python they are installed for. The additions to the standard environment module template are:
Check that a python interpreter is loaded
prereq python
Read the PYTHONLIBVERSION set by the python interpreter environment module and use this to set the correct path to the module source
set curMod module-info name
set module_root "/apps/source/python-lib/$::env(PYTHONLIBVERSION)/$curMod"
Check that the module is available for the version of Python currently loaded
if
Unknown macro: { ! [file exists "$module_root"] }
Unknown macro: { puts stderr "$curMod is not installed for Python version $}
Source (software installations) organization
Source files are located under /apps/source/ The structure is (all in lower case)
package_name/package_version
eg. the source for "tophat" version 2.3.0 is in a folder named "tophat/2.3.0".
Within the "tophat/2.3.0" folder, a folder "source" will contain the downloaded installation files (either source code or binary archive). Any compilation will be done under "tophat/2.3.0/source" - the install location (prefix) will be "tophat/2.3.0"
Python (eggs or software) organization
/source/python-lib/$
Unknown macro: {PYTHONLIBVERSION}
/packagename/packageversion (target for installation)
/source/python-lib/$
/packagename/packageversion/source (downloaded installer)
PYTHONLIBVERSION is "python2.7" etc... which is also set as an environment variable when a python interpreter environment module is loaded