Scheduling recurring compute or data transfer tasks on Scientific Computing Linux Clusters

Current active ERIS Cron Servers/nodes:
eris2cron.research.partners.org

Start by creating a BASH script that will perform the scheduled task.  The script must setup the environment for any software packages required for it to run. To submit a task to the SciC Linux Clusters scheduler and load software from the "environment modules" repository the following lines must be included:

source /etc/profile.d/lsf.sh

source /etc/profile.d/modules.sh  
### if you are using Lmod
#source /etc/profile.d/00-modulepath.sh
#source /etc/profile.d/z00_lmod.sh

EXAMPLE SCRIPT 

Overview of this example:     

• cron schedules an script (test.sh) to run every night at 02:20    
• the test.sh script in turn submits a compute job to the compute cluster to be queued and executed 

This trivial example script submits a compute job to the Scientific Computing (SciC) Linux Clusters. The compute job loads the "miniforge3" module and reports the python version. Save this as "test.sh": 

#!/bin/bash 
source /etc/profile.d/lsf.sh 

bsub -q short >/dev/null << EOF 

source /etc/profile.d/modules.sh
### if you are using Lmod
#source /etc/profile.d/00-modulepath.sh
#source /etc/profile.d/z00_lmod.sh

module load miniforge3 
python --version 
EOF

Make the script executable and test that it works before creating the scheduled task. Assuming a script named "test.sh" in your home folder "/PHShome/abc123" you would test it under an empty environment (the same as the environment it will have when executed by cron) as follows: 

$ chmod +x /PHShome/abc123/test.sh  

$ env -i /PHShome/abc123/test.sh  

Continue testing the script until it runs successfully under "env -i"

EXAMPLE CRON JOB 

Cron is a Linux tool for scheduling tasks.  Documentation can be found by typing "man 5 crontab" on SciC Linux Clusters, or with a google search for "crontab". 

Cron scheduled tasks are visible only on the node they were created on. All scheduled tasks should be created on the ERIS Cron Servers.
These servers cannot handle compute jobs.

No compute tasks may be performed on ERIS Cron Servers - they must be submitted to the cluster via "bsub" 

Connect to a Cron Server and type "crontab -e" to add an entry to your personal scheduled task list 

$ ssh eris2cron 

$ crontab -e  

This will start the "nano" text editor.  Enter details of the cron job following the crontab syntax and save the file. For example, this content in your crontab will run the test.sh script every day at 02:30 

30 2 * * * /PHShome/abc123/test.sh  

Exit from the "nano" editor with CTRL + X. If there are any problems with the syntax of the cron job an error message will appear at this time  Finally, exit from the "eris1cron" node 

$ exit  

Any text output or errors will be emailed to you.