Scheduling recurring compute or data transfer tasks on ERISOne

Start by creating a BASH script that will perform the scheduled task.  The script must setup the environment for any software packages required for it to run. To submit a task to the ERISOne cluster scheduler and load software from the "environment modules" repository the following lines must be included:

source /etc/profile.d/lsf.sh

source /etc/profile.d/modules.sh 
### if you are using Lmod
#source /etc/profile.d/00-modulepath.sh
#source /etc/profile.d/z00_lmod.sh

EXAMPLE SCRIPT 

Overview of this example:     

  • cron schedules an script (test.sh) to run every night at 02:20    
  • the test.sh script in turn submits a compute job to the compute cluster to be queued and executed 

This trivial example script submits a compute job to the ERISOne cluster. The compute job loads the "anaconda" module and reports the python version. Save this as "test.sh": 

#!/bin/bash 
source /etc/profile.d/lsf.sh

bsub -q short >/dev/null << EOF

source /etc/profile.d/modules.sh
### if you are using Lmod
#source /etc/profile.d/00-modulepath.sh
#source /etc/profile.d/z00_lmod.sh

module load anaconda/4.3.1
python --version
EOF

Make the script executable and test that it works before creating the scheduled task. Assuming a script named "test.sh" in your home folder "/PHShome/abc123" you would test it under an empty environment (the same as the environment it will have when executed by cron) as follows: 

$ chmod +x /PHShome/abc123/test.sh  
$ env -i /PHShome/abc123/test.sh  

Continue testing the script until it runs successfully under "env -i"

EXAMPLE CRON JOB 

Cron is a Linux tool for scheduling tasks.  Documentation can be found by typing "man 5 crontab" on ERISOne, or with a google search for "crontab". 

Cron scheduled tasks are visible only on the node they were created on. All scheduled tasks should be created on the node "eris1cron". No compute tasks may be performed on eris1cron itself - they must be submitted to the cluster via "bsub" 

Connect to "eris1cron" and type "crontab -e" to add an entry to your personal scheduled task list 

$ ssh eris1cron 
$ crontab -e  

This will start the "nano" text editor.  Enter details of the cron job following the crontab syntax and save the file. For example, this content in your crontab will run the test.sh script every day at 02:30 

30 2 * * * /PHShome/abc123/test.sh  

Exit from the "nano" editor with CTRL + X. If there are any problems with the syntax of the cron job an error message will appear at this time  Finally, exit from the "eris1cron" node 

$ exit  

Any text output or errors will be emailed to you.   


Related articles