CODEWITHSUNDEEP
bash
user3033194
user3033194

Reputation: 1821

Change the value of external SLURM variable

I am running a bash script to run jobs on Linux clusters, using SLURM. The relevant part of the script is given below (slurm.sh):

#!/bin/bash
#SBATCH -p parallel
#SBATCH --qos=short
#SBATCH --exclusive
#SBATCH -o out.log
#SBATCH -e err.log
#SBATCH --open-mode=append
#SBATCH --cpus-per-task=1
#SBATCH -J hadoopslurm
#SBATCH --time=01:30:00
#SBATCH --mem-per-cpu=1000
#SBATCH [email protected]
#SBATCH --mail-type=ALL
#SBATCH -N 5

I am calling this script from another script (ext.sh), a part of which is given below:

#!/bin/bash
for i in {1..3}

do
    source slurm.sh
done

..

I want to manipulate the value of the N variable is slurm.sh (#SBATCH -N 5) by setting it to values like 3,6,8 etc, inside the for loop of ext.sh. How do I access the variable programmatically from ext.sh? Please help.

Upvotes: 0

Views: 669

Answers (1)

damienfrancois
damienfrancois

Reputation: 59110

First note that if you simply source the shell script, you will not submit a job to Slurm, you will simply run the job on the submission node. So you need to write

#!/bin/bash
for i in {1..3}

do
    sbatch slurm.sh
done

Now if you want to change the -N programmatically, one option is to remove it from the file slurm.sh and add it as argument to the sbatch command:

#!/bin/bash
for i in {1..3}

do
    sbatch -N $i slurm.sh
done

The above script will submit three jobs with respectively 1, 2, and 3 nodes requested.

Upvotes: 1

Related Questions