CODEWITHSUNDEEP
linuxbashslurm
Terry BRETT
Terry BRETT

Reputation: 319

SLURM distinct multiple runs

I am trying to write a script, which will take an input from the lambda array, and have 100 distinct runs (evaluations) for each lambda. This is what I've got so far. 

#!/bin/bash -l

#SBATCH --qos=regular
#SBATCH --nodes=20
#SBATCH --time=120:00:00
#SBATCH --job-name=sis_model
#SBATCH --array=0-20
#SBATCH --distribution=block

lambda=(0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1)

for paramater in ${lambda[*]}
do
        for ip1 in {0..100}
                do
                        srun ./test ${lambda[$SLURM_ARRAY_TASK_ID]}
                done
done

Upvotes: 0

Views: 44

Answers (1)

Poshi
Poshi

Reputation: 5762

I think you are overworking the code. It is easier:

#!/bin/bash -l

#SBATCH --qos=regular
#SBATCH --nodes=20
#SBATCH --time=120:00:00
#SBATCH --job-name=sis_model
#SBATCH --array=0-19
#SBATCH --distribution=block

lambda=(0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1)

for ip1 in {1..100}
do
    srun ./test ${lambda[$SLURM_ARRAY_TASK_ID]}
done

Things to watch for: the array command goes from 0 to 19, not from 0 to 20. You only have 20 elements in the lambda list. Your first job array should be zero because the first element in the list is the number zero. If you are more comfortable with a 1..20 schema, then you will have to substract one when accessing the lambda array.

Also, if you are using a job array to process the different element in lambda in parallel, there is no need to process all of them in every job. Just let the system process a single lambda per job.

Related to the first comment, if you want 100 outputs per lambda, you should iterate 100 times, not 101.

As a final thought, your lambda could be computed on the fly, in case your list grows bigger and keeps regular. Expanding the list is only useful to my eyes when you cannot compute the value easily. But your lambda==0.05*(1+$SLURM_ARRAY_TASK_ID).

Upvotes: 1

Related Questions