CODEWITHSUNDEEP
bashheredocslurmsbatch
Coby Viner
Coby Viner

Reputation: 184

Use of a HEREDOC with SLURM sbatch --wrap

I am encountering difficulties using a (Bash) HEREDOC with a SLURM sbatch submission, via --wrap.

I would like the following to work:

SBATCH_PARAMS=('--nodes=1' '--time=24:00:00' '--mem=64000' '--mail-type=ALL')

sbatch ${SBATCH_PARAMS[@]} --job-name="MWE" -o "MWE.log" --wrap <<EOF
        SLURM_CPUS_ON_NODE=\${SLURM_CPUS_ON_NODE:-8}
        SLURM_CPUS_PER_TASK=\${SLURM_CPUS_PER_TASK:-\$SLURM_CPUS_ON_NODE}
        export OMP_NUM_THREADS=\$SLURM_CPUS_PER_TASK

        parallel --joblog "MWE-jobs.log" --resume --resume-failed -k --linebuffer -j \$((\$OMP_NUM_THREADS/4)) --link "MWE.sh {1} {2}" ::: "./"*R1*.fastq.gz ::: "./"*R2*.fastq.gz
EOF

On my current cluster, sbatch returns the below error, refusing to submit this job:

ERROR:   option --wrap requires argument

Might anyone know how I can get this to work?

Upvotes: 0

Views: 1089

Answers (3)

Carles Fenoy
Carles Fenoy

Reputation: 5357

You can just use the heredoc without the wrap provided you add the #!/bin/bash at the top of it.

Upvotes: 2

Rafael
Rafael

Reputation: 7746

Since wrap expects a string argument, you can't use a heredoc directly. Heredocs are used when a filename is expected where it's undesirable to make one.

Use a heredoc for cat, where it does expect a filename, and use its output as the string for which --wrap expects: 

SBATCH_PARAMS=('--nodes=1' '--time=24:00:00' '--mem=64000' '--mail-type=ALL')

sbatch ${SBATCH_PARAMS[@]} --job-name="MWE" -o "MWE.log" --wrap $(cat << EOF
        SLURM_CPUS_ON_NODE=\${SLURM_CPUS_ON_NODE:-8}
        SLURM_CPUS_PER_TASK=\${SLURM_CPUS_PER_TASK:-\$SLURM_CPUS_ON_NODE}
        export OMP_NUM_THREADS=\$SLURM_CPUS_PER_TASK

        parallel --joblog "MWE-jobs.log" --resume --resume-failed -k --linebuffer -j \$((\$OMP_NUM_THREADS/4)) --link "MWE.sh {1} {2}" ::: "./"*R1*.fastq.gz ::: "./"*R2*.fastq.gz
EOF)

Upvotes: 1

Coby Viner
Coby Viner

Reputation: 184

Adapting a related post on assigning a HEREDOC to a variable, but instead using cat (since I use errexit and want to avoid working-around the non-zero exit value of the read), I was able to submit my job as follows:

CMD_FOR_SUB=$(cat <<EOF
    SLURM_CPUS_ON_NODE=\${SLURM_CPUS_ON_NODE:-8}
    SLURM_CPUS_PER_TASK=\${SLURM_CPUS_PER_TASK:-\$SLURM_CPUS_ON_NODE}
    export OMP_NUM_THREADS=\$SLURM_CPUS_PER_TASK

    parallel --joblog "MWE-jobs.log" --resume --resume-failed -k --linebuffer -j \$((\$OMP_NUM_THREADS/4)) --link "MWE.sh {1} {2}" ::: "./"*R1*.fastq.gz ::: "./"*R2*.fastq.gz
EOF
)

sbatch ${SBATCH_PARAMS[@]} --job-name="MWE" -o "MWE.log" --wrap "$CMD_FOR_SUB"

While this does appear to work, I would still prefer a solution that allows sbatch to directly accept the HEREDOC.

Upvotes: 0

Related Questions