CODEWITHSUNDEEP
bashperlslurm
selami
selami

Reputation: 89

How can I convert my script for submitting SLURM jobs from Bash to Perl?

I have the following Bash script for job submission to SLURM on a cluster:

#!/bin/bash
#SBATCH -A 1234
#SBATCH -t 2-00:00 
#SBATCH -n 24

module add xxx
srun resp.com

The #SBATCH lines are SLURM commands:

module add xxx loads the Environment Module xxx (in this case I'm actually using module add gaussian, where gaussian is a computational quantum-chemistry program).

srun is the SLURM command to launch a job. resp.com includes commands for gaussian and atom coordinates.

I tried converting the Bash script to the following Perl script, but it didn't work. How can I do this in Perl?

#!/usr/bin/perl 

use strict;
use warnings;
use diagnostics;

system ("#SBATCH -A 1234");
system ("#SBATCH -t 2-00:00");
system ("#SBATCH -n 24");

system ("module add xxx");
system ("srun resp.com ");

Upvotes: 0

Views: 1351

Answers (2)

damienfrancois
damienfrancois

Reputation: 59110

What you need to do is write

#!/usr/bin/perl 

#SBATCH -A 1234
#SBATCH -t 2-00:00
#SBATCH -n 24

use strict;
use warnings;
use diagnostics;

system ("module add xxx && srun resp.com ");

and then submit it with 

sbatch my_perl_script.pl

The #SBATCH lines are comments destined to be parsed by the sbatch command. They must be comments in the submission script.

The module command modifies the environment, but that environment is lost as soon as it is called if you invoke it with system on its own as system creates a subshell. You need to either invoke it on the same subshell as srun, as shown above, or use Perl tools to load the module in the environment of the Perl script so that it is available to srun, using use Env::Modulecmd { load => 'foo/1.0' }; as mentioned elsewhere.

Upvotes: 2

tjd
tjd

Reputation: 4104

Each of your system calls creates a child process to run the program in question and returns when the child process dies.

The whole point of module is to configure the current shell by, among other things, modifying it's environment. When this process completes (dies) say goodbye to those changes. The call to srun, in it's shinny new process with a shinny new environment, hasn't got a chance.

Steps forward:

  • Understand SLURM & bash and exactly why system("#SBATCH whatever"); might not be of any value. Hint: # marks the beginning of a comment in both Bash & Perl.
  • Understand what module add is doing with xxx and how you might replicate what it's doing inside the shell within the Perl interpreter. ThisSuitIsBlackNot recommends use Env::Modulecmd { load => 'foo/1.0' }; to replicate this functionality.
  • Barring any understanding of module add, system ('module add xxx; srun resp.com') would put those two commands in the same shell process, but at this point you need to ask yourself what you've gained by adding a Perl interpreter to the mix.

Upvotes: 3

Related Questions