CODEWITHSUNDEEP
cluster-computingjob-schedulinglsf
thebigpeeler
thebigpeeler

Reputation: 95

LSF - BSUB Running a script if the job is killed

Im working with the LSF, running bsub commands.

I'm implementing the -Ep switch to run a post exec script. This works great until the Job is killed or hits a memory limit, run limit etc.

Is there any way for the job to detect its running out of resource and then run the script? or to force it to run the script even if its been killed?

I guess my other option is running job with a dependency on that job which will run the "post exec" script when it finishes.

Any thoughts?

Kind Regards,

TheBigPeeler

Upvotes: 0

Views: 915

Answers (1)

Michael Closson
Michael Closson

Reputation: 932

From the documentation, you should be seeing the behaviour that you want.

A post-execution command runs after the job finishes, regardless of the exit state of the job. Once a post-execution command is associated with a job, that command runs even if the job fails. You cannot configure the post-execution command to run only under certain conditions.

I thought that maybe the interaction with JOB_INCLUDE_POSTEXEC (lsb.params) could account for the difference, but from my test the post-exec still runs in both cases. I used runlimit (bsub -W) to trigger the job kill.

Is it possible that the post exec is running, but exits early?

What version of LSF are you using? (What's the output of mbatchd -V and sbatchd -V)

Upvotes: 1

Related Questions