Ed Bennett
Ed Bennett

Reputation: 161

How does one get Mathematica `wolframscript` to behave (in parallel or otherwise) when run via `snakemake`?

I'm trying to make use of some pieces of Mathematica code as part of a Snakemake workflow. I've translated the relevant files into Wolfram scripts (.wls) so that they can be more reliably run from the command line. However, it's hitting some problems when run.

When run in parallel without limitation, WolframScript will at some point (but not necessarily initially) complain about a problem with the license file. Rather than aborting, it then hangs, and Ctrl+Cing Snakemake puts it into a state where it's constantly printing a quit prompt that can't be interacted with.

I've used custom resources to specify that each rule using wolframscript consumes 1 mathematica_licenses, and then set the limit --resource mathematica_licenses=3 (which has been sufficiently few previously when I've used make for a similar task). It still encounters issues however; the scroll back moves fast enough that I can't tell if it's the same licensing issue, but it got into a state where it hung for hours idling the CPU before giving up.

Running with --cores 1 allowed the workflow to proceed without hitting these issues. However, the Mathematica code is the bulk of the computation time in this workflow, so if possible I'd like to parallelise it.

Are there recommended practices for running WolframScript jobs in Snakemake workflows? Is it possible to get significant parallelism? If not, can it at least be made to be guaranteed to not get stuck in an idle state waiting for user input?

Thanks!

Upvotes: 0

Views: 92

Answers (0)

Related Questions