CODEWITHSUNDEEP
pythonbioinformaticssnakemake
yippingAppa
yippingAppa

Reputation: 571

Snakemake: Running hybrid workflows

I was wondering if there is any way to configure snakemake so that certain rules can be run locally while others are run on an cluster.

For example, suppose I had rule A, which is lightweight, and rule B, which is more computationally expensive. Rule B requires the output of rule A. Is there any way for me to run rule A on a local computer and rule B be submitted as a job via SLURM?

EDIT: I should have read the documentation more thoroughly. What I said above can be accomplished using localrules.

As a modification to the above question, can I run one rule on a cluster and one rule on the cloud?

Upvotes: 2

Views: 71

Answers (1)

dariober
dariober

Reputation: 9062

A probably cluncky solution is to use ssh to execute commands on the remote host. I don't use the cloud but I assume you can ssh there or do something equivalent. So something like:

rule one:
    input:
        'data.txt',
    output:
        'results.txt',
    shell:
        r"""
        rsync -arvP {input} [email protected]:~/
        ssh [email protected] "sort {input} > {output}"
        rsync -arvP [email protected]:~/{output} ./ 
        """

Upvotes: 1

Related Questions