CODEWITHSUNDEEP
snakemake
Lance
Lance

Reputation: 81

Should Snakemake workflows using wrappers require cluster nodes to have internet access (e.g. to github)?

When attempting to run some Snakemake workflows that used wrappers on one of our clusters, I ran into issues where the jobs would fail with following error:

WorkflowError:
Failed to open source file https://github.com/snakemake/snakemake-wrappers/raw/0.77.0/bio/cutadapt/se/environment.yaml
ConnectionError: HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /snakemake/snakemake-wrappers/raw/0.77.0/bio/cutadapt/se/environment.yaml (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x2afb78d6ebe0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

The submit node of the cluster is able to get the wrapper scripts, create the conda environments, etc. but the nodes themselves do have access to the internet. Is there a way around this, or should I avoid snakemake wrappers when using clusters with no internet access?

Upvotes: 0

Views: 192

Answers (1)

Maarten-vd-Sande
Maarten-vd-Sande

Reputation: 3711

Idk if really a nice solution, but you could clone the wrappers repository and use them as local wrappers?

rule index:
    wrapper:
        "file://public/home/bio/index"

Upvotes: 2

Related Questions