CODEWITHSUNDEEP
dockernextflowsingularity-container
Anuradha Ravi
Anuradha Ravi

Reputation: 33

Starting Nextflow in a singularity container and use it to run another singularity container

I want to use nextflow in a singularity container that will further run my nextflow pipelines packed in a singularity. This will help us bypass installation of nextflow in all our test servers. I am running this on a Linux server.

Nextflow runs well on a singularity container

singularity exec --env NXF_HOME=/home/.nf-singularity \
-B /data/home/giab_NA12878/Data_LongRead_sequencing/:/input \
-B /data/home/my_tools/sifs/:/sifs \
-B /data/home/my_tools/nf_tools:/tools \
-B /data/home/databases/UCSC_download/:/db \
/data/home/my_tools/sifs/nextflow_latest.sif \
nextflow

profiles {
        foo {
                autoMounts = true
                docker.enabled = false
                singularity.enabled = true
                singularity.cacheDir = "$PWD"
                process.container = '/data/home/my_tools/sifs/wf_human_variantion_mr442.sif'
                report.overwrite = true
                timeline.overwrite = true
        }
}

I tried to launch the script like this:

#!/bin/bash
export NXF_VER=23.04.2;
export NXF_ANSI_LOG=false
singularity exec --env NXF_HOME=/home/.nf-singularity \
-B /data/home/giab_NA12878/Data_LongRead_sequencing/:/input \
-B /data/home/my_tools/sifs/:/sifs \
-B /data/home/my_tools/nf_tools:/tools \
-B /data/home/databases/UCSC_download/:/db
 -B /bin/singularity \
/data/home/my_tools/sifs/nextflow_latest.sif \
nextflow run -c /input/wf_sing_nf.config /tools/wf-human-variation -profile foo \
-w /input/nf_test/workspace --snp --str --cnv \
--bam /input/giab_NA12878_only_chr_sort.bam --bam_min_coverage 0 \
--ref /db/human_selected_onlychr.fasta --sample_name giab_hg001_nf \
--basecaller_cfg [email protected] \
--out_dir /input/nf_test/HG001_test_nextflow \
-with-report /input/nf_test/run_report_hg001_nf.html \
-with-timeline /input/nf_test/run_timeline_hg001_nf.html

I get the following error

This is epi2me-labs/wf-human-variation v2.2.4.
--------------------------------------------------------------------------------
Searching input for [.bam, .ubam] files.
WARN: Inferring genetic sex of sample as params.sex was not provided.
Pulling Singularity image docker://ontresearch/wf-human-variation-snp:sha17e686336bf6305f9c90b36bc52ff9dd1fa73ee9 [cache /home/ontresearch-wf-human-variation-snp-sha17e686336bf6305f9c90b36bc52ff9dd1fa73ee9.img]
Pulling Singularity image docker://ontresearch/wf-human-variation-str:shaa2f49ce57886426516eadd4048b6fdf9c22c7437 [cache /home/ontresearch-wf-human-variation-str-shaa2f49ce57886426516eadd4048b6fdf9c22c7437.img]
Pulling Singularity image docker://ontresearch/spectre:sha5a2890023dc7a7899f47585103b4f5762fb9d1b3 [cache /home/ontresearch-spectre-sha5a2890023dc7a7899f47585103b4f5762fb9d1b3.img]
Pulling Singularity image docker://ontresearch/wf-common:sha338caea0a2532dc0ea8f46638ccc322bb8f9af48 [cache /home/ontresearch-wf-common-sha338caea0a2532dc0ea8f46638ccc322bb8f9af48.img]
ERROR ~ Error executing process > 'cnv_spectre:getVersions'

Caused by:
  Failed to pull singularity image
  command: singularity pull  --name ontresearch-spectre-sha5a2890023dc7a7899f47585103b4f5762fb9d1b3.img.pulling.1722512267214 docker://ontresearch/spectre:sha5a2890023dc7a7899f47585103b4f5762fb9d1b3 > /dev/null
  status : 127
  message:
    singularity: error while loading shared libraries: libseccomp.so.2: cannot open shared object file: No such file or directory

 -- Check '.nextflow.log' file for details

I could add the .nextflow.log report as well if it is useful. I have tried multiple ways that the local singularity container is conecting but it isnt working. Is there something within my config I should add so that the other singularity image is also recognised? am I missing something?

Thanks for your help

Upvotes: 0

Views: 212

Answers (1)

Steve
Steve

Reputation: 54562

You are binding in singularity using -B /bin/singularity but Singularity requires some shared libraries (i.e. libseccomp). You could try also binding in /usr using -B /usr. We can test this, using the nextflow/nextflow container in Docker Hub, for example. Without -B /usr:

$ singularity exec -B /bin/singularity docker://nextflow/nextflow ldd /bin/singularity
INFO:    Using cached SIF image
/bin/singularity: /lib64/libc.so.6: version `GLIBC_ABI_DT_RELR' not found (required by /bin/singularity)
/bin/singularity: /lib64/libc.so.6: version `GLIBC_2.38' not found (required by /bin/singularity)
    linux-vdso.so.1 (0x00007a284b3cd000)
    libseccomp.so.2 => not found
    libc.so.6 => /lib64/libc.so.6 (0x00007a28487f8000)
    /lib64/ld-linux-x86-64.so.2 (0x00007a284b3cf000)

With -B /usr:

$ singularity exec -B /bin/singularity -B /usr docker://nextflow/nextflow ldd /bin/singularity
INFO:    Using cached SIF image
    linux-vdso.so.1 (0x0000768f75255000)
    libseccomp.so.2 => /usr/lib/libseccomp.so.2 (0x0000768f7522c000)
    libc.so.6 => /lib64/libc.so.6 (0x0000768f72614000)
    /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x0000768f75257000)

But this might not be enough to get this working as expected. You might also need to bind in some additional directories, like /etc, /var, etc. Your updated script (with some formatting) could end up looking like:

#!/bin/bash

export NXF_VER=23.04.2
export NXF_ANSI_LOG=false

singularity exec \
    --env NXF_HOME=/home/.nf-singularity \
    -B /data/home/giab_NA12878/Data_LongRead_sequencing/:/input \
    -B /data/home/my_tools/sifs/:/sifs \
    -B /data/home/my_tools/nf_tools:/tools \
    -B /data/home/databases/UCSC_download/:/db \
    -B /bin/singularity \
    -B /usr \
    -B /etc \
    -B /var \
    /data/home/my_tools/sifs/nextflow_latest.sif \
nextflow run /tools/wf-human-variation \
    -c /input/wf_sing_nf.config \
    -profile foo \
    -w /input/nf_test/workspace \
    -with-report /input/nf_test/run_report_hg001_nf.html \
    -with-timeline /input/nf_test/run_timeline_hg001_nf.html \
    --snp \
    --str \
    --cnv \
    --bam /input/giab_NA12878_only_chr_sort.bam \
    --bam_min_coverage 0 \
    --ref /db/human_selected_onlychr.fasta \
    --sample_name giab_hg001_nf \
    --basecaller_cfg [email protected] \
    --out_dir /input/nf_test/HG001_test_nextflow

At this point, I think a better way might just be to install singularity alongside nextflow inside your own container image. But if you have a shared filesystem, you should only have to install nextflow once, and not need to nest containers like this at all.

Upvotes: 1

Related Questions