CODEWITHSUNDEEP
jenkinsanacondajenkins-pipeline
Maarten-vd-Sande
Maarten-vd-Sande

Reputation: 3711

Anaconda in Jenkinsfile

Since the tests we are running are getting longer and longer, I thought it would be a good idea to switch from Travis CI to Jenkins (on my local computer). Setting up Jenkins was relatively straightforward, however getting my Jenkinsfile to 'work' not so much. I am trying to download miniconda -> install miniconda -> install an environment -> activate the environment -> run commands from that environment. This is what I got so far:

  environment {
    PATH = "$WORKSPACE/miniconda/bin:$PATH"
  }

  stages {
    stage('setup miniconda') {
        steps {
            sh '''#!/usr/bin/env bash
            wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
            bash miniconda.sh -b -p $WORKSPACE/miniconda
            hash -r
            conda config --set always_yes yes --set changeps1 no
            conda update -q conda

            # Useful for debugging any issues with conda
            conda info -a
            conda config --add channels defaults
            conda config --add channels conda-forge
            conda config --add channels bioconda

            # create snakemake-workflows env
            conda init bash
            conda env create -f envs/snakemake-workflows.yaml
            '''
        }
    }
    stage('Test downloading') {
        steps {
            sh '''#!/usr/bin/env bash
            conda init bash
            conda activate miniconda/envs/snakemake-workflows/
            snakemake -s workflows/download_fastq/Snakefile --directory workflows/download_fastq -n -j 48 --quiet
            '''
        }
    }

The installation of miniconda seems to work, however the next step in stage test downloading results in the error:

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

Currently supported shells are:
  - bash
  - fish
  - tcsh
  - xonsh
  - zsh
  - powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

How do I proceed from this? I need to init my conda, however it seems to have no effect (I do it when creating the environment, and when trying to activate it).

Upvotes: 2

Views: 4012

Answers (1)

Mart
Mart

Reputation: 101

If conda is installed in silent mode (-b flag), then it will not edit your bashrc (see https://docs.anaconda.com/anaconda/install/silent-mode/). So in the Test downloading stage you will have to manually init it again.

This should work: 

stage('Test downloading') {
            steps {
                sh '''#!/usr/bin/env bash
                source $WORKSPACE/miniconda/etc/profile.d/conda.sh
                conda activate miniconda/envs/snakemake-workflows/
                snakemake -s workflows/download_fastq/Snakefile --directory workflows/download_fastq -n -j 48 --quiet
                '''
            }
        }

Upvotes: 4

Related Questions