Jenkins docker removes generated files after stage completion

Question

I am building a jenkins pipeline where I am trying to convert zeeplin/jupyter notebook jsons to md files. I need nb2md installed for converting json to md files so I am using docker image for one of the stage that converts the files however as soon as that stage is completed the dockers removes all the files. I wanted to take this files and commit to git in next stage as the files are gone I am not able to do that. I moved git commit step with docker too but then I get git not found on docker. Below is my jenkins files:

void setBuildStatus(String message, String state) {
  step([
      $class: "GitHubCommitStatusSetter",
      reposSource: [$class: "ManuallyEnteredRepositorySource", url: “$git_url”],
      contextSource: [$class: "ManuallyEnteredCommitContextSource", context: "ci/jenkins/build-status"],
      errorHandlers: [[$class: "ChangingBuildStatusErrorHandler", result: "UNSTABLE"]],
      statusResultSource: [ $class: "ConditionalStatusResultSource", results: [[$class: "AnyBuildResult", message: message, state: state]] ]
  ]);
}
pipeline {
    agent none
    environment {
        PATH = '/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin'
    }
    parameters {
        string(name: 'files_list', defaultValue: '')
    }
    stages {
       stage('Convert Zeppelin/Jupyter json to md files'){
           agent {
              docker {
                image 'python:3-alpine'
                args  '--network=host'
                }
            }
            steps {
                withEnv(["HOME=${env.WORKSPACE}"]) {
                    sh '''pip3 install --target=/home/jenkins/workspace -U nb2md'''
                }
                withCredentials([
                  sshUserPrivateKey(
                    credentialsId: "github-sec-dms-dataops-ssh-key",
                    keyFileVariable: 'keyfile')
            ]){
                sh'''
                #!/bin/bash
                dir=$(pwd)
                sh $dir/jupyter-zeppelin-json-to-md-files.sh
            '''
            }
          }
        }
        stage ('Git commit'){
            agent any
            steps {
             withCredentials([
                  sshUserPrivateKey(
                    credentialsId: "github-sec-dms-dataops-ssh-key",
                    keyFileVariable: 'keyfile')
            ]){
                sh '''
                #!/bin/sh
                 eval `ssh-agent -s`
                 ssh-add $keyfile
                 git config --global user.email “user_id”
                 git config --global user.name "git"
                 git add .
                 echo "Committing files to git hub ... "
                 git commit -am "Committing markdown files.."
                 echo "Pushing code to github ..."
                 git push -u origin HEAD
                 git diff --exit-code'
                 '''}
           }
        }
    }
    post {
        success {
            node('jnlp') {
                setBuildStatus("Build succeeded", "SUCCESS");
            }
        }
        failure {
            node('jnlp') {
                setBuildStatus("Build failed", "FAILURE");
            }
        }
    }
}

How can I get the files generated in stage 1 available for stage 2?

Answer 1

Use archiveArtifacts to keep files between steps. Use args on docker step to mount host folders.

docker {
    image 'python:3-alpine'
    args  '--network=host -v <host_folder>:<container_folder>'
}

post {
    always {
        archiveArtifacts artifacts: 'generatedFile.txt', onlyIfSuccessful: true
    }
}