Jenkins scripted pipeline - multiple execution with passing artifacts between stages

Question

I was searching and could not find proper information on how to resolve the issue I have with copying the artifacts to jobs that are being executed multiple times in parallel.

I have defined scripted pipeline which executes predefined jobs in stages some which are run in parallel as follows:

• the main pipeline job is located in this structure: /jenkins/workspace/<main_job> these jobs are also preparing artifacts and I later copy them to different stage/jobs in same pipeline with the build id of the executed job.

    node() {
        stage("Creating Build") {
        def stages = [:]
        failFast: true
        stages["Core"] = {
            copyArtifacts(projectName: <job to copy from>, flatten: true, target: '../' + coreBuildJob)
            buildCore = build job: coreBuildJob
            }
        }
        stages["Content"] = {
            copyArtifacts(projectName: <job to copy from>, flatten: true, target: '../' + contentBuildJob)
            buildContent = build job: contentBuildJob
        }
        parallel(stages)
    }
  

I am using the CopyArtifact Plugin to copy the artifacts that were created but it appears that:

1. it copies the file to main job folder in the instance.
2. because of the different workspace/project location I was needed to define the 'target' location to properly copy the artifacts to required job that I execute in the script prior the jobs execution.

e.g for coreBuildJob in the Core stage:

`copyArtifacts(projectName: <job to copy from>, flatten: true, target: '../' + <job_for_execution>)`

This does helps me to resolve the issue with copying the required artifacts by these jobs but I end up with another issue in this case:

When I want this scripted pipeline job to be executed multiple times with different parameters.

The issue is that when the pipeline is executed for the 2nd time and the job that is run in one of the stages runs the 2nd time it creates the following path on the local machine:

`/jenkins/workspace/test_jobs/<job_for_execution>@2`

That means that what I have in my script is not correct, because it copies the files to:

`/jenkins/workspace/test_jobs/<job_for_execution>`

it does not copy the artifacts to proper location and they are not accessible from the executed job.

I thought of having the copyArtifacts part to be executed during the 'build job' command(as you can define in Jenkins UI with passing BUILD_ID as variable to copy artifacts like that) but I cant find any details regarding this to achieve the same behavior with the script.

How can this issue be resolved?

Answer 1

You can use stash/unstash.

After running your build, you can stash:

stash name:'data', includes: './*'

where data is the name (an identifier) and includes can be a directory, subdirectory or single file.

Then, in the stages you want to have the output of your build, use unstash:

unstash 'data'

After doing unstash, the files will be also in respective folder and you can run your other steps.

Refer to https://www.jenkins.io/doc/pipeline/examples/ for more information.