How to use multiple docker repositories in a Jenkins pipeline

Question

I have a Jenkins pipeline in which I need to log into two different docker repositories. I know how to authenticate to one repo using the following command

docker.withRegistry('https://registry.example.com', 'credentials-id')

but don't know how to do it for more than 1 repo?

Answer 1

While the other answers give some nice options for using scripted pipeline syntax to run multiple docker repositories, there is also built in support for declarative syntax.
Jenkins declarative pipeline syntax supports both running a job on a docker agent taken from a custom registry (repository), or using a dynamically built dockerfile that can be based on images from a custom registry.

Here is some more info on the two options:

1. Docker agent

Execute the Pipeline, or stage, with the given container which will be dynamically provisioned on a node pre-configured to accept Docker-based Pipelines, or on a node matching the optionally defined label parameter.
docker also optionally accepts a registryUrl and registryCredentialsId parameters which will help to specify the Docker Registry to use and its credentials. The parameter registryCredentialsId could be used alone for private repositories within the docker hub. For example:

agent {
    docker {
        image 'myregistry.com/node'
        label 'my-defined-label'
        registryUrl 'https://myregistry.com/'
        registryCredentialsId 'myPredefinedCredentialsInJenkins'
    }
}

2. dockerfile

Execute the Pipeline, or stage, with a container built from a Dockerfile contained in the source repository. In order to use this option, the Jenkinsfile must be loaded from either a Multibranch Pipeline or a Pipeline from SCM.
dockerfile also optionally accepts a registryUrl and registryCredentialsId parameters which will help to specify the Docker Registry to use and its credentials. For example:

agent {
    dockerfile {
        filename 'Dockerfile.build'
        dir 'build'
        label 'my-defined-label'
        registryUrl 'https://myregistry.com/'
        registryCredentialsId 'myPredefinedCredentialsInJenkins'
    }
}

You can then use both of these options per stage, which means using a different image and registry per stage:

pipeline {
   stages {
      stage('First') {
         agent {
            docker {
               ...  // First registry
            }
         }
         steps {
            ...
         }
      }
      stage('Second') {
         agent {
            docker {
               ...  // Second registry
            }
         }
         steps {
            ...
         }
      }
   }
}

If needed, the declarative syntax can be combined with the scripted syntax by using a script block inside one of the stages, in which the scripted syntax can be used. For example:

pipeline {
    agent {
        docker {
            image 'myregistry.com/node'
            registryUrl 'https://myregistry.com/'
            registryCredentialsId 'myPredefinedCredentialsInJenkins'
        }
    }
    stages {
        stage("Build") {
            steps {
                script {
                    echo 'Using custom registry https://myregistry2.com/'
                    docker.withRegistry('https://myregistry2.com/', 'myPredefinedCredentialsInJenkins2') {
                        // Use the second registry here
                    }
                }
            }
        }
    }
}

---

---

---

If all the built in methods are not working for you as expected, you can always create an implementation of your own, and use it across your pipeline via a shared library.
For example, you can create the following shared lib method in a useDockerRegistry.groovy file:

def call(String registry, String credentialsId, Closure body) {
    println "Log in to ${registry}"
    withCredentials([usernamePassword(credentialsId: 'credentialsId', usernameVariable: 'DOCKER_USERNAME', passwordVariable: 'DOCKER_PASSWORD')]) {
        sh "docker login -u \$DOCKER_USERNAME -p \$DOCKER_PASSWORD ${registry}"
    }
    try {
        body()
    }
    finally {
        println "Log out from ${registry}"
        sh "docker logout ${registry}"     // clean up the session
    }
}

Then in your pipelines you can use it for a single registry or for multiple nested ones.
For Example:

library "myLib"

pipeline {
   agent any
   stages {
      stage('Single Registry') { 
         steps {
            useDockerRegistry('https://myregistry.com/', 'registryCredentials') {
               ... // code using myregistry
            }
         }
      }
      stage('Multiple Registries') {
         steps {
            useDockerRegistry('https://myregistry.com/', 'registryCredentials') {
                useDockerRegistry('https://myregistry2.com/', 'registry2Credentials') {
                ... // code using myregistry & myregistry2
                }
            }
         }
      }
   }
}

You can nest as many useDockerRegistry calls as you want.
In addition, you can easily modify the logic of the function according to your needs.

Answer 2

The following worked for me, note this line:

sh "docker login -u \$NEXUSUSER -p \$NEXUSPASS base_docker_images.example.ir"

Full Jenkinsfile: ‍‍‍‍

pipeline {
    agent any
    stages {
      stage('Build docker image') {                    
        steps {
          withCredentials([usernamePassword(usernameVariable: 'NEXUSUSER',
                           passwordVariable: 'NEXUSPASS', credentialsId: 'nexus_docker')]) {
            script {
              checkout([$class: 'GitSCM',
                        branches: [[name: "*/${params.branch}"]],
                        doGenerateSubmoduleConfigurations: false,
                        extensions: [[$class: 'CleanCheckout']],
                        submoduleCfg: [],
                        userRemoteConfigs: [[credentialsId: 'user_gitlab', url: 'https://example.ir/infra/repo.git']]
                      ])
               docker.withRegistry('https://nexus.example.ir', 'nexus_docker') {
                 sh "docker login -u \$NEXUSUSER -p \$NEXUSPASS base_docker_images.example.ir"
                 def customImage = docker.build("infra/image:${params.version}")
                 customImage.push()
               }
            }
         }
       }
     }
   }
}

Thanks!

Answer 3

Following is a Jenkins pipeline for using multiple docker repositories, authenticating, building and pushing them.

pipeline {
    agent any
    
    environment {
        DOCKER_REGISTRY_1 = 'https://first.registry.com'
        DOCKER_REGISTRY_2 = 'https://second.registry.com'
        CREDENTIALS_ID_1 = 'credentials-id-1'
        CREDENTIALS_ID_2 = 'credentials-id-2'
    }

    stages {
        stage('Build and Push Image to Registry 1') {
            steps {
                script {
                    // Authenticate with the first Docker registry
                    docker.withRegistry(DOCKER_REGISTRY_1, CREDENTIALS_ID_1) {
                        // Build and push your Docker image to the first registry
                        docker.build('your-image-name:tag').push()
                    }
                }
            }
        }

        stage('Build and Push Image to Registry 2') {
            steps {
                script {
                    // Authenticate with the second Docker registry
                    docker.withRegistry(DOCKER_REGISTRY_2, CREDENTIALS_ID_2) {
                        // Build and push your Docker image to the second registry
                        docker.build('your-image-name:tag').push()
                    }
                }
            }
        }

        // Add more stages if required
    }
}

Using the above pipeline should fix the issue for you.

Answer 4

Nesting docker.withRegistry calls actually works as expected. Each call adds an entry to /home/jenkins/.dockercfg with provided credentials.

// Empty registry ('') means default Docker Hub `https://index.docker.io/v1/`
docker.withRegistry('', 'dockerhub-credentials-id') {
  docker.withRegistry('https://private-registry.example.com', 'private-credentials-id') {
    // your build steps ...
  }
}

This allows you to pull base images from Docker Hub using provided credentials to avoid recently introduced pull limits, and push results into another docker registry.

Answer 5

This is a partial answer only applicable when you are using two registries but only need credentials on one. You can nest the calls since they mostly just do a docker login that stays active for the scope of the closure and will add the registry domain name into docker pushes and such.

Use this in a scripted Jenkins pipeline or in a script { } section of a declarative Jenkins pipeline:

docker.withRegistry('https://registry.example1.com') { // no credentials here
    docker.withRegistry('https://registry.example2.com', 'credentials-id') { // credentials needed
        def container = docker.build('myImage')
        container.inside() {
            sh "ls -al" // example to run on the newly built 
        }
    }
}

Sometimes you can use two, non-nested calls to docker.withRegistry() one after the other but building is an example of when you can't if, for example, the base image for the first FROM in the Dockerfile needs one registry and the base image for a second FROM is in another registry.