How to write a multi-stage Dockerfile without from flag

Question

This is actually the continuation of this question that I asked today.

I have a multi-stage Dockerfile that uses --from flag:

FROM docker.m.our-intra.net/microsoft/dotnet:2.1-sdk
WORKDIR /app
COPY . ./aspnetapp/
WORKDIR /app/aspnetapp
RUN dotnet publish -c Release -o out

FROM docker.m.our-intra.net/microsoft/dotnet:2.1.4-aspnetcore-runtime
WORKDIR /app
COPY --from=docker.m.our-intra.net/microsoft/dotnet:2.1-sdk /app/aspnetapp/MyProject.WebApi/out ./
ENTRYPOINT ["dotnet", "MyProject.WebApi.dll"]

With the help of this file I am able to build the image locally successfully.

BUT I can't use this Dockerfile in my Jenkins pipeline because the Jenkins Server engine is less than 17.05 version and it's not going to be updated (maybe later but not now).

I'm very new in Docker and Jenkins stuff. I would appreciate if anyone can help me to modify the Dockerfile in such way that I can use it without --from flag.

UPDATE:

The upper-mentioned Dockerfile is wrong. The working version of Dockerfile with the help of which I build the image on my local machine successfully and run the app also successfully is as follows:

FROM docker.m.our-intra.net/microsoft/dotnet:2.1-sdk AS build
WORKDIR /app
COPY . ./aspnetapp/
WORKDIR /app/aspnetapp
RUN dotnet publish -c Release -o out

FROM docker.m.our-intra.net/microsoft/dotnet:2.1.4-aspnetcore-runtime AS runtime
WORKDIR /app
COPY --from=build /app/aspnetapp/MyProject.WebApi/out ./
ENTRYPOINT ["dotnet", "MyProject.WebApi.dll"]

UPDATE 2:

I'm trying to follow Carlos advice and now I have two docker files.

This is my Docker-build:

FROM docker.m.our-intra.net/microsoft/dotnet:2.1-sdk
WORKDIR /app
COPY . ./aspnetapp/
WORKDIR /app/aspnetapp
RUN dotnet publish -c Release -o out

This my Dockerfile:

FROM docker.m.our-intra.net/microsoft/dotnet:2.1.4-aspnetcore-runtime
COPY . .
ENTRYPOINT ["dotnet", "MyProject.WebApi.dll"]

This my Jenkinsfile:

def docker_repository_url = 'docker.m.our-intra.net'
def artifact_group = 'some-artifact-group'                  
def artifact_name = 'my-service-api'

pipeline {
    agent {
        label 'build'
    }
    stages {
        stage('Checkout') {
            steps {
                script {
                    echo 'Checkout...'
                    checkout scm
                    echo 'Checkout Completed'
                }
            }
        }
        stage('Build') {
            steps {
                script {
                    echo 'Build...'
                    sh 'docker version'
                    sh 'docker build -t fact:v${BUILD_NUMBER} -f Dockerfile-build .'
                    echo 'Build Completed'
                }               
            }
        }
        stage('Extract artifact') {
            steps {
                script {
                    echo 'Extract...'
                    sh 'docker create --name build-stage-container fact:v${BUILD_NUMBER}'
                    sh 'docker cp build-stage-container:/app/aspnetapp/MyProject.WebApi/out .'
                    sh 'docker rm -f build-stage-container'
                    echo 'Extract Completed'
                }               
            }
        }
        stage('Copy compiled artifact') {
            steps {
                script {
                    echo 'Copy artifact...'
                    sh "docker build -t ${docker_repository_url}/${artifact_group}/${artifact_name}:v${BUILD_NUMBER} -f Dockerfile ."
                    echo 'Copy artifact Completed'
                }               
            }
        }
        stage('Push image') {
            steps {
                script {                                                    
                    withCredentials([[
                        $class: 'UsernamePasswordMultiBinding', 
                        credentialsId: 'jenkins',
                        usernameVariable: 'USERNAME', 
                        passwordVariable: 'PASSWORD'
                    ]]) {
                            def username = env.USERNAME
                            def password = env.PASSWORD

                            echo 'Login...'
                            sh "docker login ${docker_repository_url} -u ${username} -p ${password}"
                            echo 'Login Successful' 

                            echo 'Push image...'
                            sh "docker push ${docker_repository_url}/${artifact_group}/${artifact_name}:v${BUILD_NUMBER}"   
                            echo 'Push image Completed'
                    }                               
                }               
            }
        }
    }
}

All steps are successed but when I try to run the image locally (after pulling it from Maven) or run it on OpehShift cluster it fails and says:

Did you mean to run dotnet SDK commands? Please install dotnet SDK from: http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409

What am I doing wrong?

Answer 1

This my final working solution.

Docker-build:

FROM docker.m.our-intra.net/microsoft/dotnet:2.1-sdk
WORKDIR /app
COPY . ./aspnetapp/
WORKDIR /app/aspnetapp
RUN dotnet publish -c Release -o out

Dockerfile:

FROM docker.m.our-intra.net/microsoft/dotnet:2.1.4-aspnetcore-runtime
ADD output/out /output
WORKDIR /output
ENTRYPOINT ["dotnet", "MyProject.WebApi.dll"]

Jenkinsfile:

def docker_repository_url = 'docker.m.our-intra.net'
def artifact_group = 'some-artifact-group'                  
def artifact_name = 'my-service-api'

pipeline {
    agent {
        label 'build'
    }
    stages {
        stage('Checkout') {
            steps {
                script {
                    echo 'Checkout...'
                    checkout scm
                    echo 'Checkout Completed'
                }
            }
        }
        stage('Build') {
            steps {
                script {
                    echo 'Build...'
                    sh 'docker version'
                    sh "docker build -t sometag:v${BUILD_NUMBER} -f Dockerfile-build ."
                    echo 'Build Completed'
                }               
            }
        }
        stage('Extract artifact') {
            steps {
                script {
                    echo 'Extract...'
                    sh "docker run -d --name build-stage-container sometag:v${BUILD_NUMBER}"
                    sh 'mkdir output'
                    sh 'docker cp build-stage-container:/app/aspnetapp/MyProject.WebApi/out output'
                    sh 'docker rm -f build-stage-container'
                    sh "docker rmi -f sometag:v${BUILD_NUMBER}"
                    echo 'Extract Completed'
                }               
            }
        }
        stage('Copy compiled artifact') {
            steps {
                script {
                    echo 'Copy artifact...'
                    sh "docker build -t ${docker_repository_url}/${artifact_group}/${artifact_name}:v${BUILD_NUMBER} -f Dockerfile ."
                    echo 'Copy artifact Completed'
                }               
            }
        }
        stage('Push image') {
            steps {
                script {                                                    
                    withCredentials([[
                        $class: 'UsernamePasswordMultiBinding', 
                        credentialsId: 'jenkins',
                        usernameVariable: 'USERNAME', 
                        passwordVariable: 'PASSWORD'
                    ]]) {
                            def username = env.USERNAME
                            def password = env.PASSWORD

                            echo 'Login...'
                            sh "docker login ${docker_repository_url} -u ${username} -p ${password}"
                            echo 'Login Successful' 

                            echo 'Push image...'
                            sh "docker push ${docker_repository_url}/${artifact_group}/${artifact_name}:v${BUILD_NUMBER}"   
                            echo 'Push image Completed'
                            sh "docker rmi -f ${docker_repository_url}/${artifact_group}/${artifact_name}:v${BUILD_NUMBER}"
                    }

                }               
            }
        }
    }
}

Answer 2

@Carlos answer is perfectly valid. However as you are using jenkins and pipelines you might be happy with the following alternative solution:

If you are using jenkins with dynamic pod-provisioning on a kubernetes-distribution you can do the following:

• Use a pod-template for your build which is based on <registry>/microsoft/dotnet:2.1-sdk. Compile your application within that pod in regular dotnet-way.
• Keep the second part of your Dockerfile, but just copy the compiled artifact into the docker-image.

In summary you move out the first part of your Dockerfile into the Jenkinsfile to do the application build. The second part remains to do the docker-build from the already compiled binary.

The Jenkinsfile would look similar to this:

podTemplate(
    ...,
    containers: ['microsoft/dotnet:2.1-sdk', 'docker:1.13.1'],
    ...
) {
    container('microsoft/dotnet:2.1-sdk') {
        stage("Compile Code") {
            sh "dotnet restore"
            sh "dotnet publish -c Release -o out"
        }
    }
    container('docker:1.13.1') {
        stage("Build Docker image") {
            docker.build("mydockerimage:1.0")
        }
    }
}

This Jenkinsfile is far from complete and only illustrates how it would work. Find more documentation here:

Jenkins kubernetes plugin

Jenkins docker global variable in scripted pipeline

Answer 3

TL;DR: You need to replicate the underlying functionality yourself, outside of Docker

Firstly, you are using the --from option wrong. To copy from a previous build stage, you must refer to its index or its name, e.g.:

FROM docker.m.our-intra.net/microsoft/dotnet:2.1-sdk
...

FROM docker.m.our-intra.net/microsoft/dotnet:2.1.4-aspnetcore-runtime
COPY --from=0 /app/aspnetapp/MyProject.WebApi/out ./

or

FROM docker.m.our-intra.net/microsoft/dotnet:2.1-sdk AS build-stage
...
FROM docker.m.our-intra.net/microsoft/dotnet:2.1.4-aspnetcore-runtime
COPY --from=build-stage /app/aspnetapp/MyProject.WebApi/out ./

With your current Dockerfile, it would try to copy the file from the upstream docker image, not from the previous build stage.

Secondly, you can't do multi-stage Docker builds with a version prior to 17.05. You need to replicate the underlying functionality yourself, outside of Docker.

To do so, you can have one Dockerfile to build your artifact and run a throwaway container based on that image, from which to extract the artifact. You don't need to run the container, you can simply create it with docker create (this creates the writeable container layer):

docker create --name build-stage-container build-stage-image
docker cp build-stage-container:/app/aspnetapp/MyProject.WebApi/out .

Then you can have a second Dockerfile to build an image copying the artifact extracted from the previous stage, with a simple COPY from the build context.