Pass --build-arg or use environment variables in two-step Dockerfile in Gitlab

Question

My minimum test project layout looks like this.

├── deployment
│   ├── build.sh
│   └── nginx
│       └── nginx.conf
├── Dockerfile
├── next.config.js
├── package.json
├── package-lock.json
└── pages
    ├── _app.js
    └── index.js

Contents of Dockerfile:

FROM node as build-stage

ARG K8S_SECRET_PUB
ENV K8S_SECRET_PUB ${K8S_SECRET_PUB}

ARG SRV
ENV SRV ${SRV}

WORKDIR /app
COPY package*json /app/
RUN npm install --production
COPY ./ /app/
RUN npm run export


FROM nginx:1.15-alpine

RUN rm /etc/nginx/nginx.conf
COPY --from=build-stage /app/out /www
COPY deployment/nginx/nginx.conf /etc/nginx/
EXPOSE 5000

The goal is to have the environment variables K8S_SECRET_PUB and SRV be passed to the build-process. npm run export executes next build && next export to get static files that the nginx server should be serving.

Contents of next.config.js:

require('dotenv').config();

module.exports = {
  serverRuntimeConfig: {
    srv: process.env.SRV
  },
  publicRuntimeConfig: {
    pub: process.env.K8S_SECRET_PUB
  }
};

Contents of pages/_app.js:

import App from 'next/app';
import getConfig from 'next/config';

const { serverRuntimeConfig, publicRuntimeConfig } = getConfig();

class MyApp extends App {
  render() {
    return (
      <div>
        <h1>
          {serverRuntimeConfig.srv || 'SRV not accessible from client :p'}
        </h1>
        <h1>{publicRuntimeConfig.pub || 'PUB not set'}</h1>
      </div>
    );
  }
}

export default MyApp;

When building the docker image locally via docker build --build-arg K8S_SECRET_PUB=puppy --build-arg SRV=serverval -t my_image ., I can start a container via docker run -p 5000:5000 my_image.

Accessing the running container has the expected result. Inspecting the file-system further shows that the passed through build arguments are picked up and the files were written accordingly.

However, when I push this code to Gitlab, the deployed nginx looks like this:

What I would like to accomplish is to have the Environment variables that I defined via Gitlab UI under Settings -> CI/CD be picked up and used in the build-stage defined in the Dockerfile. As we've been otherwise happy with the Auto Dev, we have not created and checked in a .gitlab-ci.yml file yet.

---

Update #1

After tinkering for a little bit, I now have access to the environment variables, but I lost the convenience of Auto DevOps.

I added a deployment/build.sh with this content:

#!/bin/sh
docker build --build-arg K8S_SECRET_PUB="${K8S_SECRET_PUB}" --build-arg SRV="${SRV}" -t my_image .

I also started on a .gitlab-ci.yml which contains this:

stages:
    - build
    - review
    - deploy
    - clean

image: docker:latest

services:
    - docker:dind

build:
    stage: build
    script:
        - sh ./deployment/build.sh
        - mkdir image
        - docker save my_image > image/my_image.tar
    artifacts:
        paths:
            - image

After pushing the repository to Gitlab, the pipeline succeeds and I can download the artifact, unzip it, load it via docker load -i image/my_image.tar and run it. And sure enough, the page loads with the defined variables from the Gitlab CI/CD UI. However, now I've lost all of the other steps of the deployment process (which is the main reason I didn't want to write the .gitlab-ci.yml in the first place).

---

Update #2

Working off the Auto DevOps template, which I found at https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml I made these changes:

• commenting out the line - template: Jobs/Build.gitlab-ci.yml
• replace the alpine with the docker image
• added CODE_QUALITY_DISABLED: "true" to the variables section, because the code quality check was taking too long
• add the services and build sections from my previous attempt described above

Now, I'm stuck on the review stage.

Application should be accessible at: http://*my_image_url*
Waiting for deployment "review-branchname-abcxyz" rollout to finish: 0 of 1 updated replicas are available...

Answer 1

It's been a few years, but I was looking to do the same recently. GitLab now exposes the AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS CI/CD variable.

You can set this in the project's Settings | CI/CD | Variables or in the .gitlab-ci.yml file:

variables:
  AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS: '--build-arg K8S_SECRET_PUB=puppy --build-arg SRV=serverval'

Answer 2

Following Update #2, these are the changes I made to get it working.

Rewrote the .gitlab-ci.yml:

image: docker:latest

variables:
  CI_APPLICATION_TAG: $CI_COMMIT_SHA
  CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG
  CODE_QUALITY_DISABLED: "true"

  KUBERNETES_VERSION: 1.11.9
  HELM_VERSION: 2.13.1

  DOCKER_DRIVER: overlay2

  ROLLOUT_RESOURCE_TYPE: deployment

stages:
  - build
  - test
  - deploy # dummy stage to follow the template guidelines
  - review
  - dast
  - staging
  - canary
  - production
  - incremental rollout 10%
  - incremental rollout 25%
  - incremental rollout 50%
  - incremental rollout 100%
  - performance
  - cleanup

include:
#  - template: Jobs/Build.gitlab-ci.yml
  - template: Jobs/Test.gitlab-ci.yml
  - template: Jobs/Code-Quality.gitlab-ci.yml
  - template: Jobs/Deploy.gitlab-ci.yml
  - template: Jobs/Browser-Performance-Testing.gitlab-ci.yml
  - template: Security/DAST.gitlab-ci.yml
  - template: Security/Container-Scanning.gitlab-ci.yml
  - template: Security/Dependency-Scanning.gitlab-ci.yml
  - template: Security/License-Management.gitlab-ci.yml
  - template: Security/SAST.gitlab-ci.yml

# Override DAST job to exclude master branch
dast:
  except:
    refs:
      - master

services:
    - docker:dind

build:
    stage: build
    script:
        - sh ./deployment/build.sh

Used more from a template I found and rewrote the deployment/build.sh:

if ! docker info &>/dev/null; then
  if [ -z "$DOCKER_HOST" -a "$KUBERNETES_PORT" ]; then
    export DOCKER_HOST='tcp://localhost:2375'
  fi
fi

if [[ -n "$CI_REGISTRY_USER" ]]; then
  echo "Logging to GitLab Container Registry with CI credentials..."
  docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
fi

if [[ -f Dockerfile ]]; then
  echo "Building Dockerfile-based application..."
else
  echo "Building Heroku-based application using gliderlabs/herokuish docker image..."
  cp /build/Dockerfile Dockerfile
fi

docker build --build-arg K8S_SECRET_PUB="${K8S_SECRET_PUB}" --build-arg SRV="${SRV}" --tag "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG" .

docker push "$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG"