docker-compose file doesn't pick up environment variable from a dedicated env file causing the seeding of database to fail

Question

Host System

• Windows 10 Pro

Software

• Docker Desktop for Windows 10
• docker --version Docker version 20.10.8, build 3967b7d
• docker-compose --version docker-compose version 1.29.2, build 5becea4c

Scenario

Provide the following:

• I wish to seed a MongoDB Container with some pre-defined JSON data that I have.
• I am aware it is possible to insert data using mongoinsert
• I wish to create a stack of the database and other apps dependent on it and seed the data upon bringing this stack up

Repository Structure

.
├── docker-compose.yml
├── mongo_seed
│   ├── Dockerfile
│   ├── factors.json
│   └── filters.json
└── .env

Code Snippets

within the mongo_seed directory the following Dockerfile exists:

FROM mongo:3.6.21

COPY factors.json /factors.json

COPY filters.json /filters.json

# ASSUMING THIS INFORMATION COMES FROM docker-compose file
ARG database_uri
ENV DATABASE_URI=$database_uri


CMD mongoimport --uri $DATABASE_URI --collection factors --drop --file /factors.json --jsonArray

CMD mongoimport --uri $DATABASE_URI --collection filters --drop --file /filters.json --jsonArray

The docker-compose.yml file in the root is as follows:

version: "3.8"

services:
    # MongoDB
    mongo:
        container_name: mongodb
        image: mongo:latest
        env_file:
            - .env
        ports:
            - "27017:27017"
        networks:
            - "qualiexplore_net"

    # Initial Seed to QualiExplore Database
    mongo-seed:
        env_file:
            - .env
        build:
            context: ./mongo_seed
            dockerfile: Dockerfile
            args:
                database_uri: ${DATABASE_URI}
        networks:
            - "qualiexplore_net"
        depends_on:
            - mongo

networks:
    qualiexplore_net:
        external: true

qualiexplore_net is created externally via

docker network create qualiexplore_net

The .env file is a dummy file:

MONGO_INITDB_ROOT_USERNAME=root
MONGO_INITDB_ROOT_PASSWORD=example
MONGO_INITDB_DATABASE=qualiexplore
DATABASE_URI=mongodb://root:example@mongodb:27017/qualiexplore

Errors

Standalone Build

The build within the mongo_seed directory produces no problems when I execute:

docker build --build-arg  database_uri=mongodb://root:example@mongodb:27017/qualiexplore .

Build via docker-compose

docker compose up # from root directory

produces the following error:

qualiexplore-stack-mongo-seed-1  | 2021-11-08T15:40:31.401+0000 error parsing command line options: error parsing uri (uri): scheme must be "mongodb" or "mongodb+srv"
qualiexplore-stack-mongo-seed-1  | 2021-11-08T15:40:31.401+0000 try 'mongoimport --help' for more information

This implies that that the database_uri as build argument passed within the the service of mongo-seed is not being applied into the docker container.

Verifications

I executed:

docker compose config

and the environment variables from the .env file seem to taken into consideration for each services in the compose file

services:
  mongo:
    container_name: mongodb
    environment:
      DATABASE_URI: mongodb://root:example@mongodb:27017/qualiexplore
      MONGO_INITDB_DATABASE: qualiexplore
      MONGO_INITDB_ROOT_PASSWORD: example
      MONGO_INITDB_ROOT_USERNAME: root
    env_file:
    - .env
    image: mongo:latest
    networks:
      qualiexplore_net: null
    ports:
    - mode: ingress
      target: 27017
      published: 27017
      protocol: tcp
  mongo-seed:
    build:
      context: ./mongo_seed
      dockerfile: Dockerfile
      args:
        database_uri: mongodb://root:example@mongodb:27017/qualiexplore
    depends_on:
      mongo:
        condition: service_started
    environment:
      DATABASE_URI: mongodb://root:example@mongodb:27017/qualiexplore
      MONGO_INITDB_DATABASE: qualiexplore
      MONGO_INITDB_ROOT_PASSWORD: example
      MONGO_INITDB_ROOT_USERNAME: root
    env_file:
    - .env
    networks:
      qualiexplore_net: null
networks:
  qualiexplore_net:
    name: qualiexplore_net
    external: true

Is there a specific syntax that is missing in order to pass the $DATABASE_URI into the CMD mongoinsert ... command in the Dockerfile?

Is there any appropriate way to seed the database without having to hard-code the URI in this scenario? I am currently taking this approach, so as to make the stack configurable by only changing the Environment Variable Files and not having to tweak the internal docker-compose configurations.

Answer 1

I was able to solve this issue thanks to a recent answer on a SE Query

It essentially describes the following steps for passing build arguments from an .env file passed through docker-compose.yml file:

1. Pass the following as build argument for the mongo-seed service:

      mongo-seed:
       env_file:
           - .env
       build:
           context: ./mongo_seed
           dockerfile: Dockerfile
           args:
               - DATABASE_URI=$DATABASE_URI
   

2. the DATABASE_URI will be an ARG in the DockerFile which will be then used as a run-time environment variable

   #..
   # Will be set through Environment Files
   ARG DATABASE_URI
   
   ENV DATABASE_URI ${DATABASE_URI}
   # ...
   
   

3. I had to adapt my Dockerfile to accept an updated version of mongoimport hence I had to bump the version to mongo:5.0 and adapted the file accordingly

    FROM mongo:5.0
    # Will be set through Environment Files
    ARG DATABASE_URI
    ARG USERNAME
    ARG PASSWORD
   
    ENV DATABASE_URI ${DATABASE_URI}
    ENV USERNAME ${USERNAME}
    ENV PASSWORD ${PASSWORD}
   
    COPY factors.json /factors.json
   
    COPY filters.json /filters.json
   
    CMD mongoimport --username ${USERNAME} --password ${PASSWORD} --uri ${DATABASE_URI} --collection factors --drop --file /factors.json && \
        mongoimport --username ${USERNAME} --password ${PASSWORD} --uri ${DATABASE_URI} --collection filters --drop --file /filters.json
   
   

4. build the image from root:

   docker-compose build mongo-seed
   

5. Bring the stack up using:

   docker-compose up
   

docker-compose.yml

version: "3.8"

services:
    # MongoDB
    mongo:
        container_name: mongodb
        image: mongo:latest
        env_file:
            - .env
        #volumes:
        #    - ./data:/data/db
        ports:
            - "27017:27017"
        networks:
            - "qualiexplore_net"

    # Initial Seed to QualiExplore Database
    mongo-seed:
        env_file:
            - .env
        build:
            context: ./mongo_seed
            dockerfile: Dockerfile
            args:
                - DATABASE_URI=$DATABASE_URI
                - USERNAME=$MONGO_INITDB_ROOT_USERNAME
                - PASSWORD=$MONGO_INITDB_ROOT_PASSWORD
        depends_on:
            - mongo
        networks:
            - "qualiexplore_net"


networks:
    qualiexplore_net:
        external: true