CODEWITHSUNDEEP
docker-composeenvironment-variables
logicOnAbstractions
logicOnAbstractions

Reputation: 2580

docker-compose environment variable loading .env, but not env_file from compose file

I don't really make sense of docker-compose's behavior with regards to environment variables files.

I've defined a few variables for an simple echo server setup with 2 flask application running.

In .env:

FLASK_RUN_PORT=5000
WEB_1_PORT=80
WEB_2_PORT=8001

Then in docker-compose.yml:

version: '3.8'

x-common-variables: &shared_envvars
  FLASK_ENV: development
  FLASK_APP: main.py
  FLASK_RUN_HOST: 0.0.0.0
  COMPOSE_PROJECT_NAME: DOCKER_ECHOES

x-volumes: &com_volumes
  - .:/project          # maps the current directory, e.g. project root that is gitted, to /proj in the container so we can live-reload

services:
  web_1:
    env_file: .env
    build:
      dockerfile: dockerfile_flask
      context: .
    ports:
      - "${WEB_1_PORT}:${FLASK_RUN_PORT}"       # flask runs on 5000 (default). docker-compose --env-file .env up loads whatever env vars specified & allows them to be used this way here.
    volumes: *com_volumes
    environment:
      <<: *shared_envvars   # DRY: defined common stuff in a shared section above, & use YAML merge language syntaxe to include that k-v mapping here. pretty neat.
      FLASK_NAME: web_1
  web_2:
    env_file: .env
    build:
      dockerfile: dockerfile_flask
      context: .
    ports:
      - "${WEB_2_PORT}:${FLASK_RUN_PORT}"       # flask by default runs on 5000 so keep it on container, and :8001 on host
    volumes: *com_volumes
    environment:
      <<: *shared_envvars
      FLASK_NAME: web_2

If I run docker-compose up with the above, everything works as expected.

However, if I simply change the name of the file .env for, say, flask.env, and then accordingly change both env_file: .env to env_file: flask.env, then I get:

(venv) [fv@fv-hpz420workstation flask_echo_docker]$ docker-compose up
WARNING: The WEB_1_PORT variable is not set. Defaulting to a blank string.
WARNING: The FLASK_RUN_PORT variable is not set. Defaulting to a blank string.
WARNING: The WEB_2_PORT variable is not set. Defaulting to a blank string.
ERROR: The Compose file './docker-compose.yml' is invalid because:

So obviously the envvars defined in the file were not loaded in that case. I know that according to the documentation, the section environement:, which I am using, overrides what is loaded in the env_file:. But those aren't the same variables. And at any rate, if that was the issue, it shouldn't work either with the first way, right?

What's wrong with the above?

Upvotes: 4

Views: 5849

Answers (1)

logicOnAbstractions
logicOnAbstractions

Reputation: 2580

Actually, the env_file is loaded AFTER the images have been built. We can verify this. With the code I have posted above, I can see that env_file.env has not been loaded at build time, because of the error message that I get (telling me WEB_PORT_1 is not set etc.).

But this could simply be that the file is never loaded. To rule that out, we build the image (say by providing the missing arguments with docker-compose build -e (...), then we can verify that it is indeed loaded (by logging its value in the flask application in my case, or a simple print to screen etc.).

This means the the content of env_file is available to the running container, but not before (such as when building the image).

If those variables are to be used within the docker-compose.yml file at BUILD time this file MUST be named .env (unless there is a way to provide a name other than the default, but if so I haven't found any). This is why changing env_file: flask.env to env_file: .env SEEMED to make it work - but the real reason why it worked then was because my ports were specified in a .env with the default name that docker-compose parses anyways. It didn't care that I specified it in docker-compose.yml file or not.

To summarize - TL;DR

  • If you need to feed environment variables to docker-compose for build-time, you must store them in a .env. No further actions needed, other than ensure this file is in the same directory as the docker-compose.yml. You can't change the default name .env
  • To provide envars at container run-time, you can put them in foo.env and then specify env_file:foo.env.
  • For run-time variable, another option is to specify them environment: [vars], if just hard-coding them in the docker-compose.yml is acceptable.. According to doc (not tested) those will override any variables also defined by the env_file

Upvotes: 19

Related Questions