Docker-compose env file not working

Question

I'm writing as docker-compose file to up MySQL instance and want to use few variable from env file: here are the files actually look like:

docker-compose.yml

version: '3.3'
services:
  db:
    image: mysql
    restart: always
    env_file:
      - ./imran.env
    environment:
      MYSQL_ROOT_PASSWORD: ${PASS}
    ports:
      - ${PORT1}: ${PORT2}

imran.env

PASS=imran123
PORT1=3306
PORT2=3306

Instead of working correct i'm getting following errors:

WARNING: The PASS variable is not set. Defaulting to a blank string.
WARNING: The PORT2 variable is not set. Defaulting to a blank string.
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.db.ports contains unsupported option: '${PORT1}

Please Help

Answer 1

I've been having issues with this today and I found the solution to it.

In the .env file, you define the variables that you need:

TEST='I have single quotes because I have special character$'

In the compose file, you must define the same variable but unset. Also, without using the equal sign:

services:
  app:
    environment:
    - TEST

Inside the container, you'll be able to run echo over your variable and see that it has been set to what you defined in the .env file.

Answer 2

It seems like the question have been there for a while, but for a curios posterity (like myself) the correct answer (as yours truly considers the one to be, at least) would be as follows:

1. As other good people mentioned in this discussion, env_file property of docker-compose.yml is setting environment variables for containers, not for docker-compose file, and is similar to docker run --env-file;

2. By default, environment variables in docker compose file is set by .env file in the same folder;

3. To change this default file, start your services with docker compose --env-file specifying an alternative .env file.

The answer comes directly from here.

Also, please note that, as mentioned here, passing an alternative dotenv file overwrites the default .env.

Answer 3

In my case it was enough to add the parameter...

    env_file:
      - ./.env-MY

... in docker-compose.yml for the desired service.

PLUS:

The name of the .env file (.env-MY, my case) is of no importance, it can be any name.

The --env-file ./.env-MY parameter will not be needed for the docker-compose command.

It is NOT necessary to declare environment variables inside docker-compose.yml, as per this example...

    environment:
      - ENV_VAR_A:${ENV_VAR_A}
      - ENV_VAR_B:${ENV_VAR_B}
      - ENV_VAR_C:${ENV_VAR_C}
      - ENV_VAR_D:${ENV_VAR_D}
      - ENV_VAR_E:${ENV_VAR_E}
      - ENV_VAR_F:${ENV_VAR_F}
      - ENV_VAR_G:${ENV_VAR_G}
      - ENV_VAR_H:${ENV_VAR_H}

This is the example content of my .env-MY...

#!/bin/bash

ENV_VAR_A="env_var_a_val"
ENV_VAR_B="env_var_b_val"
ENV_VAR_C="env_var_c_val"
ENV_VAR_D="env_var_d_val"
ENV_VAR_E="env_var_e_val"
ENV_VAR_F="env_var_f_val"
ENV_VAR_G="env_var_g_val"
ENV_VAR_H="env_var_h_val"

To test whether the environment variables were loaded run the command...

docker-compose exec <SERVICE> env

... in the desired service.

[Ref(s).: https://stackoverflow.com/a/34051804/3223785 ]

Answer 4

Had a similar issue. I had my environment variables in a .env and was using the following in the docker -compose.yml file.

environment:
  - DJANGO_SECRET_KEY:${DJANGO_SECRET_KEY}
  - ALLOWED_HOSTS:${ALLOWED_HOSTS}

Changed it to

environment:
  - DJANGO_SECRET_KEY=${DJANGO_SECRET_KEY}
  - ALLOWED_HOSTS=${ALLOWED_HOSTS}

And it worked. Notice the symbol : replaced with = Using version 3.8

Answer 5

According to this docker manual, if environment variables are used in docker-compose.yml file like:

web:
  image: "webapp:${TAG}"

or

environment:
  MYSQL_ROOT_PASSWORD: ${PASS}

Docker looks for those variables in a default .env file; So changing the default env file like docker-compose up --env-file imran.env would work.

Answer 6

Use docker compose up Does not work on legacy docker-compose up

Answer 7

i had a similar problem but a bit different.

the mysql container failed to boot up because the .env beside the docker-compose.yml was not being picked up by docker-compose.

this probably happened because i had another file beside them called .env.example which confused docker-compose.

i resolved this by moving the .env.example file to another folder, but i also tried explicitly specifying the .env file in the docker-compose.yml which also worked.

i.e:

    env_file:
      - .env

i did not have a need for an environment key in my docker-compose.yml, which makes my answer less relevant to this question but i think its close enough to help others that find this question.

Answer 8

The .env file in the project root, and the env_file: field in the Compose file are two different concepts.

The .env is for settings a default environment for Compose. Values set in this file can be used within the Compose file.

The env_file: field is for setting the default environment for a container. Values set in this can be used in the container, but not in the Compose file.

See https://docs.docker.com/compose/env-file/ for more information.

Answer 9

@nickgryg answer is good but here I will explain what I had faced when working with env. It will help others to understand.

I was also not able to access .my-project.env some variables inside docker-compose file.

.my-project.env

ROCKETTYPE=SIMPLE
RANGE=15

---

docker-compose.yml

   env_file:
      - .my-project.env
    environment:
      - SPACEX:${ROCKETTYPE}
   # It was importing RANGE env not SPACEX inside my container.

When I ran docker-compose up -d, only RANGE env accessible in my container.

---

Then I came up with two solutions :

1st : Instead of using .my-project.env file only use .env file name in docker-compose.

2nd : I did not change my env file name. Only passing .my-project.env in docker-compose command.

docker-compose --env-file .my-project.env  up -d 

---

Docker-Compose version : docker-compose version 1.25.0

Read official docker documentations

Answer 10

In my case, I was running docker-compose up from a sub directory inside the project(by mistake). So, make sure that the docker-compose.yml file and .env in the same directory, and that you are running docker-compose up in the same directory where both of the files exists.

Answer 11

I recently had to specify that I wanted my container to use .env for its environment file like:

version: '2.4'

services:

  myapp:
    build:
      context: .
    container_name: myapp01xj1
    env_file:
      - .env

Answer 12

It seems that env_file is ignored when you add environment after it. In my experience, docker-compose seems to have numerous "silly" bugs like this.

Your docker-compose.yaml has a few errors which are irrelevant to env files. I corrected them a bit.

$ tre
.
├── broken
│   ├── imran.env
│   └── docker-compose.yaml
└── works
    ├── docker-compose.yaml
    └── imran.env

$ bat broken/*
───────┬──────────────────────────────────────────────
       │ File: broken/docker-compose.yaml
───────┼──────────────────────────────────────────────
   1   │ version: '3.3'
   2   │ services:
   3   │   db:
   4   │     image: mysql
   5   │     restart: always
   6   │     env_file:
   7   │       - ./imran.env
   8   │     environment:
   9   │       MYSQL_ROOT_PASSWORD: ${PASS}
───────┴──────────────────────────────────────────────
───────┬──────────────────────────────────────────────
       │ File: broken/imran.env
───────┼──────────────────────────────────────────────
   1   │ PASS=imran123
   2   │ MYSQL_ROOT_PASSWORD=changeme
   3   │ PORT1=3306
   4   │ PORT2=3306
───────┴──────────────────────────────────────────────

$ diff broken works
diff broken/docker-compose.yaml works/docker-compose.yaml
8,9d7
<     environment:
<       MYSQL_ROOT_PASSWORD: ${PASS}

With environment, it fails:

$ cd broken && docker-compose down && docker-compose up -d
WARNING: The PASS variable is not set. Defaulting to a blank string.
Stopping broken_db_1 ... done
Removing broken_db_1 ... done
Removing network broken_default
WARNING: The PASS variable is not set. Defaulting to a blank string.
Creating network "broken_default" with the default driver
Creating broken_db_1 ... done

But without environment, it works:

$ cd works && docker-compose down && docker-compose up -d
Removing works_db_1 ... done
Removing network works_default
Creating network "works_default" with the default driver
Creating works_db_1 ... done

$ docker exec -i -t works_db_1 env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=ac76b84ca072
TERM=xterm
PASS=imran123
MYSQL_ROOT_PASSWORD=changeme
PORT1=3306
PORT2=3306
GOSU_VERSION=1.12
MYSQL_MAJOR=8.0
MYSQL_VERSION=8.0.21-1debian10
HOME=/root

Notes:

• Docker-compose apparently doesn't like ${PORT1} in ports:, you have to give the actual number. (in this case I just deleted it because ports are irrelevant for env files)
• The MySQL image crashes if you don't provide a root pass. I added one so the container could run long enough for me to dump its env and confirm that the variables are being set.
• I am using Docker version 19.03.5 on macOS Catalina 10.15.4.

Answer 13

You have some issues in your docker-compose.yaml file:

A:

A space symbol between ports values. It should be without a space:

ports:
  - ${PORT1}:${PORT2}

B:

You need to use .env file in folder where docker-compose.yaml is in order to declaring default environment variables for both docker-compose.yaml file and docker container. env_file section is used to put values into container only.

So, you should do the following:

1.

Re-name file with ENV variables to .env:

mv imran.env .env

2.

Use the following docker-compose.yaml after:

version: '3.3'
services:
  db:
    image: mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: ${PASS}
    ports:
      - ${PORT1}:${PORT2}