CODEWITHSUNDEEP
dockerdocker-compose
Johan
Johan

Reputation: 40510

Different env-file but same yml with Docker Compose?

I find it quite common that I have multiple environments (for example test and prod) but the Docker containers that I wish to start are the same in both environments. The only difference is the application configuration which I want to specify using an env-file. Since I have multiple containers and dependencies between them I want to use docker-compose. But afaik I can only specify an env-file inside the docker-compose.yml file (see docs). If this is the case then I need to clone my original docker-compose.yml to two different files (one for test and one for prod) just to point to different env files. This means that I have to maintain two docker-compose.yml files instead of one and if I make any changes I need to update both files.

Is this really according to design? Why won't docker-compose let me specify --env-file when I do docker-compose up or docker-compose run?

Upvotes: 31

Views: 23529

Answers (3)

posita
posita

Reputation: 912

It isn't a direct inclusion from the command line, but if you need a work-around before the #1765 merge (the fix for #1377) makes it into a release, you can use the extends directive along with the env_file directive. For convenience, the files from the simple examples below are reproduced in this repository.

Stupid simple example

base.yml

base:
    image: busybox
    command: bash -c 'echo "${WHO:-Simon} says, \"${SHOUTOUT:-Silence is golden.}\""'

one.env

WHO=Da Schwartz
SHOUTOUT=Get to...

one_glue.yml

one:
    extends:
        file: base.yml
        service: base
    env_file:
        - one.env

two.env

WHO=Da Schwartz
SHOUTOUT=...da choppa!

two_glue.yml

two:
    extends:
        file: base.yml
        service: base
    env_file:
        - two.env

Use

% for i in base one_glue two_glue ; do docker-compose --file "${i}.yml" up ; done
Recreating dockercomposeextendsenv_base_1...
Attaching to dockercomposeextendsenv_base_1
base_1 | Simon says, "Silence is golden."
dockercomposeextendsenv_base_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)
Recreating dockercomposeextendsenv_one_1...
Attaching to dockercomposeextendsenv_one_1
one_1 | Da Schwartz says, "Get to..."
dockercomposeextendsenv_one_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)
Recreating dockercomposeextendsenv_two_1...
Attaching to dockercomposeextendsenv_two_1
two_1 | Da Schwartz says, "...da choppa!"
dockercomposeextendsenv_two_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)

Even simplerer example

The above works if you benefit from using .env files. If you aren't so limited, you could keep the environment variable settings in the environment-specific "glue" .yml files:

red_glue.yml

red:
    extends:
        file: base.yml
        service: base
    environment:
        - WHO=Stallion
        - SHOUTOUT=I am...

blue_glue.yml

blue:
    extends:
        file: base.yml
        service: base
    environment:
        - WHO=Stallion
        - SHOUTOUT=...the law!

Use

% for i in red_glue blue_glue ; do docker-compose --file "${i}.yml" up ; done
Creating dockercomposeextendsenv_red_1...
Attaching to dockercomposeextendsenv_red_1
red_1 | Stallion says, "I am..."
dockercomposeextendsenv_red_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)
Creating dockercomposeextendsenv_blue_1...
Attaching to dockercomposeextendsenv_blue_1
blue_1 | Stallion says, "...the law!"
dockercomposeextendsenv_blue_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)

A little more complicated

For what it's worth, the approach described in this answer allows for different .env files on a per-instance basis, rather than per-invocation/environment. (I'm not sure how beneficial this is in practice, however.) In other words, you could do something like this:

testing.yml

# Only instance1 and instance2 are needed for testing

instance1:
    extends:
        file: base.yml
        service: base
    env_file:
        - test.env # environment-specific
        - instance1_test.env # instance-specific

instance2:
    extends:
        file: base.yml
        service: base
    env_file:
        - test.env
        - instance2_test.env

production.yml

# All four instances are used for production

instance1:
    extends:
        file: base.yml
        service: base
    env_file:
        - prod.env # environment-specific
        - instance1_prod.env # instance-specific

instance2:
    extends:
        file: base.yml
        service: base
    env_file:
        - prod.env
        - instance2_prod.env

instance3:
    extends:
        file: base.yml
        service: base
    env_file:
        - prod.env
        - instance3_prod.env

instance4:
    extends:
        file: base.yml
        service: base
    env_file:
        - prod.env
        - instance4_prod.env

You can start to see that extends is pretty powerful, much more so than what the #1765 merge allows for.

Update Apr 2020

Docker Compose v3+ deprecated the extends feature.

Upvotes: 7

SteveGroom
SteveGroom

Reputation: 129

Nice clear examples, however this did not initially work for me until I updated the base.yml to call the ash shell.

base.yml

base:
    image: busybox
    command: ash -c 'echo "${WHO:-Simon} says, \"${SHOUTOUT:-Silence is golden.}\""'

Upvotes: 0

Andy Shinn
Andy Shinn

Reputation: 28493

See Update #2 below. This is now possible!

This is a much requested feature of Docker Compose. Unfortunately, the answer at the moment is that you can't. I'd recommend subscribing to these GitHub issues to get a better idea when and if this feature gets implemented:

Issue #495 is actually the most commented in their issue repository at the moment. You are definitely not alone in wanting to do this.

Update:

The latest issue tracking is at https://github.com/docker/compose/issues/1377.

Update #2:

This functionality has been merged and is available as of Docker Compose 1.5.0. See https://github.com/docker/compose/blob/129092b7/docs/yml.md#variable-substitution for usage information.

Upvotes: 19

Related Questions