Docker compose containers communication

Question

I have 7 docker containers namely

1. es_search
2. postgres_db
3. fusionauth
4. mysql_db
5. auth
6. backend
7. ui

The communication between containers should be like below

• fusionauth should be able to contact es_search, postgres_db
• backend should be able to contact auth, mysql_db
• auth should be able to contact fusionauth, backend
• ui should be able to contact backend, auth

Existing docker-compose

version: '3.1'

services:
 postgres_db:
   container_name: postgres_db
   image: postgres:9.6
   environment:
     PGDATA: /var/lib/postgresql/data/pgdata
     POSTGRES_USER: ${POSTGRES_USER}
     POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
   ports:
     - ${POSTGRES_PORT}:5432
   networks:
     - postgres_db
   restart: unless-stopped
   volumes:
     - db_data:/var/lib/postgresql/data_test

 es_search:
   image: docker.elastic.co/elasticsearch/elasticsearch:6.3.1
   container_name: es_search
   environment:
     - cluster.name=fusionauth
     - bootstrap.memory_lock=true
     - "ES_JAVA_OPTS=${ES_JAVA_OPTS}"
   ports:
     - ${ES1_PORT}:9200
     - ${ES_PORT}:9300
   networks:
     - es_search
   restart: unless-stopped
   ulimits:
     memlock:
       soft: -1
       hard: -1
   volumes:
     - es_data:/usr/share/elasticsearch/data

 fusionauth:
   image: fusionauth/fusionauth-app:latest
   container_name: fusionauth
   depends_on:
     - postgres_db
     - es_search
   environment:
     DATABASE_URL: jdbc:postgresql://postgres_db:5432/fusionauth
     DATABASE_ROOT_USER: ${POSTGRES_USER}
     DATABASE_ROOT_PASSWORD: ${POSTGRES_PASSWORD}
     DATABASE_USER: ${DATABASE_USER}
     DATABASE_PASSWORD: ${DATABASE_PASSWORD}
     FUSIONAUTH_MEMORY: ${FUSIONAUTH_MEMORY}
     FUSIONAUTH_SEARCH_SERVERS: http://es_search:9200
     FUSIONAUTH_URL: http://fusionauth:9010
   networks:
    - postgres_db
    - es_search
   restart: unless-stopped
   ports:
     - ${FUSIONAUTH_PORT}:9011
   volumes:
     - fa_config:/usr/local/fusionauth/config

 db:
   container_name: db
   image: mysql:5.7
   volumes:
     - /etc/nudjur/mysql_data:/var/lib/mysql
   environment:
     - MYSQL_ROOT_PASSWORD=${MYSQL_PASSWORD}
     - MYSQL_USER=${MYSQL_USER}
   ports:
     - ${MYSQL_PORT}:3306
   command: --default-authentication-plugin=mysql_native_password
   restart: on-failure

 backend:
   container_name: backend
   links:
     - db:${MYSQL_HOST}
   depends_on:
     - db
   image: ${BACKEND_IMAGE}
   volumes:
     - ${ENV_FILE}:/backend/.env
   ports:
     - ${BACKEND_PORT}:${BACKEND_PORT}
   command: >
     bash -c "set -a && source .env && set +a"
   restart: unless-stopped

 UI: 
   container_name: UI
   image: ${UI_IMAGE}
   volumes:
     - ${ENV_FILE}:/nudjur/.env
   ports:
     - ${UI_PORT}:${UI_PORT}
   command: >
     bash -c "PORT=${UI_PORT} npm start"
   restart: unless-stopped

 auth:
   container_name: auth
   network_mode: host
   image: ${AUTH_IMAGE}
   volumes:
     - ${ENV_FILE}:/auth/.env
   ports:
     - ${AUTH_PORT}:${AUTH_PORT}
   command: >
     bash -c "set -a && source .env && set +a && python3 ./auth_bridge.py --log-level DEBUG run -p ${AUTH_PORT}"
   restart: unless-stopped

networks:
 postgres_db:
   driver: bridge
 es_search:
   driver: bridge

volumes:
 db_data:
 es_data:
 fa_config:

I am confused about how to establish communication between them.

Can someone help me with this?

Answer 1

Delete every single last networks: and container_name: option from the docker-compose.yml file. network_mode: host as you have on the auth container is incompatible with Docker network communication, and seems to usually get presented as a workaround for port-publishing issues; delete that too. You probably want the name ui: to be in lower case as well.

When you do this, Docker Compose will create a single network named default and attach all of the containers it creates to that network. They will all be reachable by their service names. Networking in Compose in the Docker documentation describes this in more detail.

Your fusionauth container has the right basic setup. Trimming out some options:

fusionauth:
   image: fusionauth/fusionauth-app:latest
   depends_on:
     - postgres_db
     - es_search
   environment:
     # vvv These host names are other Compose service names
     # vvv and default Compose networking makes them reachable
     DATABASE_URL: jdbc:postgresql://postgres_db:5432/fusionauth
     DATABASE_ET: CETERA
     FUSIONAUTH_SEARCH_SERVERS: http://es_search:9200
     FUSIONAUTH_URL: http://fusionauth:9010
   restart: unless-stopped
   ports:
     - ${FUSIONAUTH_PORT}:9011
   volumes:
     - fa_config:/usr/local/fusionauth/config
   # no networks: or container_name:

If the ui container presents something like a React or Angular front-end, remember that application runs in a browser and not in Docker, and so it will have to reach back to the physical system's DNS name or IP address and published ports:. It's common to introduce an nginx reverse proxy into this setup to serve both the UI code and the REST interface it needs to communicate with.

In principle you can set this up with multiple networks for more restrictive communications as the other answers have done. That could prevent ui from contacting postgres_db. I'd be a little surprised to see an environment where that level of control is required, and also where Compose is an appropriate deployment solution.

Also unnecessary, and frequently appearing in other questions like this, are hostname: (only sets containers' own notions of their own host names), links: (only relevant for pre-network Docker), and expose: (similarly).

Answer 2

I understand you want to restrict communications so that containers can only communicate with other services such as:

• fusionauth should be able to contact es_search, postgres_db
• backend should be able to contact auth, mysql_db
• auth should be able to contact fusionauth, backend
• ui should be able to contact backend, auth

You can use networks as you already partially do in your example to enable communication such as:

• services on the same network can reach each other using the service name or alias - i.e. es_search can be reached by other services on the same network via http://es_search:9200
• services on different network are isolated and cannot communicate with each other

You can then define your networks such as:

services:

 postgres_db:
   networks:
     - postgres_db

 es_search:
   networks:
     - es_search

  # fusionauth should be able to contact es_search, postgres_db
  fusionauth:
    networks:
     - fusionauth
     - postgres_db
     - es_search

  db:
    networks:
    - mysql_db

  # backend should be able to contact auth, mysql_db
  backend:
    networks:
    - backend
    - auth
    - mysql_db

  # ui should be able to contact backend, auth
  UI: 
    networks:
    - backend
    - auth

  # auth should be able to contact fusionauth, backend
  auth:
    networks:
    - auth
    - fusionauth
    - backend


networks:
  fusionauth:
  backend:
  auth:
  postgres_db:
  es_search:
  mysql_db:

Where all services (except ui) have their own network, and another service must be on this service's network to communicate with it.

Note: I did not use link as it is a legacy and may be removed in future releases as stated by docs.

Answer 3

from your docker-compose.yml.

postgres_db:
   ..
   networks:
   - postgres_db_net

fusionauth:
   ..
   environment:
     DATABASE_URL: jdbc:postgresql://postgres_db:5432/fusionauth
   networks:
   - postgres_db_net

• The services postgres_db and fusionauth are attached to the network postgres_db-net which enables them to communicate with each other
• communication happens by using the service-name which also works as hostname inside the container. the service fusionauth knows of the database by it's name postgres_db in the DATABASE_URL