Starting NGINX Load Balancer with Docker Compose

Question

I have been following a tutorial on how to make a load balanced application using docker-compose and nginx. However, my load balancer/coordinator doesn't work - what I am trying to do is have nginx accept requests and split them between three workers, and I want nginx and the three workers to be running in separate docker containers, but I get the following error. My compilerwebservice_worker does work correctly, and I can see all three in docker ps, and I can ping them with wget on the localhost post they are listening to.

The error message

$ docker-compose up
Starting compilerwebservice_worker1_1
Starting compilerwebservice_worker3_1
Starting compilerwebservice_worker2_1
Starting compilerwebservice_nginx_1
Attaching to compilerwebservice_worker1_1, compilerwebservice_worker3_1, compilerwebservice_worker2_1, compilerwebservice_nginx_1
nginx_1   | 2016/09/06 07:17:47 [emerg] 1#1: host not found in upstream "compiler-web-service" in /etc/nginx/nginx.conf:14
nginx_1   | nginx: [emerg] host not found in upstream "compiler-web-service" in /etc/nginx/nginx.conf:14
compilerwebservice_nginx_1 exited with code 1

NGINX Config

http {
    upstream compiler {
        least_conn;
        server worker1:4567;
        server worker2:4567;
        server worker3:4567;
    }

    server {
        listen 4567;
        location / {
                proxy_pass http://compiler;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection 'upgrade';
                proxy_set_header Host $host;
                proxy_cache_bypass $http_upgrade;
       }
    }
}

My Docker-compose file

nginx:
  build: ./src/main/nginx
  links:
      - worker2:worker2
      - worker3:worker3
      - worker1:worker1
  ports:
      - "4567:4567"

worker1:
  build: .
  ports:
      - "4567"

worker2:
  build: .
  ports:
      - "4567"

worker3:
  build: .
  ports:
      - "4567"

NGINX Docker file

# Set nginx base image
FROM nginx

# Copy custom configuration file from the current directory
COPY nginx.conf /etc/nginx/nginx.conf

Answer 1

In the below demo there are 2 express app running with port 1111 and 2222 in localhost, on calling http://localhost:8080 it should automatically choose any one of the port 1111 or 2222. here nginx uses round robin

index.js file

const express = require('express');
const app = express();
const appId = process.env.APPID;
const PORTNUMBER = appId;
app.get('/', (req, res) => {
  res.send({
    message: `Welcome to ${appId} home page running on port ${appId}`
  });
});

app.listen(PORTNUMBER, () => {
  console.log(`APP STARTED ON PORT ${appId} for APP id ${appId}`);
})
 

express app docker file

FROM node:12.13.0-alpine

WORKDIR /EXPRESSAPP

COPY ./API/package.json /EXPRESSAPP

RUN npm install

COPY ./API/. /EXPRESSAPP

CMD ["npm", "start"]
 

nginx file

 http {
  upstream backend {
    server 127.0.0.1:1111;
    server 127.0.0.1:2222;
  }

  server {
    listen 8080 default_server;
    listen [::]:8080 default_server;
    # listen [::]:8080 default_server ipv6only=on;
    server_name localhost;
    proxy_read_timeout 5m; 
    location / {
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection 'upgrade';
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_cache_bypass $http_upgrade;
      proxy_pass http://backend;
    }
  }
}

nginx dockercompose file

FROM nginx
COPY nginx.conf /etc/nginx/nginx.conf

docker-compose.yml file

    version: '3'

services:
  myapp1:
    restart: always
    container_name: myapp1
    build: ./APICONTAINER
    environment: 
      - APPID=1111
    ports:
      - "1111:1111"
    network_mode: host

  myapp2:
    restart: always
    container_name: myapp2
    build: ./APICONTAINER
    environment:
      - APPID=2222
    ports:
      - "2222:2222"
    network_mode: host

  myproxy:
    container_name: myproxy
    build: ./NGINXCONTAINER
    ports: 
      - "127.0.0.1:8080:8080"
    depends_on: 
      - myapp1
      - myapp2
    network_mode: host
 

to spin up the containers use the below command

sudo docker-compose down && sudo docker-compose up --build --force-recreate

go to below link to see the round robin nginx load balancer

http://localhost:8080

reference github link to get the full code

Answer 2

I needed to rebuild with docker-compose build between configuration changes. As I changed the name of the app, the error message indicated a server, whose name was the original one that I selected instead of the one I kept changing.