Why isn't this docker-compose yml file executing the command detailed in this dockerfile?

Question

I have two docker images: a producer and a RabbitMQ queue. My goal is to get the producer (Python) to spit entries into the queue. I then want to call docker-compose up and watch the producer add things to the queue.

Now, I'm not the brightest crayon in the box, and I'd really appreciate it if someone could point out for me why things aren't working as I intended. My producer is a simple 2 files in a ./producer directory. The dockerfile for this directory appears as so:

FROM ubuntu:latest

ENV DEBIAN_FRONTEND=noninteractive

# Prepare to install 3.7 + GDAL libraries
RUN apt-get update --fix-missing
RUN apt-get install -y software-properties-common apt-utils
RUN add-apt-repository ppa:ubuntugis/ppa
RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt-get update --fix-missing

# Install the packages for GDAL
RUN apt-get install libgdal-dev gdal-bin -y
# Install the packages for python
RUN apt-get install python3.7 python3-pip python3.7-dev -y

RUN python3.7 -m pip install pika

COPY ./example.py example.py

CMD [ "/bin/bash", "-c \"python3.7 example.py\"" ]

I've been swapping out the last line with ENTRYPOINT, CMD, various versions of /bin/bash, python3, python3.7. Running docker build -t blah && docker run --rm blah produces the behavior I expect (Can't connect to the queue because it's not running exception).

In the directory that holds ./producer, I have a docker-compose.yml file that appears as so:

version: "3.3"
services:

  # RabbitMQ that connects the backend to the frontend
  queue:
    image: rabbitmq:3-management
    expose:
      # The standard AMQP protocol port
      - 5672
    ports:
      # HTTP management UI
      - '15672:15672'
    environment:
      RABBITMQ_DEFAULT_USER: "guest"
      RABBITMQ_DEFAULT_PASS: "guest"
    networks:
      rakan:
        ipv4_address: 172.16.238.10
    deploy:
      replicas: 1
      update_config:
        parallelism: 1
        delay: 10s
      restart_policy:
        condition: on-failure
        delay: 10s
        max_attempts: 3
        window: 120s
    healthcheck:
      test: [ "CMD", "nc", "-z", "localhost", "5672" ]
      interval: 5s
      timeout: 15s
      retries: 1


  producer:
    depends_on: 
      - queue
    build: 
      context: ./producer
      dockerfile: dockerfile
    # entrypoint: "/bin/bash -c \"python example.py\""
    expose:
      - "5672"
    networks:
      rakan:
        ipv4_address: 172.16.238.11
    links:
      - "queue"
    networks:
      - rakan

networks:
  rakan:
    ipam:
      driver: default
      config:
        - subnet: "172.16.238.0/24"

What is terribly frustrating, is that docker-compose up no longer executes the command in ./producer/dockerfile. It will start up the queue, but it will choose to ignore my desired commands, saying:

Attaching to toyexample_queue_1, toyexample_producer_1
toyexample_producer_1 exited with code 0
... All the logs of the RabbitMQ ...

Why oh why is this happening? Why isn't it behaving the same way when I executed docker run? Any help would be greatly appreciated.

Things I've tried:

• Adding entrypoint to docker-compose.yml of the form python3.7 example.py, bin/bash python3.7 example.py, bash python example.py. All will terminate with error code 127, claiming that python or python3.7 is an unrecognized command.
• Adding cmd to docker-compose.yml with the same exact results as the last bullet point
• Swapping out CMD with ENTRYPOINT in the ./producer/dockerfile
• Trying different variations of python, python3.7 in both docker-compose.yml and dockerfile.

Things that I don't know are relevant but might help

• I'm running docker v19.03.8 on windows 10 (AMD Ryzen CPU)
• 0 other containers are running
• docker-compose down, docker network/container/image prune are all commands I've tried numerous during this epic journey
• The code in example.py is as so:

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('172.16.238.10'))
channel = connection.channel()

for i in range(100):
    # make a lot of noise
    channel.queue_declare(queue=f'hello_{i}')
    channel.basic_publish(exchange='',
                        routing_key='hello_{i}',
                        body='Hello World!')
    print(f" [x] Sent 'Hello World!' to {i}")

connection.close()

• Obligatory: "This isn't homework, I'm not trying to cheat, etc"

Answer 1

First, a couple of quick notes:

• EXPOSE in your Dockerfile and expose in your docker-compose.yml are in most cases unnecessary, except as a form of documentation.
• Just name your Dockerfile "Dockerfile", rather than overriding the name in your docker-compose.yml. Reducing the amount of extraneous configuration makes things easier to read (for other people and for future you).
• Don't hardcode IP addresses into your code. Use container names instead.
• Similarly, don't use the links keyword in your compose file. The DNS provided by docker is much more flexible.
• depends_on is almost completely useless. It doesn't know anything about applications, so while it will ensure that one container start after another container, it doesn't know whether or not the service you're depending on is actually up and running or not. In most cases, you want to get rid of depends_on and just implement reconnection logic in your application.

That last bullet may actually be the source of your problem: at least in my tests, it takes much longer for RabbitMQ to start accepting requests than it takes for the producer container to start up.

With all that in mind, I rewrote your docker-compose.yml to look like this:

version: "3.3"

services:

  # RabbitMQ that connects the backend to the frontend
  queue:
    image: rabbitmq:3-management
    environment:
      RABBITMQ_DEFAULT_USER: "guest"
      RABBITMQ_DEFAULT_PASS: "guest"

  producer:
    build: 
      context: ./producer

And your example.py to look like this:

import pika
import time

while True:
    try:
        connection = pika.BlockingConnection(pika.ConnectionParameters('queue'))
    except pika.exceptions.AMQPConnectionError as err:
        print('rabbitmq connection failed; retrying in 1 second...')
        time.sleep(1)
    else:
        break

channel = connection.channel()

for i in range(100):
    # make a lot of noise
    channel.queue_declare(queue=f'hello_{i}')
    channel.basic_publish(exchange='',
                        routing_key='hello_{i}',
                        body='Hello World!')
    print(f" [x] Sent 'Hello World!' to {i}")

connection.close()

Running the above configuration produces (every time):

[lars@madhatter npengra317] (master *)$ docker-compose up
Starting npengra317_queue_1    ... done
Starting npengra317_producer_1 ... done
Attaching to npengra317_queue_1, npengra317_producer_1
queue_1     | 2020-05-09 03:38:25.676 [info] <0.9.0> Feature flags: list of feature flags found:
.
.
.
queue_1     | 2020-05-09 03:38:26.599 [info] <0.9.0> Server startup complete; 3 plugins started.
queue_1     |  * rabbitmq_management
queue_1     |  * rabbitmq_web_dispatch
queue_1     |  * rabbitmq_management_agent
queue_1     |  completed with 3 plugins.
queue_1     | 2020-05-09 03:38:26.868 [info] <0.653.0> accepting AMQP connection <0.653.0> (172.25.0.3:34874 -> 172.25.0.2:5672)
producer_1  | rabbitmq connection failed; retrying in 1 second...
producer_1  | rabbitmq connection failed; retrying in 1 second...
producer_1  | rabbitmq connection failed; retrying in 1 second...
producer_1  | rabbitmq connection failed; retrying in 1 second...
producer_1  | rabbitmq connection failed; retrying in 1 second...
producer_1  | rabbitmq connection failed; retrying in 1 second...
producer_1  | rabbitmq connection failed; retrying in 1 second...
producer_1  | rabbitmq connection failed; retrying in 1 second...
producer_1  |  [x] Sent 'Hello World!' to 0
producer_1  |  [x] Sent 'Hello World!' to 1
.
.
.
producer_1  |  [x] Sent 'Hello World!' to 97
producer_1  |  [x] Sent 'Hello World!' to 98
producer_1  |  [x] Sent 'Hello World!' to 99
npengra317_producer_1 exited with code 0