André Costa
André Costa

Reputation: 117

Celery + FastAPI on docker, the app container does not sync with Celery thus I can't use it

So I'm building a webapp using FastAPI. I use Celery to run some background tasks. My code works fine on my local development machine, but when I try to dockerize it, it seems the FastAPI app running on a container, can't sync with celery. For instance I have 4 containers:

enter image description here

If I start the webapp locally on my computer, and don't start the padel-checker-web-1 container and only use the other 3, celery on docker syncs with it and works.

My problem is having the webapp running inside docker and using celery, they just don't sync.

I'm pasting my docker-compose.yml to see if someone can point me in the right direction:

version: '3.9'

services:
  web: &app
    build: ./
    command: pipenv run python -m app.main
    networks:
      - backend-tier
    ports:
      - "8000:8000"
    depends_on:
      - worker
  rabbitmq:
    image: rabbitmq:3.9-management
    environment:
      - RABBITMQ_DEFAULT_USER=andre
      - RABBITMQ_DEFAULT_PASS=secret
    networks:
      - backend-tier
    volumes:
      - ./config/rabbitmq/rabbit.conf:/etc/rabbit/rabbit.conf
    ports:
      # The rabbitMQ management plugin - running on http://localhost:15672
      - "15672:15672"
      - "5672:5672"
  worker:
    <<: *app
    environment:
      CELERY_BROKER_URL: "amqp://andre:secret@rabbitmq/"
    ports: []
    command: ['pipenv', 'run', 'celery', '-A', 'app.feed.celery_worker.celery', 'worker', '-l', 'INFO']
    depends_on:
      - rabbitmq
    networks:
      - backend-tier
  flower:
    <<: *app
    environment:
      CELERY_BROKER_URL: "amqp://andre:secret@rabbitmq/"
    networks:
      - backend-tier
    ports:
      - "5555:5555"
    depends_on:
      - rabbitmq
      - worker
    command: ['pipenv', 'run', 'celery', 'flower']

networks:
  backend-tier:
    driver: bridge

If I try to use the container, calling any task.delay() it just hangs.

enter image description here

Upvotes: 2

Views: 1080

Answers (1)

Andr&#233; Costa
Andr&#233; Costa

Reputation: 117

I've fixed my docker-compose.yml, I guess it was the depends_on:

version: '3.9'

services:
  web: &app
    build: ./
    command: pipenv run python -m app.main # && pipenv run celery -A app.feed.celery_worker.celery worker -l INFO
    environment:
      - CELERY_BROKER_URL=amqp://andre:secret@rabbitmq/
    ports:
      - "8000:8000"
    depends_on:
      - rabbitmq
  rabbitmq:
    image: rabbitmq:3.9-management
    environment:
      - RABBITMQ_DEFAULT_USER=andre
      - RABBITMQ_DEFAULT_PASS=secret
    volumes:
      - ./config/rabbitmq/rabbit.conf:/etc/rabbit/rabbit.conf
    ports:
      # The rabbitMQ management plugin - running on http://localhost:15672
      - "15672:15672"
      - "5672:5672"
  worker:
    <<: *app
    environment:
      - CELERY_BROKER_URL=amqp://andre:secret@rabbitmq/
      - C_FORCE_ROOT=1
    ports: []
    command: ['pipenv', 'run', 'celery', '-A', 'app.feed.celery_worker.celery', 'worker', '-l', 'INFO']
    depends_on:
      - rabbitmq
      - web
  flower:
    <<: *app
    environment:
      CELERY_BROKER_URL: "amqp://andre:secret@rabbitmq/"
    ports:
      - "5555:5555"
    depends_on:
      - rabbitmq
      - worker
    command: ['pipenv', 'run', 'celery', 'flower']

Upvotes: 1

Related Questions