CODEWITHSUNDEEP
queueceleryairflow
tkarahan
tkarahan

Reputation: 335

Unable to Create Queues in Airflow Workers

I'm using this compose file to run Airflow with celery executor: celery executor. There is a flower service too in the yaml file to monitor workers. I want to add queues to workers. When I run the docker-compose file as is it worked, and I could see the workers on flower's UI. When I changed command tag of worker service from "worker" to "worker -q worker_subdag, default" no worker seemed to run in flower's UI.

command: worker -q worker_subdag, default

I tried below commands too, but no luck:

airflow worker -q worker_subdag, default
airflow celery worker -q worker_subdag, default
celery worker -q worker_subdag, default

Upvotes: 0

Views: 754

Answers (1)

Gayle
Gayle

Reputation: 63

Instead of setting the queue using the the -Q flag in the command, set it in the environment section of your docker-compose:

 environment:
      <<: *airflow-common-env
      DUMB_INIT_SETSID: "0"
      AIRFLOW__OPERATORS__DEFAULT_QUEUE: worker_subdag

Upvotes: 1

Related Questions