CODEWITHSUNDEEP
djangorabbitmqcelery
Navid777
Navid777

Reputation: 3681

Celery receives tasks from rabbitmq, but not executing them

I have a Django project and have setup Celery + RabbitMQ to do heavy tasks asynchronously. When I call the task, RabbitMQ admin shows the task, Celery prints that the task is received, but the task is not executed.

Here is the task's code:

@app.task
def dummy_task():
    print("I'm Here")
    User.objects.create(username="User1")
    return "User1 Created!"

In this view I send the task to celery:

def task_view(request):
      result = dummy_task.delay()
      return render(request, 'display_progress.html', context={'task_id': result.task_id})

I run celery with this command:

$ celery -A proj worker -l info --concurrency=2 --without-gossip

This is output of running Celery:

-------------- celery@DESKTOP-8CHJOEG v5.2.7 (dawn-chorus)
--- ***** ----- -- ******* ---- Windows-10-10.0.19044-SP0 2022-08-22 10:10:04

  • *** --- * ---
  • ** ---------- [config]
  • ** ---------- .> app: proj:0x23322847880
  • ** ---------- .> transport: amqp://navid:**@localhost:5672//
  • ** ---------- .> results:
  • *** --- * --- .> concurrency: 2 (prefork) -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker) --- ***** ----- -------------- [queues] .> celery exchange=celery(direct) key=celery

[tasks] .proj.celery.debug_task . entitymatching.tasks.create_and_learn_machine . entitymatching.tasks.dummy_task

[2022-08-22 10:10:04,068: INFO/MainProcess] Connected to amqp://navid:**@127.0.0.1:5672// [2022-08-22 10:10:04,096: INFO/MainProcess] mingle: searching for neighbors [2022-08-22 10:10:04,334: INFO/SpawnPoolWorker-1] child process 6864 calling self.run() [2022-08-22 10:10:04,335: INFO/SpawnPoolWorker-2] child process 12420 calling self.run() [2022-08-22 10:10:05,134: INFO/MainProcess] mingle: all alone [2022-08-22 10:10:05,142: WARNING/MainProcess] C:\Users\Navid\PycharmProjects\proj\venv\lib\site-packages\celery\fixups\django.py:203: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments! warnings.warn('''Using settings.DEBUG leads to a memory

[2022-08-22 10:10:05,142: INFO/MainProcess] celery@DESKTOP-8CHJOEG ready. [2022-08-22 10:10:05,143: INFO/MainProcess] Task entitymatching.tasks.dummy_task[97f8a2eb-0006-4d53-ba6a-7b9f8649c84a] received [2022-08-22 10:10:05,144: INFO/MainProcess] Task entitymatching.tasks.dummy_task[17190479-0784-46b1-8dc6-870ead41e9c6] received [2022-08-22 10:11:36,384: INFO/MainProcess] Task proj.celery.debug_task[af3d633f-7b9a-4441-b375-9ce217a40ab3] received

But "I'm Here" is not printed, and User1 is not created.

RabbitMQ shows that there are 3 "unack" messages in the queue: enter image description here

Upvotes: 3

Views: 2287

Answers (1)

scaryhamid
scaryhamid

Reputation: 400

You did not provide enough info although I think you have problem with your worker pools.

try adding

--pool=solo

at the end of your run command. it will be like

celery -A proj worker -l info --concurrency=2 --without-gossip --pool=solo

although on production I recommend using gevent as your pool.

celery -A proj worker -l info --concurrency=2 --without-gossip --pool=gevent

Upvotes: 4

Related Questions