CODEWITHSUNDEEP
pythoncelery
Stephen
Stephen

Reputation: 156

Python Celery Asynchronous Tasks

I'm running Python 2.7.10 and Celery 3.1.19 using RabbitMQ 3.5.6.

I am afraid that I just don't understand how Celery works. Here is my tasks.py file:

from celery import Celery
app = Celery('tasks', backend='rpc://', broker='amqp://guest@localhost//')

@app.task
def add(a, b):
    return a + b

And here is a transcript of my playing with this:

In [20]: res0 = add.delay(1,2)

In [21]: res0.ready()
Out[21]: True

In [22]: res0.get()
Out[22]: 3

In [23]: res1 = add.delay(2,3)

In [24]: res1.ready()
Out[24]: True

In [25]: res1.get()
Out[25]: 5
--- try in a different order ----
In [26]: res0 = add.delay(1,2)

In [27]: res1 = add.delay(2,3)

In [28]: res0.ready()
Out[28]: True

In [29]: res1.ready()
Out[29]: False

In [30]: res0.get()
Out[30]: 3

I thought that Celery could run the tasks asynchronously? Why do both adds work on the top, but not on the bottom? I must be misusing or misunderstanding Celery. Can someone help me out?

Thanks for the help!

Update:

Another transcript of playing with this. Is Celery quantum mechanical where the state only collapses when you inspect what has happened?! I'm so confused!

In [12]: res0 = add.delay(1,2)

In [13]: res1 = add.delay(2,3)

In [14]: res0.ready()
Out[14]: True

In [15]: res1.ready()
Out[15]: False

In [16]: res0.result
Out[16]: 3
--- another try ----
In [17]: res0 = add.delay(1,2)

In [18]: res1 = add.delay(2,3)

In [19]: res1.ready()
Out[19]: True

In [20]: res0.ready()
Out[20]: False

In [21]: res1.result
Out[21]: 5

Upvotes: 0

Views: 328

Answers (1)

Stephen
Stephen

Reputation: 156

It looks like I've stumbled into a corner of Celery that most people don't think to enter, but here it is. It turns out that if you don't use the rpc:// backend, but rather use an explicit reference to RabbitMQ, it works:

# this works
app = Celery('tasks', backend='amqp://guest@localhost//', broker='amqp://guest@localhost//')

The rpc:// method is given in the official docs here, but it is a broken example. So - don't use rpc:// if you want true asynchronous behavior!

Upvotes: 2

Related Questions