CODEWITHSUNDEEP
djangocelery
Palisand
Palisand

Reputation: 1362

Do I need to use celery.result.forget when using a database backend?

I've come across the following warning:

Backends use resources to store and transmit results. To ensure that resources are released, you must eventually call get() or forget() on EVERY AsyncResult instance returned after calling a task.

I am currently using the django-db backend and I am wondering about the consequences of not heeding this warning. What resources will not be "released" if I don't forget an AsyncResult? I'm not worried about cleaning up task results from my database. My primary concern is with the availability of workers being affected.

Upvotes: 5

Views: 2628

Answers (2)

Niel Godfrey P. Ponciano
Niel Godfrey P. Ponciano

Reputation: 10709

Celery seems to have a setting for this which is result_expires. The documentation explains it all:

result_expires

Default: Expire after 1 day.

Time (in seconds, or a timedelta object) for when after stored task tombstones will be deleted.

But as @2ps mentioned, celery-beat must be running for database backends which as documented tells that:

A built-in periodic task will delete the results after this time (celery.backend_cleanup), assuming that celery beat is enabled. The task runs daily at 4am.

For other types of backends e.g. AMQP, it seems not necessary as documented:

Note

For the moment this only works with the AMQP, database, cache, Couchbase, and Redis backends.

When using the database backend, celery beat must be running for the results to be expired.

Upvotes: 0

2ps
2ps

Reputation: 15956

I've actually never seen that warning. As long as you're running celery beat, you'll be fine. Celery has a default periodic task that it sets up for you scheduled to run at 4:00 AM. That task deletes any expired results in your database if you are using a db-based backend like postgres or mysql.

Upvotes: 3

Related Questions