CODEWITHSUNDEEP
pythondjangocelery
Adrian Rutkowski
Adrian Rutkowski

Reputation: 309

Celery PeriodicTask per user

I'm working on project which main future will be running periodically one type of async task for each user. Every user will be able to configure task (running daily, weekly etc. at specified time). Also task will use some data stored by user. Now I'm wondering which approach should be better: allow users to create own PeriodicTask (by using some restricted endpoint of course) or create single PeriodicTask (for example running every 5 minutes) which will iterate over all users and determine if task should be queued or not for current user? I think I will use AMPQ as broker.

Upvotes: 2

Views: 485

Answers (1)

Jerzyk
Jerzyk

Reputation: 3742

periodic tasks scheduler in celery is not designed to handle thousands of scheduled tasks, so from performance perspective, much better solution is to have one task that is running at the smallest interval (e.g. if you allow user to sechedule dayly, weekly, monthly - running task daily is enough)

such approach is as well more stable - every time schedule changes, all of the schedule records are reloaded

plus is more secure because you do not expose or use any internal mechanisms for tasks execution

Upvotes: 1

Related Questions