Reputation: 7091
Heroku/Celery: Simultaneous tasks on one worker?
Is it possible to use Celery to set up multiple updating tasks to run simultaneously on Django/Heroku on just ONE worker? If I schedule certain functions to run every 5 minutes, will they automatically overlap in terms of when they start running, or will they wait till all other tasks are finished? I'm new to Celery and frankly vary confused over what it can do? ):
Upvotes: 2
Views: 1384
Answers (1)
Reputation: 10846
By default Celery uses multiprocessing to perform concurrent execution of tasks. Celery worker launches a pool of processes to consume tasks. The number of processes in a pool is set by --concurrency argument and defaults to the number of CPUs available on the machine.
So if the concurrency level is greater than one then the tasks will be processed in parallel.
Upvotes: 4
Related Questions
- Celery worker concurrency
- Celery --beat on Heroku vs Worker and Clock processes
- My worker Celery works in localhost but not in heroku
- Have to manually start worker on heroku?
- Celery creating a new connection for each task
- Django with celery: scheduled task (ETA) executed multiple times in parallel
- How to economically make limited use of worker processes in Heroku
- What is the proper way to shedule Celery tasks to maximize worker productivity?
- Heroku Django: Running a Worker
- Idling a Worker in Heroku, with Django