CODEWITHSUNDEEP
djangoherokucelery
Andreas Dolk
Andreas Dolk

Reputation: 114767

celery multi crashes on heroku

I've deployed a django app to heroku which uses celery for long running tasks. For performance reasons (time & memory) I spawn two workers with different concurrency settings and route the tasks. It works very well on my local environment, but the workers crash immediately on heroku.

2014-07-08T15:24:11.300344+00:00 heroku[worker.1]: Starting process with command `python manage.py celery multi start single multi -c:multi 8 -c:single 1 -Q:single single -Q:multi multi --loglevel=INFO`
2014-07-08T15:24:11.944619+00:00 heroku[worker.1]: State changed from starting to up
2014-07-08T15:24:13.386256+00:00 app[worker.1]: celery multi v3.1.8 (Cipater)
2014-07-08T15:24:13.386418+00:00 app[worker.1]: > Starting nodes...
2014-07-08T15:24:13.807448+00:00 app[worker.1]:     > single@f5bd1193-6b06-4822-8bea-6c45640d66c7: OK
2014-07-08T15:24:14.318063+00:00 app[worker.1]:     > multi@f5bd1193-6b06-4822-8bea-6c45640d66c7: OK
2014-07-08T15:24:16.242842+00:00 heroku[worker.1]: Process exited with status 0
2014-07-08T15:24:16.258695+00:00 heroku[worker.1]: State changed from up to crashed
2014-07-08T15:24:16.259282+00:00 heroku[worker.1]: State changed from crashed to starting
2014-07-08T15:24:27.430541+00:00 heroku[worker.1]: State changed from starting to up
2014-07-08T15:24:28.621064+00:00 app[worker.1]: celery multi v3.1.8 (Cipater)
2014-07-08T15:24:28.621229+00:00 app[worker.1]: > Starting nodes...
2014-07-08T15:24:29.103659+00:00 app[worker.1]:     > single@deb6a436-1ab7-4310-9a97-f137ad9bf682: OK
2014-07-08T15:24:29.667462+00:00 app[worker.1]:     > multi@deb6a436-1ab7-4310-9a97-f137ad9bf682: OK
2014-07-08T15:24:30.948397+00:00 heroku[worker.1]: State changed from up to crashed
2014-07-08T15:24:26.827871+00:00 heroku[worker.1]: Starting process with command `python manage.py celery multi start single multi -c:multi 8 -c:single 1 -Q:single single -Q:multi multi --loglevel=INFO`
2014-07-08T15:24:30.938523+00:00 heroku[worker.1]: Process exited with status 0

That's all I can get from the logs... Do I need some special settings for heroku or does celery multi not work on a single dyno (anymore)? I fed google with the most obvious keywords but didn't see an answer.

Any help or hints are highly appreciated!

Upvotes: 3

Views: 507

Answers (1)

joshowen
joshowen

Reputation: 315

Celery Multi starts the nodes as daemons, then exits immediately. The process monitor sees an exit code and thinks the program has crashed, so it restarts the dyno.

See this post for a workaround: https://groups.google.com/forum/#!topic/celery-users/FDIrKPHv33s

Upvotes: 1

Related Questions