CODEWITHSUNDEEP
djangoredisdjango-celerycelery-task
Sajjad Ali
Sajjad Ali

Reputation: 114

Fix Celery Issue 'can't pickle <class 'module'>: attribute lookup module on builtins failed

I am running celery 4.1.1 on windows and sending requests to redis(on ubuntu), Redis is properly connected and tested from windows side. But when i run this command

celery -A acmetelbi worker --loglevel=info

i get this long error:

[tasks]
. accounts.tasks.myprinting
. acmetelbi.celery.debug_task
 [2019-08-02 11:46:44,515: CRITICAL/MainProcess] Unrecoverable error: 
PicklingErr
or("Can't pickle <class 'module'>: attribute lookup module on builtins 
failed",)

Traceback (most recent call last):
File "c:\acmedata\virtualenv\bi\lib\site- 
packages\celery\worker\worker.py", line 205, in start
self.blueprint.start(self)
File "c:\acmedata\virtualenv\bi\lib\site-packages\celery\bootsteps.py", 
line 119, in start step.start(parent)
File "c:\acmedata\virtualenv\bi\lib\site-packages\celery\bootsteps.py", 
line 370, in start return self.obj.start()
File "c:\acmedata\virtualenv\bi\lib\site- 
packages\celery\concurrency\base.py",
line 131, in start self.on_start()
File "c:\acmedata\virtualenv\bi\lib\site- 
packages\celery\concurrency\prefork.p
y", line 112, in on_start
**self.options)
File "c:\acmedata\virtualenv\bi\lib\site-packages\billiard\pool.py", line 
1007 , in __init__ self._create_worker_process(i)
File "c:\acmedata\virtualenv\bi\lib\site-packages\billiard\pool.py", line 
1116, in _create_worker_process w.start()
File "c:\acmedata\virtualenv\bi\lib\site-packages\billiard\process.py", 
line 124, in start self._popen = self._Popen(self)
File "c:\acmedata\virtualenv\bi\lib\site-packages\billiard\context.py", 
line 383, in _Popen return Popen(process_obj)
File "c:\acmedata\virtualenv\bi\lib\site- 
packages\billiard\popen_spawn_win32.py", 
line 79, in __init__ reduction.dump(process_obj, to_child)
File "c:\acmedata\virtualenv\bi\lib\site-packages\billiard\reduction.py", 
line 99, in dump ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'module'>: attribute lookup 
module on builtins failed

(bi) C:\acmedata\bi_solution\acmetelbi>Traceback (most recent call last):
File "<string>", line 1, in <module>
File "c:\acmedata\virtualenv\bi\lib\site-packages\billiard\spawn.py", 
line 165, in spawn_main exitcode = _main(fd)
File "c:\acmedata\virtualenv\bi\lib\site-packages\billiard\spawn.py", 
line 207, in _main self = pickle.load(from_parent)
EOFError: Ran out of input

i am scratching my head and unable to understand how to fix this. Please help!

my Code for creating a task in django app.

@task()
def myprinting(self):
    print("I am task")

and in settings.py :

Other Celery settings

CELERY_BEAT_SCHEDULE = {
'task-number-one': {
    'task': 'accounts.tasks.myprinting',
    'schedule': crontab(minute='*/30'),
     },

Upvotes: 1

Views: 892

Answers (1)

Sajjad Ali
Sajjad Ali

Reputation: 114

after spending many days in research i have come to conclusion that celery have limitation on windows and if you want to run celery on windows then you must have to run it with gevent command:

python manage.py celery worker -P gevent --loglevel=INFO

and then after running this worker process start the celery beat accordingly to start processing.

Upvotes: 1

Related Questions