Not able to run celeryd as normal user but runs with root

Question

I have installed celeryd and django-celery

i run via python manage.py celeryd

If i run it via root then i get this message

WARNING/MainProcess] /opt/python27/lib/python2.7/site-packages/celery-2.2.7-py2.7.egg/celery/apps/worker.py:124: UserWarning: Running celeryd with superuser privileges is not encouraged! "Running celeryd with superuser privileges is not encouraged!")

But if i run via normal user i get this

[2011-07-12 11:13:56,508: WARNING/MainProcess] Traceback (most recent call last):
[2011-07-12 11:13:56,508: WARNING/MainProcess] File "manage.py", line 14, in <module>
[2011-07-12 11:13:56,508: WARNING/MainProcess] execute_manager(settings)
[2011-07-12 11:13:56,508: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
[2011-07-12 11:13:56,508: WARNING/MainProcess] utility.execute()
[2011-07-12 11:13:56,508: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
[2011-07-12 11:13:56,509: WARNING/MainProcess] self.fetch_command(subcommand).run_from_argv(self.argv)
[2011-07-12 11:13:56,509: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
[2011-07-12 11:13:56,509: WARNING/MainProcess] self.execute(*args, **options.__dict__)
[2011-07-12 11:13:56,509: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
[2011-07-12 11:13:56,509: WARNING/MainProcess] output = self.handle(*args, **options)
[2011-07-12 11:13:56,509: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/django_celery-2.2.4-py2.7.egg/djcelery/management/commands/celeryd.py", line 21, in handle
[2011-07-12 11:13:56,510: WARNING/MainProcess] worker.run(*args, **options)
[2011-07-12 11:13:56,510: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/celery-2.2.7-py2.7.egg/celery/bin/celeryd.py", line 96, in run
[2011-07-12 11:13:56,510: WARNING/MainProcess] return self.app.Worker(**kwargs).run()
[2011-07-12 11:13:56,510: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/celery-2.2.7-py2.7.egg/celery/apps/worker.py", line 135, in run
[2011-07-12 11:13:56,510: WARNING/MainProcess] self.run_worker()
[2011-07-12 11:13:56,510: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/celery-2.2.7-py2.7.egg/celery/apps/worker.py", line 235, in run_worker
[2011-07-12 11:13:56,510: WARNING/MainProcess] worker.start()
[2011-07-12 11:13:56,511: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/celery-2.2.7-py2.7.egg/celery/worker/__init__.py", line 250, in start
[2011-07-12 11:13:56,511: WARNING/MainProcess] blocking(component.start)
[2011-07-12 11:13:56,511: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/kombu-1.1.6-py2.7.egg/kombu/syn.py", line 14, in blocking
[2011-07-12 11:13:56,511: WARNING/MainProcess] return __sync_current(fun, *args, **kwargs)
[2011-07-12 11:13:56,511: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/kombu-1.1.6-py2.7.egg/kombu/syn.py", line 30, in __blocking__
[2011-07-12 11:13:56,511: WARNING/MainProcess] return fun(*args, **kwargs)
[2011-07-12 11:13:56,511: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/celery-2.2.7-py2.7.egg/celery/concurrency/base.py", line 64, in start
[2011-07-12 11:13:56,511: WARNING/MainProcess] self.on_start()
[2011-07-12 11:13:56,512: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/celery-2.2.7-py2.7.egg/celery/concurrency/processes/__init__.py", line 37, in on_start
[2011-07-12 11:13:56,512: WARNING/MainProcess] self._pool = self.Pool(processes=self.limit, **self.options)
[2011-07-12 11:13:56,512: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/celery-2.2.7-py2.7.egg/celery/concurrency/processes/pool.py", line 482, in __init__
[2011-07-12 11:13:56,512: WARNING/MainProcess] self._setup_queues()
[2011-07-12 11:13:56,512: WARNING/MainProcess] File "/opt/python27/lib/python2.7/site-packages/celery-2.2.7-py2.7.egg/celery/concurrency/processes/pool.py", line 654, in _setup_queues
[2011-07-12 11:13:56,526: WARNING/MainProcess] self._inqueue = SimpleQueue()
[2011-07-12 11:13:56,526: WARNING/MainProcess] File "/opt/python27/lib/python2.7/multiprocessing/queues.py", line 328, in __init__
[2011-07-12 11:13:56,527: WARNING/MainProcess] self._rlock = Lock()
[2011-07-12 11:13:56,527: WARNING/MainProcess] File "/opt/python27/lib/python2.7/multiprocessing/synchronize.py", line 121, in __init__
[2011-07-12 11:13:56,527: WARNING/MainProcess] SemLock.__init__(self, SEMAPHORE, 1, 1)
[2011-07-12 11:13:56,527: WARNING/MainProcess] File "/opt/python27/lib/python2.7/multiprocessing/synchronize.py", line 49, in __init__
[2011-07-12 11:13:56,528: WARNING/MainProcess] sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
[2011-07-12 11:13:56,528: WARNING/MainProcess] OSError
[2011-07-12 11:13:56,528: WARNING/MainProcess] :
[2011-07-12 11:13:56,529: WARNING/MainProcess] [Errno 13] Permission denied
[2011-07-12 11:13:56,531: INFO/MainProcess] process shutting down

I have added the group read write execute permission in all files under celery egg and chnaged the groupof all celery files to user but still it get ssame error

Answer 1

You are not giving r/w access for the user that runs celery to the shared memory (/dev/shm), that it's accessed by multiprocessing.

You should add this to your /etc/fstab:

none /dev/shm tmpfs rw,nosuid,nodev,noexec 0 0

And restart your linux.

---

EDIT:

You can try adding permissions manually to /dev/shm:

sudo chmod u+rw /dev/shm