How to debug basic issues configuring django to be served with apache and mod-wsgi?

Question

It is common that people have trouble setting up django to be serves by apache and mod-wsgi. The common symptom is "Import error" ... for some reason (generally slightly different in each case) settings.py or similar doesn't import (see "Related" in the right column on this page for numerous examples!).

I have read through the other questions on this topic, and none appear to have a solution that works for my situation (one was a basic misconfig by the poster identified by an answerer - I don't appear to have this issue, others apply to using wsgi from other modules etc).

When apache/mod-wsgi fails to load, how can you debug it?

What can you do to make something give you a better message than "Import error"?

Obviously, I'm looking for a method that will identify what's wrong in my case. But I really would like to know how to approach debugging this sort of failure: there seems to be no way that I can find to get information about what it is causing it to fail.

---

In my case, I am trying to do what appears to be a straightforwards django deploy with mod-wsgi - really by the book, the same as the doc, but get the error:

ImportError: Could not import settings 'cm_central.settings' (Is it on sys.path? Is there an import error in the settings file?): No module named cm_central.settings

I can't see why it can't find this module.

/home/cmc/src/cm_central/cm_central/settings.py exists, can be loaded by pythonn without error, and in fact works OK with ./manage.py runserver.

Is it possible that there is some import error occurring in the context of apache, that doesn't occur when I load it myself? I'm wondering because of the words "Is there an import error in the settings file?" ... why does it ask that? If there were an import error, how would I debug it?

I have this in /etc/apache2/sites-enabled/cm-central:

<VirtualHost *:80>

  WSGIScriptAlias / /home/cmc/src/cm_central/cm_central/wsgi.py
  WSGIDaemonProcess cm-central.johalla.de python-path=/home/cmc/src/cm_central:/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages
  WSGIProcessGroup cm-central.johalla.de

  <Directory /home/cmc/src/cm_central/cm_central>
    <Files wsgi.py>
      Order deny,allow
      Allow from all
    </Files>
  </Directory>

</VirtualHost>

And this in wsgi.py: (it I haven't modified it from what django generated)

import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "cm_central.settings")

from django.core.wsgi import get_wsgi_application
from dj_static import Cling

application = Cling(get_wsgi_application())

The full traceback is:

[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40] mod_wsgi (pid=10273): Target WSGI script '/home/cmc/src/cm_central/cm_central/wsgi.py' cannot be loaded as Python module.
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40] mod_wsgi (pid=10273): Exception occurred processing WSGI script '/home/cmc/src/cm_central/cm_central/wsgi.py'.
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40] Traceback (most recent call last):
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/src/cm_central/cm_central/wsgi.py", line 16, in <module>
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     application = Cling(get_wsgi_application())
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/core/wsgi.py", line 14, in get_wsgi_application
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     django.setup()
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/__init__.py", line 20, in setup
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/conf/__init__.py", line 46, in __getattr__
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     self._setup(name)
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/conf/__init__.py", line 42, in _setup
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     self._wrapped = Settings(settings_module)
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]   File "/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages/django/conf/__init__.py", line 98, in __init__
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40]     % (self.SETTINGS_MODULE, e)
[Sun Nov 09 12:00:01 2014] [error] [client 192.168.122.40] ImportError: Could not import settings 'cm_central.settings' (Is it on sys.path? Is there an import error in the settings file?): No module named cm_central.settings
[Sun Nov 09 12:04:06 2014] [notice] Graceful restart requested, doing restart
[Sun Nov 09 12:04:06 2014] [notice] Apache/2.2.22 (Debian) PHP/5.4.4-14+deb7u14 mod_wsgi/3.3 Python/2.7.3 configured -- resuming normal operations
[Sun Nov 09 21:34:15 2014] [error] Not Found: /

If an answer could help me spot what's wrong, that'd be great - but even better is an answer to "how to debug this". How does one find out why the settings.py won't load?

Answer 1

At last, an answer to the high level question: how to debug?

And it's so obvious:

print out everything that matters, from settings.py, and look in the apache log to see what you got, then compare it to what you get in development!

The linked article suggests this set of prints in settings.py:

import sys, os

print "__name__ =", __name__
print "__file__ =", __file__
print "os.getpid() =", os.getpid()
print "os.getcwd() =", os.getcwd()
print "os.curdir =", os.curdir
print "sys.path =", repr(sys.path)
print "sys.modules.keys() =", repr(sys.modules.keys())
print "sys.modules.has_key('mysite') =", sys.modules.has_key('mysite')
if sys.modules.has_key('mysite'):
  print "sys.modules['mysite'].__name__ =", sys.modules['mysite'].__name__
  print "sys.modules['mysite'].__file__ =", sys.modules['mysite'].__file__
  print "os.environ['DJANGO_SETTINGS_MODULE'] =", os.environ.get('DJANGO_SETTINGS_MODULE', None)

For my purposes, I also added DATABASE_URL, and later EMAIL_HOST_USER etc

The article also has a good explanation of how the whole startup process happens, which is worthwhile.

Note that it is a bit out of date with django itself, so the final "alternative wsgi.py" that he comes up with may not be so applicable, but the debugging technique certainly is.

Answer 2

there could be quite different reasons for your issue.

permissions - maybe your wsgi daemon process is running under a different user/group than when you interactively try with python manage.py runserver. maybe the permissions set on your files make a difference then. maybe try to interactively use same user as for the daemon or vice versa. mod_wsgi has nice options so you can run your daemons under separate users, which is good for security also due to separation.

sys.path could be different (although it somehow looks correct in your daemon config), the cwd could be different.

as you seem to use a virtualenv you maybe also want to follow the special setup instructions for Mod-wsgi and Virtualenvs

Answer 3

The error message is pretty explicit already -- mod_wsgi cannot find your settings file (line 2, os.environ.setdefault), check your system path (I understand that Django can find your settings file when you use runserver, but mod_wsgi cannot). To check your system path, you can modify the wsgi.py file and print / log the sys.path at the very top of the file:

import sys
print sys.path

In addition to the virtual env docs pointed to by @Thomas Waldmann, you may also need to manually append the project directory into the sys.path, as mentioned here in the Django config docs. Example code I use is below:

ALLDIRS = ['/usr/local/pythonenv/assessments/lib/python2.6/site-packages']

import os
import sys
import site

# from https://code.google.com/p/modwsgi/wiki/VirtualEnvironments

sys.path.insert(0, '/var/www/assessments/assessments-main/')  # settings.py file here
sys.path.insert(1, '/var/www/assessments/')

prev_sys_path = list(sys.path)
for directory in ALLDIRS:
    site.addsitedir(directory)

new_sys_path = []
for item in list(sys.path):
    if item not in prev_sys_path:
        new_sys_path.append(item)
        sys.path.remove(item)

sys.path[:0] = new_sys_path

os.environ['DJANGO_SETTINGS_MODULE'] = 'assessments-main.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Answer 4

First, to the real question here. How to get more information about what exactly is failing? I might be wrong, but I'm almost sure there is no tools/packages/logs or whatever which would give you more infos on the problem than what you've already got: the traceback. So I guess the only way of debugging this errors, is the "traceback" method ;-)

--- Now to the problem:

Maybe it's a permission problem. In your WSGIDaemonProcess you don't specify any user or group, and maybe apache is not allowed to read/execute the files. Can you try adding user and group to this line like this:

    WSGIDaemonProcess cm-central.johalla.de user=<username> group=<username> python-path=/home/cmc/src/cm_central:/home/cmc/virtualenvs/cmc/lib/python2.7/site-packages

Also, you're not using a DocumentRoot. Maybe that's why it's not finding the right path? In my vhosts, I always include a DocumentRoot, which has the same path as the Directory but with a leading slash, in your case:

<VirtualHost *:80>
    DocumentRoot /home/cmc/src/cm_central/cm_central/

So maybe that solves your problem.

Answer 5

You could try adding this to your wsgi.py:

path = os.path.join(os.path.dirname(__file__), "..") # Adapt the path here to match the root of your django project
if path not in sys.path:
    sys.path.append(path)

to ensure that your Django project is in the python path ?

If an answer could help me spot what's wrong, that'd be great - but even better is an answer to "how to debug this". How does one find out why the settings.py won't load?

It's hard to answer that question if we don't know exactly what you have already tried. Based on the traceback (Is it on sys.path? Is there an import error in the settings file?), I'd say :

• Check that your django project base folder is in the python path (which I haven't read you have done, that's why I made the suggestion above)

• Check that there are no exeptions raised when loading your settings.py file (which you already did)

Answer 6

Can you try adding an empty __init__.py file in the cm_central directory? Depending on how the settings are being imported, this would be needed.