mod_python unable to locate the web.py module under a virtual environment

Question

I have a small web.py Python application that I would like to serve under Apache using mod_python. The web.py framework, as well as other third-party Python modules the application in question relies upon, are installed in a virtual environment. The virtual environment is created inside: /home/ayaz/Sandbox/Scrapper/

The relevant snippet from the virtual host configuration for Apache that I am using in order to set up this application is this:

<Location "/api">
    PythonPath "['/home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/', '/home/ayaz/project/'] + sys.path"
    #PythonHandler wsgiref.modpython_gateway::handler
    PythonHandler modpython_gateway::handler
    SetHandler python-program 
    PythonOption wsgi.application device_api::main
    PythonOption device_api /api/
</Location>

On the browser when I try to access the /api URL however, I get a 500 from the server with the error in the logs saying that the web module imported from within the device_api.py file (which is my application) could not be found; in other words, I see an ImportError. I am not sure why it is unable to find the web module.

I know that the PythonPath directive is working, at least partially, judging from the fact that the Python interpreter is able to find the device_api.py file from the path defined in that directive. But, it is unable to find the rest of the modules for which the path is also defined in the same directive.

Any help with this will be deeply appreciated.

Thanks!

UPDATE #1

Ned's reply had me looking through the site-packages directory for the virtual environment I have. While the permissions looked fine inside the directory to me, I realized that mod_python/Apache was not able to read the eggs. Since I installed all the packages inside the virtual environment using easy_install, they are all in the forms of eggs. So, for example, if I moved the directory /home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/web.py-0.34-py2.5.egg/web/ into /home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/ (essentially taking it out of the egg file/directory), mod_python stopped complaining about the missing web module (of course, it then started complaining about the missing rest of the third-party modules).

I have Apache configured to run as my user and group, that is ayaz, and I checked that the permissions on the /home/ayaz/.python-eggs directory were fine.

I then used this Using eggs with mod_python tricked explained on the Django deployment documentation page. But it didn't help. Eventually, I bit the bullet, and for each third-party module that mod_python complained it couldn't find, I moved the actual directory from inside the egg for that module outside of the egg and into the site-packages directory for the virtual environment. This made mod_python find the modules.

Now, I am really not sure why this is happening, and why mod_python isn't able to read through the eggs.

Answer 1

So as to answer this question, I am going to copy the update paragraphs from my question and post them as the answer (as they seem to explain to some extent how I manage to get rid of the problem).

Ned's reply had me looking through the site-packages directory for the virtual environment I have. While the permissions looked fine inside the directory to me, I realized that mod_python/Apache was not able to read the eggs. Since I installed all the packages inside the virtual environment using easy_install, they are all in the forms of eggs. So, for example, if I moved the directory /home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/web.py-0.34-py2.5.egg/web/ into /home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/ (essentially taking it out of the egg file/directory), mod_python stopped complaining about the missing web module (of course, it then started complaining about the missing rest of the third-party modules).

I have Apache configured to run as my user and group, that is ayaz, and I checked that the permissions on the /home/ayaz/.python-eggs directory were fine.

I then used this Using eggs with mod_python tricked explained on the Django deployment documentation page. But it didn't help. Eventually, I bit the bullet, and for each third-party module that mod_python complained it couldn't find, I moved the actual directory from inside the egg for that module outside of the egg and into the site-packages directory for the virtual environment. This made mod_python find the modules.

Now, I am really not sure why this is happening, and why mod_python isn't able to read through the eggs.

Answer 2

Could it be a file or directory permission problem? Verify that the all of the files and directories in your virtualenv site-packages are accessible from the user name that Apache and mod_python are running under.