WSGI cannot import locally installed packages

Question

I am setting up a Python 3 Flask application on a VPS via WSGI. I am not using a virtualenv. When I go to my domain, I get an Internal Service Error. Checking the error log, I see I am getting an ImportError when attempting to import flask_bootstrap:

[Thu Oct 23 12:14:50.984561 2014] [:error] [pid 4930:tid 139674382092032] [client 54.86.72.53:62650] mod_wsgi (pid=4930): Target WSGI script '/var/www/davidystephenson/davidystephenson.wsgi' cannot be loaded as Python module.
[Thu Oct 23 12:14:50.984628 2014] [:error] [pid 4930:tid 139674382092032] [client 54.86.72.53:62650] mod_wsgi (pid=4930): Exception occurred processing WSGI script '/var/www/davidystephenson/davidystephenson.wsgi'.
[Thu Oct 23 12:14:50.984692 2014] [:error] [pid 4930:tid 139674382092032] [client 54.86.72.53:62650] Traceback (most recent call last):
[Thu Oct 23 12:14:50.984730 2014] [:error] [pid 4930:tid 139674382092032] [client 54.86.72.53:62650]   File "/var/www/davidystephenson/davidystephenson.wsgi", line 8, in <module>
[Thu Oct 23 12:14:50.984843 2014] [:error] [pid 4930:tid 139674382092032] [client 54.86.72.53:62650]     from davidystephenson import app as application
[Thu Oct 23 12:14:50.984864 2014] [:error] [pid 4930:tid 139674382092032] [client 54.86.72.53:62650]   File "/var/www/davidystephenson/davidystephenson/__init__.py", line 4, in <module>
[Thu Oct 23 12:14:50.984926 2014] [:error] [pid 4930:tid 139674382092032] [client 54.86.72.53:62650]     import flask_bootstrap
[Thu Oct 23 12:14:50.984968 2014] [:error] [pid 4930:tid 139674382092032] [client 54.86.72.53:62650] ImportError: No module named flask_bootstrap

However, I have flask_boostrap installed. sudo pip3 install flask_boostrap returns:

Requirement already satisfied (use --upgrade to upgrade): flask-bootstrap in /usr/local/lib/python3.4/dist-packages
Requirement already satisfied (use --upgrade to upgrade): Flask>=0.8 in /usr/local/lib/python3.4/dist-packages (from flask-bootstrap)
Requirement already satisfied (use --upgrade to upgrade): Werkzeug>=0.7 in /usr/local/lib/python3.4/dist-packages (from Flask>=0.8->flask-bootstrap)
Requirement already satisfied (use --upgrade to upgrade): Jinja2>=2.4 in /usr/local/lib/python3.4/dist-packages (from Flask>=0.8->flask-bootstrap)
Requirement already satisfied (use --upgrade to upgrade): itsdangerous>=0.21 in /usr/local/lib/python3.4/dist-packages (from Flask>=0.8->flask-bootstrap)
Requirement already satisfied (use --upgrade to upgrade): markupsafe in /usr/local/lib/python3.4/dist-packages (from Jinja2>=2.4->Flask>=0.8->flask-bootstrap)
Cleaning up...

I get no such error when running the WSGI file directly. I added a line to my application's __init__.py that prints output my available python3 modules:

mods = sorted(["%s==%s" % (i.key, i.version) for i in pip.get_installed_distributions()])

flask_boostrap does appear to be installed. python3 davidystephenson.wsgi outputs:

['chardet==2.0.1', 'colorama==0.2.5', 'command-not-found==0.3', 'flake8==2.2.2', 'flask-bootstrap==3.2.0.2', 'flask-mongoengine==0.7.1', 'flask-wtf==0.10.2', 'flask==0.10.1', 'html5lib==0.999', 'itsdangerous==0.24', 'jinja2==2.7.3', 'markupsafe==0.23', 'mccabe==0.2.1', 'mongoengine==0.8.7', 'pep8==1.5.7', 'pycurl==7.19.3', 'pyflakes==0.8.1', 'pygobject==3.12.0', 'pyinotify==0.9.4', 'pymongo==2.7.2', 'pyopenssl==0.13', 'python-apt==0.9.3.5', 'pyyaml==3.11', 'requests==2.2.1', 'six==1.5.2', 'ufw==0.34-rc-0ubuntu2', 'unattended-upgrades==0.1', 'urllib3==1.7.1', 'werkzeug==0.9.6', 'wtforms==2.0.1']

If I remove the import flask_bootstrap, and have the page simply return a string, it loads correctly. So I made the page output the same list of available python packages, and flask_bootstrap is now missing, though flask is still there:

['argparse==1.2.1', 'chardet==2.0.1', 'colorama==0.2.5', 'configobj==4.7.2', 'flask==0.10.1', 'html5lib==0.999', 'iotop==0.6', 'itsdangerous==0.24', 'jinja2==2.7.3', 'markupsafe==0.23', 'pam==0.4.2', 'pyinotify==0.9.4', 'pyopenssl==0.13', 'pyserial==2.6', 'python-apt==0.9.3.5', 'python-debian==0.1.21-nmu2ubuntu2', 'requests==2.2.1', 'six==1.5.2', 'ssh-import-id==3.21', 'twisted-core==13.2.0', 'urllib3==1.7.1', 'werkzeug==0.9.6', 'wsgiref==0.1.2', 'zope.interface==4.0.5']

I have an configuration file in /etc/apache2/sites-available that manages the site:

<VirtualHost *:80>
    ServerName davidystephenson.com
    ServerAdmin davidystephenson.com
    WSGIDaemonProcess davidystephenson.com python path=davidystephenson.com:/usr/lib/python3.4/site-packages
    WSGIScriptAlias / /var/www/davidystephenson/davidystephenson.wsgi
    <Directory /var/www/davidystephenson/davidystephenson/>
        Order allow,deny
        Allow from all
    </Directory>
    Alias /static /var/www/davidystephenson/davidystephenson/static
    <Directory /var/www/davidystephenson/davidystephenson/static/>
        Order allow,deny
        Allow from all
    </Directory>
    ErrorLog ${APACHE_LOG_DIR}/error.log
    LogLevel warn
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

My WSGI file reads:

usr/bin/python3
import sys
import logging

logging.basicConfig(stream=sys.stderr)
sys.path.insert(0, '/var/www/davidystephenson/')

from davidystephenson import app as application

What should I do?

Answer 1

As @dirn mentioned it's defaulting to the system python, not python3. Since you are not using any virtualenv, all your packages should be somewhere in /usr/lib/python3.4/dist-packages.

You can find that from your interpreter using this.

>>> import site; site.getsitepackages()

To tell wsgi about that, you have to addd this path to your conf just before WSGIScriptAlias.

WSGIPythonPath /var/www/davidystephenson/davidystephenson/:/usr/lib/python3/site-packages

Note: You can also specify the path in daemon mode

WSGIDaemonProcess davidystephenson.com python-path=davidystephenson.com:/usr/lib/python3.4/site-packages

Here is a simple conf file. I don't know why you are not using virtualenv. But it is highly recommended to use it.

Some useful resources: flask docs