ipython install new modules

Question

I am used to the R functionality of installing packages and I am trying to do the same thing with ipython. Sometimes the following method works but then again sometimes it doesn't and I would like to finally find out why it only works half the time.

Normally to install a module (like the requests module for example) I would type the following after opening a fresh terminal:

$ sudo pip install requests
Password: ******* 

This would then be followed by a message indicating that the install was successful or that it has already been installed.

Requirement already satisfied (use --upgrade to upgrade): 
requests in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
Cleaning up... 

Which suggests that the code can be accessed. And indeed if I run python now from the terminal it shows a good response without any errors whatsoever.

$ python
ActivePython 2.7.2.5 (ActiveState Software Inc.) based on
Python 2.7.2 (default, Jun 24 2011, 12:20:15) 
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> 

I now open PyLab through Alfred and it gives me an error.

Welcome to pylab, a matplotlib-based Python environment [backend: WXAgg].
For more information, type 'help(pylab)'.

In [1]: import requests
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/Users/vincentwarmerdam/<ipython-input-1-686486c241c8> in <module>()
----> 1 import requests

ImportError: No module named requests

I've read some help from another question on StackOverflow (here) which suggests that I install the module from ipython shell. This gives an even more baffling response:

In [2]: !pip install requests
Requirement already satisfied (use --upgrade to upgrade): requests in     
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
Cleaning up...

In [3]: import requests
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/Users/vincentwarmerdam/<ipython-input-3-686486c241c8> in <module>()
----> 1 import requests

ImportError: No module named requests

This seems very strange to me. Are there multiple versions of python installed on the system? How could I check this? Do I need to point ipython to the location of the installed code?

Answer 1

This sorted me out running Spyder on Windows. As the answer above, to find the directory:

import sys

sys.path

This site : www.geeksforgeeks.org/how-to-install-pip-on-windows/

Directed me to this: https://bootstrap.pypa.io/get-pip.py

Use cmd to run that script from the directory of the version of Python you want to add modules to. Then just use pip to add the modules, again from the command line. I didn't even bother adding any of it to my PATH, but you could do that.

Answer 2

actually there is a much much much more elegant solution. when pip is installed then within python you can do things like this as well:

import pip

def install(package):
   pip.main(['install', package])

install('requests') 

which is easier. once logged into a virtualenv you can just make sure that you have what you need in the session you are in. easy.

edit

Another alternative would be to use the %%bash magic.

%%bash
pip install requests

edit2

If you want the standard output, one could even use the exclamation bang.

! pip install requests

edit3

From within ipython this is the safest installation method.

%pip install requests

This ensures that everything is installed in the virtualenv that your ipython is installed in.

Answer 3

I had this same problem when trying to install patool, but it turned out it was due to more than one interpreter installed, and when I ran it from ipython it worked.

Answer 4

If the new packages installed are imported on Terminal but not imported to ipython notebook then it is very likely that you have two versions of python installed on your library. Due to this there are separate site-packages directory from where packages are being imported on terminal and ipython notebook. To check if this is the case. On terminal use:

import sys
sys.path

This will show where the python modules are located for python you using on terminal. Now, in ipython notebook use:

import sys
sys.path

This will show where the python modules are located for python you using on terminal.

Now, if the two path are different you know you are using two different installations of python. To solve this problem, copy installed packages from site-packages directory of terminal python to site-packages directory of ipython.

Answer 5

Here's what I did that made it work; open up iypthon through the command line and type

import sys
sys.path 

This shows a list of folders where other python modules are located. For me this was:

['',
 '/Library/Frameworks/Python.framework/Versions/7.3/bin',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/pandas-0.10.0-py2.7-macosx-10.5-i386.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/googlemaps-1.0.2-py2.7.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/oauth-1.0.1-py2.7.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/oauth2-1.5.211-py2.7.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/httplib2-0.7.7-py2.7.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/selenium-2.28.0-py2.7.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/jellyfish-0.2.0-py2.7-macosx-10.5-i386.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/python_yelp-0.1.1-py2.7.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/pymongo-2.4.2_-py2.7-macosx-10.5-i386.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/lucene_querybuilder-0.1.6-py2.7.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/mechanize-0.2.5-py2.7.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/html2text-3.200.3-py2.7.egg',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python27.zip',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/plat-darwin',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/plat-mac',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/plat-mac/lib-scriptpackages',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/lib-tk',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/lib-old',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/lib-dynload',
 '/Users/vincentwarmerdam/Library/Python/2.7/lib/python/site-packages',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/PIL',
 '/Library/Python/2.7/site-packages',
 '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/IPython/extensions] 

With this information, I now knew where ipython looks for the modules that one can import. So I downloaded the requests library manually, added it to the same root directory such that the following directory exists:

/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/requests 

This folder contains the python modules that belong to requests. The only thing I now had to do was to make sure that ipython knows that this folder exists. Which was done by updating the sys.path.

req_link = '/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/requests'
sys.path.append(req_link) 

After this I no longer got the error.

import requests 

Just works.

Also after restarting ipython, I found that ipython automatically updates the new path into the sys.path list.