CODEWITHSUNDEEP
python-2.7tensorflowconda
Geoffrey Negiar
Geoffrey Negiar

Reputation: 849

Link Tensorflow from usr/bin/python to a conda environment python

I have Tensorflow with GPU installed on my main usr/bin/python, e.g. on an AWS EC2 machine and I'd like to use it within a conda environment. Is this possible without reinstalling tensorflow for each environment ?

EDIT: I found out about the virtual-env --system-site-packages option. Maybe an equivalent for Conda ? And especially for packages like Tensorflow-gpu that need to be built for the specific hardware, and are not available directly on pip.

Upvotes: 1

Views: 531

Answers (1)

Nehal J Wani
Nehal J Wani

Reputation: 16629

If tensorflow is installed in the system, then it will probably be present in one of these four directories:

/usr/local/lib/python2.7/dist-packages
/usr/local/lib/python2.7/site-packages
/usr/lib/python2.7/dist-packages
/usr/lib/python2.7/site-packages

To have the system packages also be available in conda, you can do the following:

(root) ~/condaexpts $ cat <<EOF > $CONDA_PREFIX/lib/python2.7/site-packages/systempkgs.pth
/usr/local/lib/python2.7/dist-packages
/usr/local/lib/python2.7/site-packages
/usr/lib/python2.7/dist-packages
/usr/lib/python2.7/site-packages
EOF

Then verify that these paths have been added:

(root) ~/condaexpts $ python
Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:42:40) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import sys
>>> sys.path
['', '/home/ubuntu/condaexpts/m2/lib/python27.zip', '/home/ubuntu/condaexpts/m2/lib/python2.7', '/home/ubuntu/condaexpts/m2/lib/python2.7/plat-linux2', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-tk', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-old', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-dynload', '/home/ubuntu/condaexpts/m2/lib/python2.7/site-packages', '/home/ubuntu/condaexpts/m2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/site-packages', '/usr/lib/python2.7/dist-packages']

And now you can import any system package.

If you don't want this change to be permanent for your conda environment, you can always add the directory containing system packages to PYTHONPATH environment variable:

(root) ~/condaexpts $ PYTHONPATH=$PYTHONPATH:/usr/local/lib/python2.7/dist-packages/ python
Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:42:40) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import sys
>>> sys.path
['', '/home/ubuntu/condaexpts', '/usr/local/lib/python2.7/dist-packages', '/home/ubuntu/condaexpts/m2/lib/python27.zip', '/home/ubuntu/condaexpts/m2/lib/python2.7', '/home/ubuntu/condaexpts/m2/lib/python2.7/plat-linux2', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-tk', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-old', '/home/ubuntu/condaexpts/m2/lib/python2.7/lib-dynload', '/home/ubuntu/condaexpts/m2/lib/python2.7/site-packages', '/home/ubuntu/condaexpts/m2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg']

Upvotes: 1

Related Questions