CODEWITHSUNDEEP
pythonpipanacondaconda
AlessioG
AlessioG

Reputation: 614

Anaconda: not explainable behaviour of packages installed through conda pip

I'm trying to export a conda env built using conda + pip (installed through conda), but I'm missing some of the packages I have installed through pip. Below the details of the process I have followed.

The OS I'm running on is:

bdauser@testsuselinux:~> cat /etc/os-release
NAME="SLES"
VERSION="12-SP3"
VERSION_ID="12.3"
PRETTY_NAME="SUSE Linux Enterprise Server 12 SP3"
ID="sles"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:12:sp3"

This is my anaconda version:

bdauser@testsuselinux:~> conda list anaconda
# packages in environment at /home/bdauser/anaconda3:
#
anaconda                  5.0.1            py36hd30a520_1
anaconda-client           1.6.5            py36h19c0dcd_0
anaconda-navigator        1.6.9            py36h11ddaaa_0
anaconda-project          0.8.0            py36h29abdf5_0

I have created and activated the anaconda env through: 

bdauser@testsuselinux:~> conda create --name libmad python=3.5
bdauser@testsuselinux:~> source activate libmad

I then have installed pip for conda through:

(libmad) bdauser@testsuselinux:~> conda install pip

I then successfully installed a package using pip for conda: 

(libmad) bdauser@testsuselinux:~> /home/bdauser/anaconda3/envs/libmad/bin/pip install nltk==3.4

I have checked if the package is actually installed through: 

(libmad) bdauser@testsuselinux:~/anaconda3/envs/libmad/bin> /home/bdauser/anaconda3/envs/libmad/bin/pip install nltk==3.4
Requirement already satisfied: nltk==3.4 in /home/bdauser/anaconda3/envs/libmad/lib/python3.5/site-packages (3.4)
Requirement already satisfied: six in /home/bdauser/anaconda3/envs/libmad/lib/python3.5/site-packages (from nltk==3.4) (1.12.0)
Requirement already satisfied: singledispatch in /home/bdauser/anaconda3/envs/libmad/lib/python3.5/site-packages (from nltk==3.4) (3.4.0.3)

At this point I have tried to export the conda env:

(libmad) bdauser@testsuselinux:~/anaconda3/envs/libmad/bin> conda env export
name: libmad
channels:
- defaults
dependencies:
- ca-certificates=2019.1.23=0
- certifi=2018.8.24=py35_1
- libedit=3.1.20181209=hc058e9b_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=8.2.0=hdf63c60_1
- libstdcxx-ng=8.2.0=hdf63c60_1
- ncurses=6.1=he6710b0_1
- openssl=1.0.2r=h7b6447c_0
- pip=10.0.1=py35_0
- python=3.5.6=hc3d631a_0
- readline=7.0=h7b6447c_5
- setuptools=40.2.0=py35_0
- sqlite=3.28.0=h7b6447c_0
- tk=8.6.8=hbc83047_0
- wheel=0.31.1=py35_0
- xz=5.2.4=h14c3975_4
- zlib=1.2.11=h7b6447c_3
prefix: /home/bdauser/anaconda3/envs/libmad

But nltk does not show up as you can see.

I tried to run a simple conda list and I can see the nltk package:

(libmad) bdauser@testsuselinux:~/anaconda3/envs/libmad/bin> conda list
# packages in environment at /home/bdauser/anaconda3/envs/libmad:
#
ca-certificates           2019.1.23                     0
certifi                   2018.8.24                py35_1
libedit                   3.1.20181209         hc058e9b_0
libffi                    3.2.1                hd88cf55_4
libgcc-ng                 8.2.0                hdf63c60_1
libstdcxx-ng              8.2.0                hdf63c60_1
ncurses                   6.1                  he6710b0_1
nltk                      3.4                       <pip>
openssl                   1.0.2r               h7b6447c_0
pip                       10.0.1                   py35_0
pip                       19.1.1                    <pip>
python                    3.5.6                hc3d631a_0
readline                  7.0                  h7b6447c_5
setuptools                41.0.1                    <pip>
setuptools                40.2.0                   py35_0
singledispatch            3.4.0.3                   <pip>
six                       1.12.0                    <pip>
sqlite                    3.28.0               h7b6447c_0
tk                        8.6.8                hbc83047_0
wheel                     0.31.1                   py35_0
xz                        5.2.4                h14c3975_4
zlib                      1.2.11               h7b6447c_3

If I run the same command with the export flag, I don't see nltk again:

(libmad) bdauser@testsuselinux:~/anaconda3/envs/libmad/bin> conda list --export
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: linux-64
ca-certificates=2019.1.23=0
certifi=2018.8.24=py35_1
libedit=3.1.20181209=hc058e9b_0
libffi=3.2.1=hd88cf55_4
libgcc-ng=8.2.0=hdf63c60_1
libstdcxx-ng=8.2.0=hdf63c60_1
ncurses=6.1=he6710b0_1
openssl=1.0.2r=h7b6447c_0
pip=10.0.1=py35_0
python=3.5.6=hc3d631a_0
readline=7.0=h7b6447c_5
setuptools=40.2.0=py35_0
sqlite=3.28.0=h7b6447c_0
tk=8.6.8=hbc83047_0
wheel=0.31.1=py35_0
xz=5.2.4=h14c3975_4
zlib=1.2.11=h7b6447c_3

Can anyone explain this behaviour? What am I doing wrong here? The goal is to export the conda env to another machine with all the packages installed through pip.

Thanks in advance, Alessio

Upvotes: 2

Views: 316

Answers (1)

merv
merv

Reputation: 77098

I suspect the issue is that you updated pip through PyPI and then used that to install your package. I think Conda has issues recognizing packages not installed through the Conda-installed pip. Instead, try using only pip installed through Conda:

conda create -n libmad python=3.5 pip
conda activate libmad
pip install nltk==3.4

Tip: You don't have to provide a full path to pip if you're in the activated environment.

Technically, we shouldn't have to include pip in the create step, because Conda always installs pip whenever python is installed; we'll leave it here for good measure. Also note, that you can get newer versions of pip through the Conda Forge channel (up to 18.0 for Python 3.5).

Then, conda env export gives:

name: libmad
channels:
  - defaults
dependencies:
  - ca-certificates=2019.1.23=0
  - certifi=2018.8.24=py35_1
  - libcxx=4.0.1=hcfea43d_1
  - libcxxabi=4.0.1=hcfea43d_1
  - libedit=3.1.20181209=hb402a30_0
  - libffi=3.2.1=1
  - ncurses=6.1=h0a44026_1
  - openssl=1.0.2r=h1de35cc_0
  - pip=10.0.1=py35_0
  - python=3.5.6=hc167b69_0
  - readline=7.0=h1de35cc_5
  - setuptools=40.2.0=py35_0
  - sqlite=3.28.0=ha441bb4_0
  - tk=8.6.8=ha441bb4_0
  - wheel=0.31.1=py35_0
  - xz=5.2.4=h1de35cc_4
  - zlib=1.2.11=h1de35cc_3
  - pip:
    - nltk==3.4
    - singledispatch==3.4.0.3
    - six==1.12.0

Upvotes: 2

Related Questions