CODEWITHSUNDEEP
yamlpytorchcondaminiconda
dumbledad
dumbledad

Reputation: 17527

Specifying cpu-only for pytorch in conda YAML file

I can set up a conda environment successfully as follows:

conda create --name temp python=3.8.5
conda install pytorch==1.6.0 torchvision==0.7.0 cpuonly -c pytorch

I then save the environment to a YAML config file. The looks like this:

name: temp
channels:
  - pytorch
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - blas=1.0=mkl
  - ca-certificates=2020.10.14=0
  - certifi=2020.6.20=pyhd3eb1b0_3
  - cpuonly=1.0=0
  - freetype=2.10.4=h5ab3b9f_0
  - intel-openmp=2020.2=254
  - jpeg=9b=h024ee3a_2
  - lcms2=2.11=h396b838_0
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libedit=3.1.20191231=h14c3975_1
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.1.0=hdf63c60_0
  - libpng=1.6.37=hbc83047_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - libtiff=4.1.0=h2733197_1
  - lz4-c=1.9.2=heb0550a_3
  - mkl=2020.2=256
  - mkl-service=2.3.0=py38he904b0f_0
  - mkl_fft=1.2.0=py38h23d657b_0
  - mkl_random=1.1.1=py38h0573a6f_0
  - ncurses=6.2=he6710b0_1
  - ninja=1.10.1=py38hfd86e86_0
  - numpy=1.19.2=py38h54aff64_0
  - numpy-base=1.19.2=py38hfa32c7d_0
  - olefile=0.46=py_0
  - openssl=1.1.1h=h7b6447c_0
  - pillow=8.0.1=py38he98fc37_0
  - pip=20.2.4=py38h06a4308_0
  - python=3.8.5=h7579374_1
  - pytorch=1.6.0=py3.8_cpu_0
  - readline=8.0=h7b6447c_0
  - setuptools=50.3.0=py38h06a4308_1
  - six=1.15.0=py_0
  - sqlite=3.33.0=h62c20be_0
  - tk=8.6.10=hbc83047_0
  - torchvision=0.7.0=py38_cpu
  - wheel=0.35.1=py_0
  - xz=5.2.5=h7b6447c_0
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.4.5=h9ceee32_0
prefix: /data/anaconda/envs/temp

But if I try making a conda environment from the following file:

name: temp
channels:
  - defaults
  - pytorch
dependencies:
  - python==3.8.5
  - pytorch==1.6.0=py3.8_cpu_0

it fails, with the following incompatibilities:

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Package ld_impl_linux-64 conflicts for:
python==3.8.5 -> ld_impl_linux-64
Package sqlite conflicts for:
python==3.8.5 -> sqlite[version='>=3.32.3,<4.0a0|>=3.33.0,<4.0a0']
Package * conflicts for:
pytorch==1.6.0=py3.8_cpu_0 -> *[track_features=cpuonly]
Package ncurses conflicts for:
python==3.8.5 -> ncurses[version='>=6.2,<7.0a0']
Package mkl conflicts for:
pytorch==1.6.0=py3.8_cpu_0 -> mkl[version='>=2018']
Package blas conflicts for:
pytorch==1.6.0=py3.8_cpu_0 -> blas=[build=mkl]
Package zlib conflicts for:
python==3.8.5 -> zlib[version='>=1.2.11,<1.3.0a0']
Package openssl conflicts for:
python==3.8.5 -> openssl[version='>=1.1.1g,<1.1.2a']
Package python conflicts for:
pytorch==1.6.0=py3.8_cpu_0 -> python[version='>=3.8,<3.9.0a0']
Package xz conflicts for:
python==3.8.5 -> xz[version='>=5.2.5,<6.0a0']
Package libffi conflicts for:
python==3.8.5 -> libffi[version='>=3.3,<3.4.0a0']
Package libgcc-ng conflicts for:
python==3.8.5 -> libgcc-ng[version='>=7.3.0']
Package tk conflicts for:
python==3.8.5 -> tk[version='>=8.6.10,<8.7.0a0']
Package pip conflicts for:
python==3.8.5 -> pip
Package ninja conflicts for:
pytorch==1.6.0=py3.8_cpu_0 -> ninja
Package readline conflicts for:
python==3.8.5 -> readline[version='>=8.0,<9.0a0']
Package numpy conflicts for:
pytorch==1.6.0=py3.8_cpu_0 -> numpy[version='>=1.11']

Why? How could such a simple configuration, which was cut down from a successful environment, fail? How should I specify the CPU-only version of pytorch 1.6.0 in a YAML config file for a conda environment?

Upvotes: 7

Views: 6363

Answers (2)

Blithering
Blithering

Reputation: 600

Option 1

Add cpuonly as a package

name: temp
channels:
  - defaults
  - pytorch
dependencies:
  - python==3.8.5
  - cpuonly
  - pytorch==1.6.0

Option 2

Add +cpuonly suffix to the relevant packages

name: temp
channels:
  - defaults
  - pytorch
dependencies:
  - python==3.8.5
  - pytorch==1.6.0+cpuonly

Upvotes: 8

merv
merv

Reputation: 76820

For systems that have optional CUDA support (Linux and Windows) PyTorch provides a mutex metapackage cpuonly that when installed constrains the pytorch package solve to only non-CUDA builds. Going through the PyTorch installation widget will suggest including the cpuonly package when selecting "NONE" of the CUDA option

enter image description here

I don't know the internals of how to build packages that use such mutex metapackages, but mutex metapackages are documented with metapackages in general, and the docs include links to MKL vs OpenBLAS examples.

Exactly why the simple YAML you started with fails is still unclear to me, but my guess is that cpuonly constrains more than just the pytorch build and having the specific pytorch build alone is not sufficient to constrain its dependencies.

Upvotes: 9

Related Questions