Conda does not look for libpthread and libpthread_nonshared at the right place when compiling inside an env

Question

I have a python package with a Cython extension that I'm trying to compile.

I get the following error as the output of pip install:

 /home/user/anaconda3/envs/benchopt_lasso/bin/x86_64-conda-linux-gnu-c++ -pthread -shared 
     -B /home/user/anaconda3/envs/benchopt_lasso/compiler_compat -L/home/user/anaconda3/envs/benchopt_lasso/lib -Wl,
     -rpath=/home/user/anaconda3/envs/benchopt_lasso/lib -Wl,--no-as-needed -Wl,--sysroot=/ 
     -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined 
     -Wl,-rpath,/home/user/anaconda3/envs/benchopt_lasso/lib 
     -Wl,-rpath-link,/home/user/anaconda3/envs/benchopt_lasso/lib -L/home/user/anaconda3/envs/benchopt_lasso/lib -march=nocona 
     -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe 
     -isystem /home/user/anaconda3/envs/benchopt_lasso/include -DNDEBUG 
     -D_FORTIFY_SOURCE=2 -O2 -isystem /home/user/anaconda3/envs/benchopt_lasso/include build/temp.linux-x86_64-3.8/celer/lasso_fast.o 
     -o build/lib.linux-x86_64-3.8/celer/lasso_fast.cpython-38-x86_64-linux-gnu.so

/home/user/anaconda3/envs/benchopt_lasso/compiler_compat/ld: cannot find /lib64/libpthread.so.0

/home/user/anaconda3/envs/benchopt_lasso/compiler_compat/ld: cannot find /usr/lib64/libpthread_nonshared.a

collect2: error: ld returned 1 exit status

These libs are not in /usr/lib64, they are in /home/user/anaconda3/envs/benchopt_lasso/x86_64-conda-linux-gnu/sysroot/usr/lib64/

Why is conda's compiler not finding them ? I'm suspecting the option -sysroot=/, whereas the default value points to the correct location

/home/user/anaconda3/envs/benchopt_lasso/bin/x86_64-conda-linux-gnu-c++ --print-sysroot
/home/user/anaconda3/envs/benchopt_lasso/bin/../x86_64-conda-linux-gnu/sysroot

Steps to reproduce (with another package), in a fresh terminal, with the following env.yml file:

name: reprod 
channels:
  - conda-forge
  - nodefaults
dependencies:
  - python=3.8
  - numpy
  - cython
  - compilers
  - pip

conda env create -n reprod -f env.yml
conda activate reprod 
pip install git+https://bitbucket.org/mathurinm/cd_solver.git@module_structure -v

Changing to python=3.9 in the env requirement solves the issue.

Answer 1

TL;DR - when building a project with pyproject.toml, the default is to use build-isolation. This install setuptools from pypi, which is not configured to work with conda compilers. If you use the option --no-build-isolation, the package builds.

---

The issue here is that pyproject.toml build isolation process is not compatible with conda env that have its own compilers. There has been several report of this incompatibility, for instance this is mentionned here.

The core issue here is that setuptools relies on a package called sysconfigdata_xxx to get info on how to build packages (see here) and in particular fetch the linker flags. By default, it fetches the package '_sysconfigdata_{abi}_{platform}_{multiarch}'. When setuptools is installed with conda, it is patched to fetch a different sysconfig defined in the env variable _CONDA_PYTHON_SYSCONFIGDATA_NAME, set in the conda env. The _sysconfig_* module then uses the correct compilation flags.

With the build isolation, a new env is create (on linux in /tmp/pip-build-env-XXX/) and a new version of setuptools, different from the one in conda is installed. This reverts setuptools to the orignal behavior, i.e. fetching the orignal _sysconfig_* file. In python3.8, this package set the flag --sysroot=/ for the linker flags. Thus, the shared library for pthread are search not in conda libraries but in system /lib64 where it seems the right version are not present, which explains why this breaks for this config. This _sysconfig_* module has been changed in python3.9, and the --sysroot=/ flag is not set anymore. This is why it work on python3.9.

To solve this without breaking the build isolation, I think the best way would be to manage to automatically set _PYTHON_SYSCONFIGDATA_NAME (which is the primary source to find the module name) to point on _CONDA_PYTHON_SYSCONFIGDATA_NAME (which point to the right config) in the build env when it is called. This requires patching pip with the following patch (this works locally when patching pip in the reproducer env):

index daeb7fb..a1c92b4 100644
--- a/pip/_internal/build_env.py
+++ b/pip/_internal/build_env_modified.py
@@ -128,7 +128,8 @@ class BuildEnvironment:
     def __enter__(self) -> None:
         self._save_env = {
             name: os.environ.get(name, None)
-            for name in ("PATH", "PYTHONNOUSERSITE", "PYTHONPATH")
+            for name in ("PATH", "PYTHONNOUSERSITE", "PYTHONPATH",
+                         "_PYTHON_SYSCONFIGDATA_NAME")
         }
 
         path = self._bin_dirs[:]
@@ -145,6 +146,8 @@ class BuildEnvironment:
                 "PYTHONPATH": os.pathsep.join(pythonpath),
             }
         )
+        if '_PYTHON_SYSCONFIGDATA_NAME' not in os.environ:
+            os.environ['_PYTHON_SYSCONFIGDATA_NAME'] = os.environ['_CONDA_PYTHON_SYSCONFIGDATA_NAME']
 
     def __exit__(
         self,

Maybe this could be propagated on conda-forge/pip-feedstock but this seems unpractical. I did not see any way to change env variable in the pip build system just with config files or setup.py.