CODEWITHSUNDEEP
python-3.xubuntu-20.04chromadb
Sinister _
Sinister _

Reputation: 61

hnswlib package issue while installing chromadb in ubuntu

I am using ubuntu 20.04 focal and trying to install chromadb by using 'pip install chromdb' but I am getting following error

  Building wheel for hnswlib (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for hnswlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [21 lines of output]
      running bdist_wheel
      running build
      running build_ext
      creating tmp
      x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/namanc/codes/langchian_chromadb/venv/include -I/usr/include/python3.10 -c /tmp/tmp822lkln8.cpp -o tmp/tmp822lkln8.o -std=c++14
      x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/namanc/codes/langchian_chromadb/venv/include -I/usr/include/python3.10 -c /tmp/tmpw33wg22s.cpp -o tmp/tmpw33wg22s.o -fvisibility=hidden
      building 'hnswlib' extension
      creating build
      creating build/temp.linux-x86_64-cpython-310
      creating build/temp.linux-x86_64-cpython-310/python_bindings
      x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-izlfj5h4/overlay/lib/python3.10/site-packages/pybind11/include -I/tmp/pip-build-env-izlfj5h4/overlay/lib/python3.10/site-packages/numpy/core/include -I./hnswlib/ -I/home/namanc/codes/langchian_chromadb/venv/include -I/usr/include/python3.10 -c ./python_bindings/bindings.cpp -o build/temp.linux-x86_64-cpython-310/./python_bindings/bindings.o -O3 -fopenmp -DVERSION_INFO=\"0.7.0\" -std=c++14 -fvisibility=hidden
      In file included from /tmp/pip-build-env-izlfj5h4/overlay/lib/python3.10/site-packages/pybind11/include/pybind11/detail/../attr.h:13,
                       from /tmp/pip-build-env-izlfj5h4/overlay/lib/python3.10/site-packages/pybind11/include/pybind11/detail/class.h:12,
                       from /tmp/pip-build-env-izlfj5h4/overlay/lib/python3.10/site-packages/pybind11/include/pybind11/pybind11.h:13,
                       from /tmp/pip-build-env-izlfj5h4/overlay/lib/python3.10/site-packages/pybind11/include/pybind11/functional.h:12,
                       from ./python_bindings/bindings.cpp:2:
      /tmp/pip-build-env-izlfj5h4/overlay/lib/python3.10/site-packages/pybind11/include/pybind11/detail/../detail/common.h:266:10: fatal error: Python.h: No such file or directory
        266 | #include <Python.h>
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for hnswlib
Failed to build hnswlib
ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects

I have tried to update the gcc complier too but error still persists

I tried to update the ubuntu and its gcc complier

Upvotes: 5

Views: 10068

Answers (5)

Eldinur the Kolibri
Eldinur the Kolibri

Reputation: 562

The error message suggests that the installation of chromdb is failing due to a problem with building the hnswlib wheel. The error specifically mentions that it cannot find the Python.h header file.

To resolve this issue, you need to install the development headers for Python. You can do this by running the following command (since you are on ubuntu):

sudo apt-get install python3-dev

This command will install the necessary development files for Python. Afterward, you can try installing chromdb again using pip install chromdb.

If you still encounter the same error, you can try updating your pip version to the latest version by running:

pip install --upgrade pip

Then attempt the installation again using pip install chromdb.

If the issue persists, please provide any additional error messages or logs you receive during the installation process, as they might provide further insight into the problem.

Upvotes: 7

Mario Cardoso
Mario Cardoso

Reputation: 34

I tryied all the solutions here and many others in the web. After some try and error in my computer as experiment, the only thing that worked for me was using python 3.8 and it's pip to install create and use a virtual environment that will install chromadb in it. I tried python 3.11 and 3.10 but none of them worked.

Upvotes: 0

John Ostrowick
John Ostrowick

Reputation: 85

This worked for me:

sudo apt-get install python3-dev
python3.10 -m pip install --upgrade pip
sudo apt install build-essential
python3.10 -m pip install hnswlib

Upvotes: 1

hot_pan_stand_back
hot_pan_stand_back

Reputation: 1

I was having almost the exact same error message when trying pip install chromadb (the only differences were I'm using python3.11, looks like you're on 3.10. Second, it looks like you're using Langchain and I'm running from a python script).

What ended up working for me was sudo apt install python3.11-dev, you should try installing python3.10. I had also tried sudo apt install python3-dev and this didn't work (which I think means I don't have 3.11 set as default, not sure).

I'd recommend trying this and if that doesn't work, try running a script outside of langchain (chromadb's homepage has a quick example). If it works standalone but not in langchain then you know it's langchain that's now the issue.

Upvotes: 0

MasterRohit
MasterRohit

Reputation: 31

As suggested by @Eldinur the Kolibri in their answer, installing development headers for Python does solve this. Although You may have to run python version-specific installation.

For python3.8, it will be like:

sudo apt-get install python3.8-dev

Upvotes: 3

Related Questions