CODEWITHSUNDEEP
pythonpandaspip
SilverXer0
SilverXer0

Reputation: 491

ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects

I'm trying to install embeddinghub on mac (tried pip3 install embeddinghub and pip3 install pandas embeddinghub protobuf), but I get the same error saying "could not build wheels for hnswlib, which is required to install pyproject.toml-based projects." I can't find any solutions regarding hnswlib. I've tried a lot of solutions regarding PEP517 and building wheels for other things, but none of the solutions helped. Here's the full error:

Error

Collecting pandas
  Using cached pandas-1.5.0-cp310-cp310-macosx_11_0_arm64.whl (10.8 MB)
Collecting embeddinghub
  Using cached embeddinghub-0.0.1.post12-py3-none-any.whl
Collecting protobuf
  Using cached protobuf-4.21.7-cp37-abi3-macosx_10_9_universal2.whl (484 kB)
Collecting pytz>=2020.1
  Using cached pytz-2022.4-py2.py3-none-any.whl (500 kB)
Collecting python-dateutil>=2.8.1
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting numpy>=1.21.0
  Using cached numpy-1.23.3-cp310-cp310-macosx_11_0_arm64.whl (13.3 MB)
Collecting grpcio==1.40.0
  Using cached grpcio-1.40.0-cp310-cp310-macosx_10_10_universal2.whl
Collecting hnswlib==0.5.2
  Using cached hnswlib-0.5.2.tar.gz (29 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy>=1.21.0
  Using cached numpy-1.21.1-cp310-cp310-macosx_12_0_universal2.whl
Collecting six==1.16.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: hnswlib
  Building wheel for hnswlib (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for hnswlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      running bdist_wheel
      running build
      running build_ext
      creating var
      creating var/folders
      creating var/folders/3h
      creating var/folders/3h/0w98c6q56bl4x4c2qm31np_c0000gn
      creating var/folders/3h/0w98c6q56bl4x4c2qm31np_c0000gn/T
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/Users/s/venv/include -I/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c /var/folders/3h/0w98c6q56bl4x4c2qm31np_c0000gn/T/tmp5hthbs8k.cpp -o var/folders/3h/0w98c6q56bl4x4c2qm31np_c0000gn/T/tmp5hthbs8k.o -std=c++14
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/Users/s/venv/include -I/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c /var/folders/3h/0w98c6q56bl4x4c2qm31np_c0000gn/T/tmpipibvgip.cpp -o var/folders/3h/0w98c6q56bl4x4c2qm31np_c0000gn/T/tmpipibvgip.o -fvisibility=hidden
      building 'hnswlib' extension
      creating build
      creating build/temp.macosx-10.9-universal2-cpython-310
      creating build/temp.macosx-10.9-universal2-cpython-310/python_bindings
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -I/private/var/folders/3h/0w98c6q56bl4x4c2qm31np_c0000gn/T/pip-build-env-hhht57g1/overlay/lib/python3.10/site-packages/pybind11/include -I/private/var/folders/3h/0w98c6q56bl4x4c2qm31np_c0000gn/T/pip-build-env-hhht57g1/overlay/lib/python3.10/site-packages/numpy/core/include -I./hnswlib/ -I/Users/s/venv/include -I/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c ./python_bindings/bindings.cpp -o build/temp.macosx-10.9-universal2-cpython-310/./python_bindings/bindings.o -O3 -march=native -stdlib=libc++ -mmacosx-version-min=10.7 -DVERSION_INFO=\"0.5.2\" -std=c++14 -fvisibility=hidden
      clang: error: the clang compiler does not support '-march=native'
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for hnswlib
Failed to build hnswlib
ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects

Upvotes: 47

Views: 117309

Answers (25)

Limza Dewangan
Limza Dewangan

Reputation: 1

I also faced this issue on Windows OS, this is worked for me

Download the build tools from download the visual-cpp-build-tools Open the installer, under Workloads select "Desktop development with C++" then click install. Restart the computer.

Upvotes: 0

James Cail
James Cail

Reputation: 1

conda install conda-forge::hnswlib

works in Windows.

Upvotes: 0

Ganesha Nagarajappa
Ganesha Nagarajappa

Reputation: 1

I encountered with same issue on RHEL 9.3. Here is the error message. Please suggest me how to resolve this issue

In file included from /tmp/pip-build-env-bqi319ry/overlay/lib/python3.9/site-packages/pybind11/include/pybind11/attr.h:13, from /tmp/pip-build-env-bqi319ry/overlay/lib/python3.9/site-packages/pybind11/include/pybind11/detail/class.h:12, from /tmp/pip-build-env-bqi319ry/overlay/lib/python3.9/site-packages/pybind11/include/pybind11/pybind11.h:12, from /tmp/pip-build-env-bqi319ry/overlay/lib/python3.9/site-packages/pybind11/include/pybind11/functional.h:14, from ./python_bindings/bindings.cpp:2: /tmp/pip-build-env-bqi319ry/overlay/lib/python3.9/site-packages/pybind11/include/pybind11/detail/common.h:274:10: fatal error: Python.h: No such file or directory 274 | #include <Python.h> | ^~~~~~~~~~ compilation terminated. error: command '/usr/bin/g++' failed with exit code 1 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for hnswlib Failed to build hnswlib ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (hnswlib)

Upvotes: 0

Gokhan Ozturk
Gokhan Ozturk

Reputation: 36

I have tried everything mentioned here, however nothing worked. I use an intel chipset mac.

Finally I explicitly gave the archflags proper to mine, before the pip command.

P.S. All the dependencies are under requirements.txt in my case.

ARCHFLAGS="-arch x86_64" pip3 install -r requirements.txt

Upvotes: 0

JoydeepG
JoydeepG

Reputation: 1

Error: This is for Mac

Building wheels for collected packages: chroma-hnswlib
  Building wheel for chroma-hnswlib (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for chroma-hnswlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [66 lines of output]
      running bdist_wheel on mac
File "<string>", line 102, in build_extensions
        File "<string>", line 69, in cpp_flag
      RuntimeError: Unsupported compiler -- at least C++11 support is needed!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for chroma-hnswlib
Failed to build chroma-hnswlib
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (chroma-hnswlib)*

Solution: This means, correct version of C++ has not been installed.

To install that, I did the following two things,

  1. Installed C++ extension for VS Code, (which will not install the C++ compiler)
  2. Then install C++ compiler by xcode-select –install, in the mac terminal (This takes around 20 mins.)
  3. After that pip install -r requirements.txt

This has resolved the issue

Upvotes: 0

Shuky A
Shuky A

Reputation: 11

I have faced this issue on Windows OS, this is what worked for me:

  1. Download the build tools from Microsoft C++ Build Tools.
  2. Open the installer, under Workloads select "Desktop development with C++" then click install.
  3. Restart the computer.

Previously, I had tried installing only "Individual components," as mentioned earlier, but that didn't work. Installing the entire "Desktop development with C++" workload resolved the issue.

Upvotes: 1

Rob
Rob

Reputation: 836

If you're using macOS, try setting an environment variable:

export HNSWLIB_NO_NATIVE=1

And then retry...

Upvotes: 63

Atul
Atul

Reputation: 688

Try using poetry version 1.6.1. It worked for me

Upvotes: 0

pgiouroukis
pgiouroukis

Reputation: 61

For Mac, I downloaded Xcode and it was fixed.

Upvotes: 0

Dr. Sushant_Bahekar
Dr. Sushant_Bahekar

Reputation: 161

I have faced similar issue on Windows OS, To resolve this

  1. Download built tools : https://visualstudio.microsoft.com/visual-cpp-build-tools/
  2. Open it and In Workloads select "Desktop development with C++" click install
  3. Restart your Python kernel
  4. !pip install chromadb==0.3.25

Upvotes: 1

Fahad Nadeem
Fahad Nadeem

Reputation: 411

For Ubuntu 20 and above, install dev tools and build essentials for Python version you're working with:

For Python3.10:

sudo apt install python3.10-dev && sudo apt-get install build-essential -y

For Python3.11:

sudo apt install python3.11-dev && sudo apt-get install build-essential -y

Upvotes: 8

Anil Kumar
Anil Kumar

Reputation: 445

On windows, without having "Microsoft Visual C++" installed below command worked :

conda activate your_env_name
conda install -c conda-forge chromadb

Upvotes: 1

Quantum Prophet
Quantum Prophet

Reputation: 455

Ok got it.... so a long winded solution for my use case but here it goes similar to one above:

  1. download https://visualstudio.microsoft.com/visual-cpp-build-tools/

  2. navigate to "Individual components", find select those that were called out above by someone else:

enter image description here

and

enter image description here

  1. finish the install

  2. reboot computer

  3. Then did a !pip install chromadb==0.3.26 selecting this specific version which worked for me. I do not think the most current one will work at this time, though I could be wrong.

Solutions I tried but didn't work for me, however worked for others:

  1. !pip install wheel

  2. !pip install --upgrade setuptools

Upvotes: 10

Atmanand Gauns
Atmanand Gauns

Reputation: 1

Its a problem of Visual Studio actually, i was trying to install hnswlib and to built the wheel for hnswlib, it requires the Visual studio 2022, so i upgraded, but the code written in python was not including the headers for ucrt from Windows 10 toolkit SDK. So there was no way to change the python code to include the library and also its not a good idea to change the system level files of python to include this :"-IC:\Program Files (x86)\Windows Kits\10\Include\10.0.22621.0\ucrt".

So the best solution i found is, to install the package using conda :

  1. Install conda using Anaconda or miniconda if you dont have one
  2. Create a conda virtual environment : conda create -n localgpt python==3.10.0.
  3. install the hnswlib package first from conda-forge : https://anaconda.org/conda-forge/hnswlib.
  4. Now install the requirements for your project using pip : pip install -r requirements.txt This should work fine, as packages form conda-forge dont build the wheels on your system.

Upvotes: 0

Rumaysa
Rumaysa

Reputation: 1

If you're on Mac you can install/update xcode dev tools:

xcode-select --install

This has worked for me.

Upvotes: 0

Pate Sting
Pate Sting

Reputation: 11

if you using windows, try set HNSWLIB_NO_NATIVE=1

Upvotes: 1

Broadsword27
Broadsword27

Reputation: 9

Downgraded Python to 3.10.10

export HNSWLIB_NO_NATIVE=1 
pip install HNSWLIB

Upvotes: 0

XFHL
XFHL

Reputation: 1

Ok so I also encoutered this problem when installing lang chain package to my Windows devices. I have download microsoft visual studio before, and I figured that you can just run the microsoft visual studio before you install, it will then work.

Upvotes: 0

Sathisbdst
Sathisbdst

Reputation: 1

In Mac, install GCC. It should work.

brew info gcc
brew install gcc

Upvotes: 0

Sam
Sam

Reputation: 1

None of these solutions worked for me, what did work for me in the end was using Anaconda to do the build rather than VisualStudio.

Upvotes: 0

Juan Rodriguez
Juan Rodriguez

Reputation: 469

In Ubuntu, these commands solved this problem, perhaps a Mac alternative could be useful too:

sudo apt install python3-dev
sudo apt-get install build-essential -y

Upvotes: 20

Varada
Varada

Reputation: 1075

I have faced similar issue on Windows OS, while doing pip install chromadb==0.3.22.

To Resolve this issue,

You need to download https://visualstudio.microsoft.com/visual-cpp-build-tools/ first.

Next, navigate to "Individual components", find these two

enter image description here

and Windows 10 SDK

enter image description here

source: https://github.com/chroma-core/chroma/issues/189#issuecomment-1454418844

Upvotes: 77

BuzzCloudAU
BuzzCloudAU

Reputation: 163

This worked for me

Install Microsoft Visual C++ Build Tools on your system. You can download and install it from the official Microsoft website: https://visualstudio.microsoft.com/visual-cpp-build-tools/.

Upvotes: 2

IndPythCoder
IndPythCoder

Reputation: 753

Clang compiler is not found. check c/c++ version clang -v. if not found

sudo xcode-select --install

Upvotes: 0

Tony Mai
Tony Mai

Reputation: 81

clang: error: the clang compiler does not support '-march=native'

error: command '/usr/bin/clang' failed with exit code 1

It looks like the problem is caused by the clang compiler, which did not support the flag -march=native previously.

The following commit added support of the flag, and you may need to install the latest Xcode for it.

https://github.com/llvm/llvm-project/commit/fcca10c69aaab539962d10fcc59a5f074b73b0de

Upvotes: 1

Related Questions