CODEWITHSUNDEEP
python-3.xflaskscikit-learnvirtual-machinedebian-based
Sarmad Rezayat
Sarmad Rezayat

Reputation: 31

Problem with building Scikit-Learn for flask api on debian VM

I'm working on a project which is a Flask API that you can visit the API here. I deployed the project on Debian 11 VM, which globally runs python 3.9, but I made a 3.7 venv and my project runs in this environment because of dependencies in my code.

After the deployment, as you can see the API is running without any problem and there is no error in the apache, but when I try to use any features of the app it is not working and the error in my apache error log shows me that there is a problem in building scikit-learn.

4] ERROR:app:Exception on /login [POST]
[wsgi:error] Traceback (most recent call last):
[Wed Mar 22 09:19:15.300434 2023] [wsgi:error]  File "/home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/sklearn/__check_build/__init__.py", li>
[Wed Mar 22 09:19:15.300437 2023] [wsgi:error] from ._check_build import check_build  # noqa
[Wed Mar 22 09:19:15.300440 2023] [wsgi:error] ModuleNotFoundError: No module named 'sklearn.__check_build._check_build'
[Wed Mar 22 09:19:15.300442 2023] [wsgi:error]
[Wed Mar 22 09:19:15.300444 2023] [wsgi:error] During handling of the above exception, another exception occurred:
[Wed Mar 22 09:19:15.300446 2023] [wsgi:error] Traceback (most recent call last):
[Wed Mar 22 09:19:15.300450 2023] [wsgi:error]  File "/home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/flask/app.py", line 2528, in wsgi_app
[Wed Mar 22 09:19:15.300452 2023] [wsgi:error]  response = self.full_dispatch_request()
[Wed Mar 22 09:19:15.300454 2023] [wsgi:error]    File "/home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/flask/app.py", line 1825, in full_disp>
[Wed Mar 22 09:19:15.300457 2023] [wsgi:error]    rv = self.handle_user_exception(e)
[Wed Mar 22 09:19:15.300459 2023] [wsgi:error]  File "/home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/flask/app.py", line 1823, in full_disp>
[Wed Mar 22 09:19:15.300461 2023] [wsgi:error]    rv = self.dispatch_request()
[Wed Mar 22 09:19:15.300463 2023] [wsgi:error]   File "/home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_>
[Wed Mar 22 09:19:15.300465 2023] [wsgi:error]    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
[Wed Mar 22 09:19:15.300467 2023] [wsgi:error]   File "/home/admin/decision-support-system/app/api.py", line 546, in loginValidation
[Wed Mar 22 09:19:15.300469 2023] [wsgi:error]     from app.structure import machine_learning as starter
[Wed Mar 22 09:19:15.300471 2023] [wsgi:error]   File "/home/admin/decision-support-system/app/structure/machine_learning.py", line 8, in <module>
[Wed Mar 22 09:19:15.300473 2023] [wsgi:error]    from sklearn.compose import ColumnTransformer
[Wed Mar 22 09:19:15.300476 2023] [wsgi:error]    File "/home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/sklearn/__init__.py", line 81, in <mod>
[Wed Mar 22 09:19:15.300478 2023] [wsgi:error]      from . import __check_build  # noqa: F401
[Wed Mar 22 09:19:15.300480 2023] [wsgi:error]  File "/home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/sklearn/__check_build/__init__.py", li>
[Wed Mar 22 09:19:15.300494 2023] [wsgi:error]    raise_build_error(e)
[Wed Mar 22 09:19:15.300496 2023] [wsgi:error]    File "/home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/sklearn/__check_build/__init__.py", li>
[Wed Mar 22 09:19:15.300498 2023] [wsgi:error] raise ImportError(
[Wed Mar 22 09:19:15.300500 2023] [wsgi:error]  ImportError: No module named 'sklearn.__check_build._check_build'
[Wed Mar 22 09:19:15.300502 2023] [wsgi:error] ___________________________________________________________________________
[Wed Mar 22 09:19:15.300504 2023] [wsgi:error]  Contents of /home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/sklearn/__check_build:
[Wed Mar 22 09:19:15.300507 2023] [wsgi:error] __pycache__               __init__.py               setup.py
[Wed Mar 22 09:19:15.300509 2023] [wsgi:error] _check_build.cpython-37m-x86_64-linux-gnu.so
[Wed Mar 22 09:19:15.300511 2023] [wsgi:error]  ___________________________________________________________________________
[Wed Mar 22 09:19:15.300513 2023] [wsgi:error]  It seems that scikit-learn has not been built correctly.
[Wed Mar 22 09:19:15.300515 2023] [wsgi:error]
[Wed Mar 22 09:19:15.300516 2023] [wsgi:error] If you have installed scikit-learn from source, please do not forget
[Wed Mar 22 09:19:15.300518 2023] [wsgi:error]  to build the package before using it: run `python setup.py install` or
[Wed Mar 22 09:19:15.300520 2023] [wsgi:error]`make` in the source directory.
[Wed Mar 22 09:19:15.300522 2023] [wsgi:error] 
[Wed Mar 22 09:19:15.300524 2023] [wsgi:error]If you have used an installer, please check that it is suited for your
[Wed Mar 22 09:19:15.300526 2023] [wsgi:error] Python version, your operating system and your platform.

So I tried everything uninstall and reinstall it with pip or building it from a source like github. It never works!

I activated the environment in the terminal of the VM through an ssh connection and tried to import sklearn, and asked for the version, and this was the result.

sklearn.show_versions()
/home/admin/decision-support-system/env3.7/lib/python3.7/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")

System:
    python: 3.7.7 (default, Mar 16 2023, 01:45:54)  [GCC 10.2.1 20210110]
executable: /home/admin/decision-support-system/env3.7/bin/python
   machine: Linux-5.10.0-21-amd64-x86_64-with-debian-11.6

Python dependencies:
          pip: 23.0.1
   setuptools: 67.6.0
      sklearn: 1.0.2
        numpy: 1.18.5
        scipy: 1.7.3
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.3.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

It seems that everything is ok in the VENV. I also import the pip list result below: I see another post here , but my problem is that sklearn is working on the VENV!

pip list
Package                Version
---------------------- ---------
absl-py                1.4.0
aniso8601              8.0.0
appdirs                1.4.3
astor                  0.8.1
astunparse             1.6.3
async-timeout          4.0.2
attrs                  19.3.0
cachetools             4.2.4
certifi                2022.12.7
cffi                   1.14.0
chardet                3.0.4
charset-normalizer     3.1.0
click                  8.1.3
colorama               0.4.3
cryptography           2.9.2
cycler                 0.10.0
distlib                0.3.0
filelock               3.0.12
fire                   0.3.1
flasgger               0.9.4
Flask                  2.2.3
Flask-Cors             3.0.10
flask-restplus         0.13.0
Flask-SQLAlchemy       3.0.2
flask-swagger-ui       3.25.0
gast                   0.3.3
google-auth            1.35.0
google-auth-oauthlib   0.4.6
google-pasta           0.2.0
greenlet               2.0.2
grpcio                 1.51.3
h5py                   2.10.0
idna                   3.4
importlib-metadata     6.0.0
importlib-resources    5.12.0
itsdangerous           2.1.2
Jinja2                 3.0.3
joblib                 1.2.0
jsonschema             3.2.0
Keras                  2.4.0
Keras-Applications     1.0.8
Keras-Preprocessing    1.1.2
kiwisolver             1.2.0
Markdown               3.4.1
MarkupSafe             2.1.2
marshmallow            3.5.1
matplotlib             3.3.2
mistune                0.8.4
mohawk                 1.1.0
numpy                  1.18.5
oauthlib               3.2.2
opt-einsum             3.3.0
pandas                 1.3.4
pathspec               0.8.0
Pillow                 8.0.1
pip                    23.0.1
pipdate                0.3.1
pkgutil_resolve_name   1.3.10
protobuf               4.22.1
pyasn1                 0.4.8
pyasn1-modules         0.2.8
pycparser              2.20
PyJWT                  1.7.1
pymongo                3.10.1
pyOpenSSL              19.1.0
pyparsing              2.4.7
pyrsistent             0.16.0
python-dateutil        2.8.1
pytz                   2019.3
PyYAML                 5.3.1
redis                  4.5.1
repoze.lru             0.7
requests               2.28.2
requests-hawk          1.1.1
requests-oauthlib      1.3.1
Routes                 2.4.1
rsa                    4.9
scikit-learn           1.0.2
scipy                  1.7.3
setuptools             67.6.0
six                    1.16.0
SQLAlchemy             2.0.6
structure              0.1.dev16
tensorboard            2.2.2
tensorboard-plugin-wit 1.8.1
tensorflow             2.2.2
tensorflow-estimator   2.2.0
termcolor              2.2.0
Theano                 1.0.4
threadpoolctl          3.1.0
tinynetrc              1.3.0
typing_extensions      4.5.0
urllib3                1.26.15
uWSGI                  2.0.18
Werkzeug               2.2.3
wheel                  0.40.0
wrapt                  1.15.0
zipp                   3.15.0

There is also something that is worth mentioning when I try to build the library from a source, it has some errors that mention numpy in them but as you can see, the pip install works, and the import sklearn also works fine!

Is there any idea about what would be the solution here?

Upvotes: 0

Views: 130

Answers (1)

adrin
adrin

Reputation: 4896

The error you're getting means scikit-learn is not built properly. Also, note that python 3.7 is no longer supported by scikit-learn, that's why you're having and old version installed.

I would try creating the environment with micromamba/mamba/conda and install from conda-forge channel.

Upvotes: 0

Related Questions