PYTHON 3.7.4 NOT USING SQLITE 3.29.0

Question

OS/Software installed:

root@TACIT admin]# cat /etc/*release*
CentOS Linux release 7.6.1810 (Core) 

root@TACIT admin]# python3.7 --version
Python 3.7.4

[root@TACIT admin]# sqlite3 --version
3.29.0 2019-07-10 17:32:03 fc82b73eaac8b36950e527f12c4b5dc1e147e6f4ad2217ae43ad82882a88bfa6

(T3PSA) [root@TACIT src]# django-admin --version
2.2

Software locations:

[root@TACIT admin]# which python3.7
/usr/local/bin/python3.7

[root@TACIT admin]# which sqlite3
/usr/bin/sqlite3

(T3PSA) [root@TACIT src]# which django-admin
/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/bin/django-admin

I compiled Python 3.7.4 from source and installed following these instructions without any problems:

https://tecadmin.net/install-python-3-7-on-centos/

I upgraded from Sqlite 3.7.17 to Sqlite 3.29.0 following these instructions without any problems (other than I had to install some additional ".so" libraries):

https://linuxhint.com/upgrade-to-latest-sqlite3-on-centos7/

Unfortunately Python 3.7.4 is still using the old version of Sqlite3 (3.7.17):

[root@TACIT admin]# python3.7
Python 3.7.4 (default, Aug 16 2019, 16:34:12) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version        
'3.7.17'

I'm using Django and need it to run at least v3.8.3 (see the very end of the following output):

(T3PSA) [root@TACIT src]# python manage.py makemigrations
Traceback (most recent call last):
  File "manage.py", line 21, in <module>
main()
  File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/core/management/__init__.py", line 357, in execute
django.setup()
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/apps/registry.py", line 114, in populate
app_config.import_models()
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/apps/config.py", line 211, in import_models
self.models_module = import_module(models_module_name)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/contrib/auth/models.py", line 2, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/contrib/auth/base_user.py", line 47, in <module>
class AbstractBaseUser(models.Model):
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/models/base.py", line 117, in __new__
new_class.add_to_class('_meta', Options(meta, app_label))
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/models/base.py", line 321, in add_to_class
value.contribute_to_class(cls, name)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/models/options.py", line 204, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/__init__.py", line 28, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/utils.py", line 201, in __getitem__
backend = load_backend(db['ENGINE'])
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/utils.py", line 110, in load_backend
return import_module('%s.base' % backend_name)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 66, in <module>
check_sqlite_version()
  File "/root/.local/share/virtualenvs/T3PSA-6bzDXn0f/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 63, in check_sqlite_version
raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)
django.core.exceptions.ImproperlyConfigured: **SQLite 3.8.3 or later is required (found 3.7.17).**

I've looked at the following posts to try to figure this out but I think I don't know enough about compiling code to figure out the proper solution. Nothing I have tried from these links works.

https://superuser.com/questions/1425212/where-does-django-look-for-sqlite-instance-sqlite-3-8-3-or-later-is-required

https://unix.stackexchange.com/questions/434100/updating-the-sqlite-version-used-by-python-3-on-centos-7

django can't find new sqlite version? (SQLite 3.8.3 or later is required (found 3.7.17))

https://superuser.com/questions/1447285/python-wont-compile-with-a-new-sqlite3-on-centos-7

It could be that the instructions I'm following don't work because my software installation locations/paths are different from the users in the above posts, I'm not sure... that's why I noted "Software locations:" above. Or maybe I need to uninstall Python 3.7 somehow rather than 'reinstall over the top' of an existing installation? I installed from source, not sure if I need to do this and if so, how to go about it properly.

Any help resolving this would be greatly appreciated. Thanks in advance!

=========================================================================

This section added after getting instructions from FlipperPA

Followed instructions exactly to "Upgrade the CentOS 7 system install of SQLite to 3.29 by compiling it from source". Below is the result.

[root@TACIT src]# sqlite3 --version
3.29.0 2019-07-10 17:32:03 fc82b73eaac8b36950e527f12c4b5dc1e147e6f4ad2217ae43ad82882a88bfa6

I had already deleted my original source compile directory (/usr/src/Python-3.7.4) several times yesterday as I tried all of the other articles from the original post. So starting from scratch, I did this:

cd /usr/src
tar -zxvf Python-3.7.4.tgz
cd Python-3.7.4
./configure --enable-optimizations
make altinstall
make clean
make install

[root@TACIT Python-3.7.4]# python3.7
Python 3.7.4 (default, Aug 17 2019, 08:20:39) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.7.17'  

Python is still using the wrong version of SQLite. I did the 'make altinstall' command because CentOS 7.6 comes with Python 2.7.5 by default and I read that I shouldn't mess with that. So 'make altinstall' installs Python 3.7.4 in another location so as to leave the default installation of python fully intact.

=========================================================================

Manually uninstalling Python 3.7.4 (installed from source)

So I did the following:

find / -name *3.7*

The resultant list of files is below. The first group of files/folders I left alone. The second group of files/folders I removed.

/var/lib/yum/yumdb/s/27f7f0189d0898d0e87007d97102619629a4de6d-sqlite-3.7.17-8.el7-x86_64
/var/lib/yum/yumdb/s/e942412b453fd568720258d674b18e7fcd06a20f-sqlite-devel-3.7.17-8.el7-x86_64
/usr/lib/python2.7/site-packages/backports.ssl_match_hostname-3.7.0.1.dist-info
/usr/lib/python2.7/site-packages/configparser-3.7.3.dist-info
/usr/lib/python2.7/site-packages/flake8-3.7.7.dist-info
/usr/share/doc/sqlite-3.7.17
/usr/share/doc/git-1.8.3.1/RelNotes/1.5.3.7.txt

/usr/bin/easy_install-3.7
/usr/bin/pip3.7
/usr/bin/python3.7m
/usr/bin/python3.7
/usr/bin/python3.7m-config
/usr/bin/pydoc3.7
/usr/bin/idle3.7
/usr/bin/2to3-3.7
/usr/bin/pyvenv-3.7
/usr/lib/pkgconfig/python-3.7.pc
/usr/lib/python3.7
/usr/lib/python3.7/lib2to3/Grammar3.7.4.final.0.pickle
/usr/lib/python3.7/lib2to3/PatternGrammar3.7.4.final.0.pickle
/usr/lib/python3.7/config-3.7m-x86_64-linux-gnu
/usr/lib/python3.7/config-3.7m-x86_64-linux-gnu/libpython3.7m.a
/usr/lib/libpython3.7m.a
/usr/share/man/man1/python3.7.1
/usr/include/python3.7m
/usr/local/bin/easy_install-3.7
/usr/src/Python-3.7.4.tgz
/usr/src/Python-3.7.4
/usr/src/Python-3.7.4/Doc/whatsnew/3.7.rst
/usr/src/Python-3.7.4/build/lib.linux-x86_64-3.7
/usr/src/Python-3.7.4/build/temp.linux-x86_64-3.7
/usr/src/Python-3.7.4/build/temp.linux-x86_64-3.7/usr/src/Python-3.7.4
/usr/src/Python-3.7.4/build/scripts-3.7
/usr/src/Python-3.7.4/build/scripts-3.7/pydoc3.7
/usr/src/Python-3.7.4/build/scripts-3.7/idle3.7
/usr/src/Python-3.7.4/build/scripts-3.7/2to3-3.7
/usr/src/Python-3.7.4/build/scripts-3.7/pyvenv-3.7
/usr/src/Python-3.7.4/libpython3.7m.a

I then did the following steps:

tar -zxvf Python-3.7.4.tgz
cd Python-3.7.4
./configure --prefix=/usr --enable-optimizations
make
make altinstall
[root@TACIT Python-3.7.4]# python3.7
Python 3.7.4 (default, Aug 17 2019, 16:34:42) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3    
>>> sqlite3.sqlite_version
'3.7.17'

Not sure what else I can do here.

Answer 1

change

def check_sqlite_version():
if Database.sqlite_version_info < (3, 8, 3):
    raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)

to:

def check_sqlite_version():
if Database.sqlite_version_info < (3, 7, 17):
    raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)

Answer 2

UPDATE 2020-10-03:

I've compiled a version of _sqlite.so which you can drop at the root of your Python 3.8 project:

https://github.com/FlipperPA/django-s3-sqlite/raw/master/shared-objects/python-3-8/_sqlite3.so

If you drop this at the root of a Python or Django project, it should be picked up when Python's SQLite 3 wrapped attempts to _import sqlite, overriding this module: https://github.com/python/cpython/tree/b6f5b9dd21a37fa97af38521941025259f1556e0/Modules/_sqlite

Good luck!

ORIGINAL MESSAGE:

This is an issue with the included version of SQLite on CentOS 7 being quite old. Here's how I solved it, IIRC:

• Upgrade the CentOS 7 system install of SQLite to 3.29 by compiling it from source. Download the source, and then be sure to ./configure --prefix=/usr to upgrade the system version. Then make and sudo make install. You must include the prefix or it'll install to /usr/local instead by default!
• Make sure the system version installed is 3.29 with sqlite3 --version
• Re-compile Python 3.7.x. Be sure to do a make clean before sudo make install if you've already installed it.
• Open python3.7 and check the version. import sqlite3, followed by sqlite3.sqlite_version

That should work; the only difference is I installed from Python 3.6 from IUS Community. After upgrading the system SQLite, I did a sudo yum remove python36u then a sudo yum install python36u and the Python included SQLite had successful upgraded to 3.29. Good luck!

Answer 3

I think that OP must fix the path to sqlite3 while installing Python:

sudo LD_RUN_PATH=/usr/lib ./configure --enable-optimizations
sudo LD_RUN_PATH=/usr/lib make altinstall