Cannot connect Django to existing MySQL database on M1 mac

Question

UPDATE: I found a solution and possible cause; posted an answer with steps to make it work below.

So, I am having trouble connecting Django to an existing MySQL database (created from a dump for dev purposes). Here is my steps and all the terminal output:

Get database ready

1. Install MySql 5.7

2. Start MySQL brew services start mysql@5.7

3. Open MySQL Workbench and connect via password

4. Create new schema

5. Open MySQL Database dump; insert use climate at the top of file, run script -> Database is ready

Setting up Django

• create new python venv python3 -m venv good-env
• Activate venv source good-env/bin/activate
• Installing django python -m pip install Django

Downloading Django-3.2.2-py3-none-any.whl (7.9 MB)
|████████████████████████████████| 7.9 MB 3.8 MB/s 
Collecting sqlparse>=0.2.2
 Using cached sqlparse-0.4.1-py3-none-any.whl (42 kB)
Collecting asgiref<4,>=3.3.2
 Using cached asgiref-3.3.4-py3-none-any.whl (22 kB)
Collecting pytz
 Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB)
Installing collected packages: sqlparse, asgiref, pytz, Django
Successfully installed Django-3.2.2 asgiref-3.3.4 pytz-2021.1 sqlparse-0.4.1
WARNING: You are using pip version 20.1.1; however, version 21.1.1 is available.
You should consider upgrading via the '/Users/aaronkurz/Dev/2/good-env/bin/python -m pip install --upgrade pip' command.

• Creating a project django-admin startproject climatevisual; cd climatevisual

Connecting Django with the database

• Installing required drivers (I know only mysqlclient should suffice, but that led to the same result so I started trying to install both...)
  • pip install mysqlclient

Installing collected packages: mysqlclient
Successfully installed mysqlclient-2.0.3
WARNING: You are using pip version 20.1.1; however, version 21.1.1 is available.
You should consider upgrading via the '/Users/aaronkurz/Dev/2/good-env/bin/python3 -m pip install --upgrade pip' command.

• pip install mysql-connector-python

Collecting mysql-connector-python
  Using cached mysql_connector_python-8.0.24-py2.py3-none-any.whl (319 kB)
Collecting protobuf>=3.0.0
  Using cached protobuf-3.15.8-cp38-cp38-macosx_10_9_x86_64.whl (1.0 MB)
Collecting six>=1.9
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, protobuf, mysql-connector-python
Successfully installed mysql-connector-python-8.0.24 protobuf-3.15.8 six-1.16.0
WARNING: You are using pip version 20.1.1; however, version 21.1.1 is available.
You should consider upgrading via the '/Users/aaronkurz/Dev/2/good-env/bin/python3 -m pip install --upgrade pip' command.

• editing climatevisual/settings.py

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'climate',
        'USER': 'root',
        'PASSWORD': 'yourpassword',
        'HOST': 'localhost',
        'PORT': '3306',
    }
}

• auto generating the models python manage.py inspectdb THIS IS WHERE THE ERROR OCCURS:

Traceback (most recent call last):
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/MySQLdb/__init__.py", line 18, in <module>
    from . import _mysql
ImportError: dlopen(/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/MySQLdb/_mysql.cpython-38-darwin.so, 2): Symbol not found: _mysql_affected_rows
  Referenced from: /Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/MySQLdb/_mysql.cpython-38-darwin.so
  Expected in: flat namespace
 in /Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/MySQLdb/_mysql.cpython-38-darwin.so

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    main()
  File "manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    django.setup()
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/apps/config.py", line 301, in import_models
    self.models_module = import_module(models_module_name)
  File "/Users/aaronkurz/opt/anaconda3/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/contrib/auth/models.py", line 3, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/contrib/auth/base_user.py", line 48, in <module>
    class AbstractBaseUser(models.Model):
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/db/models/base.py", line 122, in __new__
    new_class.add_to_class('_meta', Options(meta, app_label))
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/db/models/base.py", line 326, in add_to_class
    value.contribute_to_class(cls, name)
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/db/models/options.py", line 207, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/utils/connection.py", line 15, in __getattr__
    return getattr(self._connections[self._alias], item)
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/utils/connection.py", line 62, in __getitem__
    conn = self.create_connection(alias)
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/db/utils.py", line 204, in create_connection
    backend = load_backend(db['ENGINE'])
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/db/utils.py", line 111, in load_backend
    return import_module('%s.base' % backend_name)
  File "/Users/aaronkurz/opt/anaconda3/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 15, in <module>
    import MySQLdb as Database
  File "/Users/aaronkurz/Dev/2/good-env/lib/python3.8/site-packages/MySQLdb/__init__.py", line 24, in <module>
    version_info, _mysql.version_info, _mysql.__file__
NameError: name '_mysql' is not defined

If this step would have worked; I would next have followed/finished the tutorial here.

I have already tried countless things,

• reinstalling MySQL,
• using the newest version of MySQL (even though I had to edit the DB dump then and it would not be possible in the real project since the project needs this version of MySQL)
• I have tried the solution the OP of this thread proposed
• also this answer
• also [this answer](also [this answer]) (and many more tbh)

Does somebody have a clue what is going wrong here?

If it helps: I am running macOS 11.3.1. Thanks!

Answer 1

So I found a solution to my problem! I think the problem is that there are some compatibility issues between the MySQLdb module and the M1 mac I am using. But I could connect the database and import the model using PyMySQL instead! The steps I took are the following:

• Change my settings.py database section to this:

import pymysql
# Database
# https://docs.djangoproject.com/en/3.2/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'mysql.connector.django',
        'NAME': 'climate',
        'USER': 'root',
        'PASSWORD': 'yourpassword',
        'HOST': 'localhost',
        'PORT': '3306',
    }
}
# Fake PyMySQL's version and install as MySQLdb
# https://adamj.eu/tech/2020/02/04/how-to-use-pymysql-with-django/
pymysql.version_info = (1, 4, 2, "final", 0)
pymysql.install_as_MySQLdb()

• run `pip install pymysql
• then run python manage.py inspectdb > models.py
• then run python manage.py migrate

Done!

Answer 2

I found there is an issue using pymysql because my Django project is using JSONField and the app is crashing with error that it requires MySQL 5.7+ version, even when my MySQL server version is 8.1. However I found that mysql-connector-python solves this for me. Run:

pip install mysql-connector-python

to get the module installed and configure your DATABASES like this:

DATABASES = {
  'default': {
    'ENGINE': 'mysql.connector.django',
     # Rest of your configuration
  }
}

This seems to fix all issues with working with MySQL in Django on Mac Silicon processors.