Django - Custom logging handler class can't be picked up

Question

I'm developing a Django app (Django 2.0.7, Python 3.7, Windows 10, PyCharm) and I need to code a custom logging handler in order to store log messages into the database.

I'm getting this error when i try to run the application:

pydev debugger: process 27148 is connecting

Connected to pydev debugger (build 192.6817.19)
pydev debugger: process 28448 is connecting

Unhandled exception in thread started by <_pydev_bundle.pydev_monkey._NewThreadStartupWithTrace object at 0x000001F1C52AD648>
Traceback (most recent call last):
  File "C:\Program Files\Python\lib\logging\config.py", line 387, in resolve
    found = getattr(found, frag)
AttributeError: module 'Inmobiliaria.inmobiliaria.utils.logs' has no attribute 'handlers'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python\lib\logging\config.py", line 562, in configure
    handler = self.configure_handler(handlers[name])
  File "C:\Program Files\Python\lib\logging\config.py", line 712, in configure_handler
    klass = self.resolve(cname)
  File "C:\Program Files\Python\lib\logging\config.py", line 389, in resolve
    self.importer(used)
  File "C:\Users\facun\Desktop\DEV\forreal\backend\Inmobiliaria\inmobiliaria\utils\logs\handlers.py", line 2, in <module>
    from Inmobiliaria.inmobiliaria.services.application_log_services import create_application_log
  File "C:\Users\facun\Desktop\DEV\forreal\backend\Inmobiliaria\inmobiliaria\services\application_log_services.py", line 1, in <module>
    from Inmobiliaria.inmobiliaria.models import Applicationlog
  File "C:\Users\facun\Desktop\DEV\forreal\backend\Inmobiliaria\inmobiliaria\models.py", line 9, in <module>
    UserModel = get_user_model()
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\contrib\auth\__init__.py", line 194, in get_user_model
    return django_apps.get_model(settings.AUTH_USER_MODEL, require_ready=False)
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\apps\registry.py", line 192, in get_model
    self.check_apps_ready()
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\apps\registry.py", line 127, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm 2019.2.2\helpers\pydev\_pydev_bundle\pydev_monkey.py", line 747, in __call__
    ret = self.original_func(*self.args, **self.kwargs)
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\utils\autoreload.py", line 225, in wrapper
    fn(*args, **kwargs)
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\core\management\commands\runserver.py", line 112, in inner_run
    autoreload.raise_last_exception()
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\utils\autoreload.py", line 248, in raise_last_exception
    raise _exception[1]
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\core\management\__init__.py", line 327, in execute
    autoreload.check_errors(django.setup)()
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\utils\autoreload.py", line 225, in wrapper
    fn(*args, **kwargs)
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\__init__.py", line 19, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "C:\Users\facun\Envs\for-real\lib\site-packages\django\utils\log.py", line 73, in configure_logging
    logging_config_func(logging_settings)
  File "C:\Program Files\Python\lib\logging\config.py", line 799, in dictConfig
    dictConfigClass(config).configure()
  File "C:\Program Files\Python\lib\logging\config.py", line 570, in configure
    '%r' % name) from e
ValueError: Unable to configure handler 'db_handler'

And this is my code so far:

----settings.py (handlers part of LOGGING dict)----

'handlers': {
        'debug_file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': 'Logs/DebugTest.log',
        },
        'info_file': {
            'level': 'INFO',
            'class': 'logging.handlers.RotatingFileHandler',
            'maxBytes': 1024*1024*10,  # 10MB
            'backupCount': 10,
            'filename': 'Logs/InfoTest.log',
            'formatter': 'verbose',
        },
        'db_handler': {
            'level': 'INFO',
            'class': 'Inmobiliaria.inmobiliaria.utils.logs.handlers.DatabaseHandler',
            'formatter': 'verbose',
        },
    },

----handlers.py----

import logging
from Inmobiliaria.inmobiliaria.services.application_log_services import create_application_log


class DatabaseHandler(logging.Handler):
    """
    Logging handler that inserts log messages into the database
    """
    def __init__(self):
        logging.Handler.__init__(self)

    def emit(self, record):
        msg: logging.LogRecord = self.format(record)
        ret_val = create_application_log(msg.msg, msg.levelname, msg.thread, msg.process, msg.filename, msg.lineno, msg.created)
        print(ret_val)

This handlers.py file is located at

Inmobiliaria
    inmobiliaria
        utils
            logs
                handlers.py <-----

I hope you can help me.

Many thanks!

Answer 1

I have the same problem, and I found that import the django models inside a def function may work, just like:

    def emit(self, record):
        from mylogs.models import MysqlLogs
        log = MysqlLogs()
        log.desc = record.msg
        log.save()

Answer 2

This error is caused by a chicken-and-egg problem. If I understand correctly, the handler is logging to the database by using a django model. All django models require settings to be loaded and django apps to be initialized before they can be used. But logging has to be initialized before the apps to can be loaded. In order to fix this problem, you'd have to remove the handler's dependency on the model by, for example, creating a connection outside of the django model and inserting it. Alternatively, you can try lazy importing the use of the model by making it a local import.