CODEWITHSUNDEEP
pythondjangodjango-modelsmod-wsgidjango-commands
FlipperPA
FlipperPA

Reputation: 14351

Django: disable initial system checks in production?

I've scoured the documentation and am looking for a Django setting that disables system checks (not just silences them) in production. I have a project with 20,000+ models which are autogenerated to create RESTful endpoints. These system checks take quite some time:

https://docs.djangoproject.com/en/1.11/ref/checks/#models

Having the systems check in development is necessary, even though it causes manage.py 20-30 minutes to fire up. However, any time I publish a new version to production, the first HTTP request to a production node takes 20-30 minutes to respond as well! I'd obviously like to avoid this, because after the initial request, the site is lightning fast.

While the answer in comments below references a solution to get runserver to come up more quickly, I'm looking for a solution for production, not our development environment.

I've looked around for a setting like DISABLED_SYSTEM_CHECKS but have only come across SILENCED_SYSTEM_CHECKS (see here), but that just seems to silence the output rather than not running the checks that take the time. Does such an animal exist? I'm running mod_wsgi in production. I've seen requires_system_checks for individual commands, but am looking for a project-wide solution. Thanks very much.

Upvotes: 2

Views: 1670

Answers (2)

FlipperPA
FlipperPA

Reputation: 14351

I posted this question long ago, and never posted the solution I ended up using. The root problem was that with that many models (we're now up to 60,000!), each one would be validated on load. urls.py contained all the end point routes, which would import the DRF ViewSets, which in turn would load the serializers and models up-front. So what I did was create a sort of lazy loading, and we've open sourced it:

https://pypi.org/project/automagic-rest/

The key element was modifying the ViewSet to load the model on-demand in __init__, rather than up front:

self.model = getattr(
    import_module(f"{self.python_path_name}.models.{self.schema_name}"),
    f"{self.schema_name}_{self.table_name}_model",
)

You can view the full source in context here: https://github.com/wharton/automagic-rest/blob/master/automagic_rest/views.py#L53

This required developing a convention for DRF's basename to hold the database name, app name, schema name, and model name, but it works well. The initial load time is now about 45 seconds for 60,000 models, rather than over three hours.

Upvotes: 0

radtek
radtek

Reputation: 36360

You could create a DISABLE_CHECKS setting and force skip the check from within the check functions themselves. I noticed even if you set SILENCED_SYSTEM_CHECKS in settings.py, certain manage.py commands will still run the checks (such as migrate). Here is what I use:

import logging

from django.conf import settings
from django.core.checks import Error
from django.db import connections
from django.core.cache import caches

def check_cache_connectivity(app_configs, **kwargs):
    """
    Check cache
    :param app_configs:
    :param kwargs:
    :return:
    """
    errors = []

    # Short circuit here, checks still ran by manage.py cmds regardless of SILENCED_SYSTEM_CHECKS
    if settings.DISABLE_CHECKS:
        return errors

    cache_settings = settings.CACHES.keys()
    for cur_cache in cache_settings:
        try:
            key = 'check_cache_connectivity_{}'.format(cur_cache)
            caches[cur_cache].set(key, 'connectivity_ok', 30)
            value = caches[cur_cache].get(key)
            print("Cache '{}' connection ok, key '{}', value '{}'".format(cur_cache, key, value))
        except Exception as e:
            msg = "ERROR: Cache {} looks to be down. {}".format(cur_cache, e)
            print(msg)
            logging.exception(msg)
            errors.append(
                Error(
                    msg,
                    hint="Unable to connect to cache {}, set as {}. {}"
                         "".format(cur_cache, settings.CACHES[cur_cache], e),
                    obj='CACHES.{}'.format(cur_cache),
                    id='content_services.E002',
                )
            )
    return errors

I use this in my build environment where I want most if not all of my custom checks ignored. Hope it helps!

Upvotes: 1

Related Questions