How can I use Django's test client on multiple databases with ATOMIC_REQUESTS?

Question

I have two databases setup in my settings.py, each with ATOMIC_REQUESTS:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'lolomg',
        'USER': 'lolomg',
        'PASSWORD': 'password',
        'HOST': 'localhost',
        'PORT': '5432',
        'ATOMIC_REQUESTS': True,
    },

    'analytics': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'analytics',
        'USER': 'lolomg',
        'PASSWORD': 'password',
        'HOST': 'localhost',
        'PORT': '5432',
        'ATOMIC_REQUESTS': True,
    },
}

If I then write any tests that use Django's test client, e.g.

from django.test import TestCase

class TestEndpoints(TestCase):

    def test_books(self):
        self.client.get("api/v1/books")

I will get a traceback and an exception that look like this:

======================================================================
FAIL: test_books (lolomg.books.tests.test_api.EndpointsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jordi/vcs/lolomg/lolomg/books/tests/test_api.py", line 6, in test_books
    resp = self.client.get(self.url)
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/test/client.py", line 535, in get
    response = super().get(path, data=data, secure=secure, **extra)
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/test/client.py", line 347, in get
    **extra,
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/test/client.py", line 422, in generic
    return self.request(**r)
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/test/client.py", line 503, in request
    raise exc_value
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python3.7/contextlib.py", line 73, in inner
    with self._recreate_cm():
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/db/transaction.py", line 175, in __enter__
    if not connection.get_autocommit():
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/db/backends/base/base.py", line 379, in get_autocommit
    self.ensure_connection()
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/db/backends/base/base.py", line 217, in ensure_connection
    self.connect()
  File "/home/jordi/vcs/lolomg/lib/python3.7/site-packages/django/test/testcases.py", line 144, in __call__
    raise AssertionError(self.message)

AssertionError: Database connections to 'analytics' are not allowed in this test. Add 'analytics' to lolomg.books.tests.test_api.EndpointsTests.databases to ensure proper test isolation and silence this failure.

By following that stack trace, we see that this is happening because the test client is trying to establish a transaction for the analytics database due to ATOMIC_REQUESTS being set. Since Django 2.2, however, connexions to databases other than default have to now be explicitly enabled in tests:

https://docs.djangoproject.com/en/3.0/topics/testing/tools/#django.test.TransactionTestCase.databases

That's cool, but I actually don't want to connect to the analytics database at all for this test. In fact, there's only a handful of tests that require connecting to this database, so it seems wrong to even consider this database for all other tests.

While I could do databases = '__all__' for any test class that uses the test client, this also seems wrong. I just don't want the test client to be attempting to do anything at all with the analytics database for most tests.

So what should I do instead? How can I keep ATOMIC_REQUESTS on every database but only enable the analytics database for the tests that actually need it?

Answer 1

I'm still not hugely familiar with Django 2 yet but it seems that possibly the only way to achieve this is to disable the ATOMIC_REQUESTS setting for the analytics database in the test runner and then turn it back on explicitly in the tests which cover it.

See https://docs.djangoproject.com/en/2.2/topics/testing/tools/#overriding-settings

I don't think this is ideal as you will be running your tests in a different environment than your code will be running but I'm not sure there's another way without just enabling all databases for all tests.