pytest and Failed: Database access not allowed, use the "django_db" mark, or the "db" or "transactional_db" fixtures to enable it

Question

During invoking pytest from the shell I get the following output, because my test is stored in apps.business.metrics.tools.tests.py, and during import of the module

apps/business/metrics/widgets/employees/utilization.py

makes a live call to SQL during module invocation. This is done by

get_metric_columns('EmployeeUtilization', shapers=SHAPERS)

and pytest complaints:

➜ pytest
=========================================================================== test session starts ===========================================================================
platform linux -- Python 3.6.8, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
Django settings: config.settings.local (from ini file)
rootdir: /home/dmitry/Projects/analytics/backend, inifile: pytest.ini
plugins: django-3.4.7, pylama-7.6.6, django-test-plus-1.1.1, celery-4.2.1
collected 60 items / 1 errors                                                                                                                                             

================================================================================= ERRORS ==================================================================================
__________________________________________________________ ERROR collecting apps/business/metrics/tools.tests.py __________________________________________________________
../../../.pyenv/versions/3.6.8/envs/cam/lib/python3.6/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
apps/business/metrics/__init__.py:3: in <module>
    from .widgets import *  # noqa
apps/business/metrics/widgets/__init__.py:1: in <module>
    from . import help  # noqa
apps/business/metrics/widgets/help.py:1: in <module>
    from .employees.utilization import EmployeeSwarmUtilization
apps/business/metrics/widgets/employees/utilization.py:19: in <module>
    get_metric_columns('EmployeeUtilization', shapers=SHAPERS)
apps/business/metrics/tools.py:132: in get_metric_columns
    m = get_metric(metric, period=p, shapers=shapers)
apps/business/metrics/data/__init__.py:23: in get_metric
    return metrics[name](*args, **kwargs)
apps/business/metrics/data/abstract.py:441: in __init__
    self._to_dataframe(self.sql or self._ingest())
apps/business/metrics/data/abstract.py:472: in _to_dataframe
    source, connection, params=query_params, index_col=self.index
../../../.pyenv/versions/3.6.8/envs/cam/lib/python3.6/site-packages/pandas/io/sql.py:381: in read_sql
    chunksize=chunksize)
../../../.pyenv/versions/3.6.8/envs/cam/lib/python3.6/site-packages/pandas/io/sql.py:1413: in read_query
    cursor = self.execute(*args)
../../../.pyenv/versions/3.6.8/envs/cam/lib/python3.6/site-packages/pandas/io/sql.py:1373: in execute
    cur = self.con.cursor()
../../../.pyenv/versions/3.6.8/envs/cam/lib/python3.6/site-packages/django/db/backends/base/base.py:255: in cursor
    return self._cursor()
../../../.pyenv/versions/3.6.8/envs/cam/lib/python3.6/site-packages/django/db/backends/base/base.py:232: in _cursor
    self.ensure_connection()
E   Failed: Database access not allowed, use the "django_db" mark, or the "db" or "transactional_db" fixtures to enable it.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================================= 1 error in 2.43 seconds =========================================================================

Is there a way to handle such situation with pytest?

I understand I can convert the get_metric_columns('EmployeeUtilization', shapers=SHAPERS) into a partial func and change the implementation, but is there any other way around?

Answer 1

In my case test imported some serializer that had field like this:

some_field = MultipleChoiceField(
    choices=Model.objects.values_list("name", flat=True), required=False
)

I fixed test by making my own validation method for simple CharField

some_field = serializers.CharField(required=False)

def validate_some_field(self, values):
    choices = Model.objects.values_list("name", flat=True)
    for value in values:
        if name not in choices:
            raise serializers.ValidationError(
                f"Invalid name: {name }"
            )
    return values

Answer 2

As the error suggests, you should use @pytest.mark.django_db, db or transactional_db.

@pytest.mark.django_db:

import pytest

from django.contrib.auth.models import User

@pytest.mark.django_db # Here
def test_1():
    count = User.objects.all().count()    
    assert count == 0

db:

import pytest

from django.contrib.auth.models import User
         # ↓↓ Here
def test_1(db):
    count = User.objects.all().count()    
    assert count == 0

transactional_db:

import pytest

from django.contrib.auth.models import User
         # ↓ ↓ ↓ Here ↓ ↓ ↓
def test_1(transactional_db):
    count = User.objects.all().count()    
    assert count == 0

@pytest.mark.django_db with @pytest.fixture:

import pytest

from django.contrib.auth.models import User

@pytest.fixture
def fixture_1():
    count = User.objects.all().count() 
    return count

@pytest.mark.django_db # Here
def test_1(fixture_1):
    count = fixture_1
    assert count == 0

test_1(db, fixture_1) with @pytest.fixture:

import pytest

from django.contrib.auth.models import User

@pytest.fixture
def fixture_1():
    count = User.objects.all().count() 
    return count
         # ↓↓ Here
def test_1(db, fixture_1):
    count = fixture_1
    assert count == 0

fixture_1(db) with @pytest.fixture:

import pytest

from django.contrib.auth.models import User

@pytest.fixture
def fixture_1(db): # <- db
    count = User.objects.all().count() 
    return count

def test_1(fixture_1):
    count = fixture_1
    assert count == 0

test_1(transactional_db, fixture_1) with @pytest.fixture:

import pytest

from django.contrib.auth.models import User

@pytest.fixture
def fixture_1():
    count = User.objects.all().count() 
    return count
         # ↓ ↓ ↓ Here ↓ ↓ ↓
def test_1(transactional_db, fixture_1):
    count = fixture_1
    assert count == 0

fixture_1(transactional_db) with @pytest.fixture:

import pytest

from django.contrib.auth.models import User

@pytest.fixture
def fixture_1(transactional_db): # <- transactional_db
    count = User.objects.all().count() 
    return count

def test_1(fixture_1):
    count = fixture_1
    assert count == 0

Answer 3

If you have the mark and you're still getting the error, ensure you're not accessing the database from outside a test function.

For example:

@pytest.mark.django_db
class TestEndpoint:
    user = User.objects.create()    # Accessing the database outside a test function will raise the error

    def test_endpoint(self):
        pass

Rather, move any code that accesses the database into the test function. If it's an object you wanna re-use, I suggest making it a fixture.

Answer 4

Another way to solve this is to inherit from the Django TestCase in your test class:

from django.test import TestCase


class TestExampleTestCase(TestCase):
    def test_one():
        ...

Make sure you import django.test.TestCase and not unittest.TestCase.

The accepted answer should also work, but this will give you additional tooling provided by the Django test framework and is the standard way of writing tests according to the official Django docs on testing.

Answer 5

Solution:

import pytest


@pytest.mark.django_db
class TestExample:
    def test_one():
        ...

Assume that you've created a TestExample class inside your test file and it should be decorated with @pytest.mark.django_db. It should solve your problem.