CODEWITHSUNDEEP
djangounit-testingsettingsstoragepatch
nvie
nvie

Reputation: 376

How to elegantly swap out (patch) a Django FileSystemStorage setting in your unit tests?

I'm faced with the following problem. I have a Model that looks somewhat like this:

class Package(models.Model):
    name = models.CharField(max_length=64)
    file = models.FileField(upload_to="subdir",
                            storage=settings.PACKAGE_STORAGE,
                            null=True)

Essential in this example is the storage= argument to the FileField constructor. It is filled with a value from settings.py. In there is the following code:

from django.core.files.storage import FileSystemStorage
PACKAGE_STORAGE = FileSystemStorage(location="/var/data", base_url="/")

For production use, this works fine. But in my unit tests, uploads I make are now written to /var/data, which contains production data. I tried to swap out the PACKAGE_STORE in packages/tests.py like this

from django.conf import settings     # This is line 1
from tempfile import mkdtemp
settings.PACKAGE_STORAGE = FileSystemStorage(location=mkdtemp(), base_url="/")

# rest of the imports and testing code below

but the real problem is that before the test file is loaded, the packages app and its models have been loaded already, and therefore, the PACKAGE_STORAGE setting has been resolved before I'm able to change it in the test setup code.

Is there an elegant way to override this specific setting in a testing context?

Upvotes: 7

Views: 3162

Answers (7)

freds
freds

Reputation: 31

If you are using pytest-django and want to keep a single Django configuration file you can create a conftest.py and have that setting overridden for every test executed (source)

@pytest.fixture(autouse=True)
def use_file_system_storage(settings):
    settings.PACKAGE_STORAGE = FileSystemStorage(location=mkdtemp(), base_url="/")

You can also change your app before Django sets up using pytest-django. See here.

Upvotes: 1

Ivan Rostovsky
Ivan Rostovsky

Reputation: 676

For prod we use a s3 and boto3 but for unit tests it is enough to use great lib dj-inmemorystorage which just loads the file to memory. So, I just override the default file storage in my test settings:

DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'

then in the test:

record.receipt_file = SimpleUploadedFile('receipt.txt', b'')

Upvotes: 0

brettm
brettm

Reputation: 199

Late answer but it took me a long time to find this nice little library;

https://pypi.org/project/django-override-storage/

Upvotes: 0

Tommaso Barbugli
Tommaso Barbugli

Reputation: 12031

if you run the test through django this should work 

if 'test' in sys.argv:
    settings.DEFAULT_FILE_STORAGE = FileSystemStorage(location=mkdtemp(), base_url="/")

of course after ;)

DEFAULT_FILE_STORAGE = FileSystemStorage(location="/var/data", base_url="/")

Upvotes: 5

jterrace
jterrace

Reputation: 67123

I just solved this by adding to my custom test runner. To see how to add a custom test runner, see Defining a test runner in the Django documentation. My code looks something like this:

import shutil
import tempfile
from django.test.simple import DjangoTestSuiteRunner
from django.conf import settings

class CustomTestRunner(DjangoTestSuiteRunner):

    def setup_test_environment(self, **kwargs):
        super(CustomTestRunner, self).setup_test_environment(**kwargs)
        self.backup = {}
        self.backup['DEFAULT_FILE_STORAGE'] = settings.DEFAULT_FILE_STORAGE
        settings.DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
        self.backup['MEDIA_ROOT'] = settings.MEDIA_ROOT
        self.temp_media_root = tempfile.mkdtemp(prefix="myapp-tests")
        settings.MEDIA_ROOT = self.temp_media_root

    def teardown_test_environment(self, **kwargs):
        super(CustomTestRunner, self).teardown_test_environment(**kwargs)
        for name, value in self.backup.iteritems():
            setattr(settings, name, value)

    def run_tests(self, test_labels, **kwargs):
        try:
            test_results = super(CustomTestRunner, self).run_tests(test_labels, **kwargs)
        finally:
            shutil.rmtree(self.temp_media_root, ignore_errors=True)

This overrides some of the custom test suite methods. setup_test_environment backs up the previous settings and stores them in a class attribute. The teardown_test_environment sets them back to what they were before. The run_tests method uses a try/finally block to make sure that the temporary directory is deleted after the tests, even if an exception happens.

Upvotes: 3

meshantz
meshantz

Reputation: 1626

Don't know if this counts as elegant, but you could use a different settings file for testing...

Something like:

# test_settings.py

from settings import *

PACKAGE_STORAGE = FileSystemStorage(location='/test/files', base_url="/")

Then run your test using the test settings, python manage.py test --settings=test_settings.

Upvotes: 10

Filip Dupanović
Filip Dupanović

Reputation: 33680

Override the underlying storage implementation for instances of FileField on your model dynamically:

def setUp(self):
     self._field = Package._meta.get_field_by_name('file')[0]
     self._default_storage = self._field.storage
     test_storage = FileSystemStorage(location=mkdtemp(),
                                      base_url="/")

     self._field.storage = test_storage

def tearDown(self):
     self._field = self._default_storage

Upvotes: 5

Related Questions