CODEWITHSUNDEEP
pythondjangodjango-migrations
geckon
geckon

Reputation: 8774

Django migrations and FileSystemStorage depending on settings

In my Django app I use a FileSystemStorage for generated files. I initialize it like this:

import os
from urlparse import urljoin

from django.conf import settings
from django.core.files.storage import FileSystemStorage

gen_files_storage = FileSystemStorage(location=os.path.join(settings.MEDIA_ROOT, 'generated/'), base_url=urljoin(settings.MEDIA_URL, 'generated/'))

When I want to create a new file I use:

from django.core.files.base import ContentFile
from django.db import models

def next_number():
    # num = ...
    return num

gen_file = models.FileField(storage=gen_files_storage)
gen_file.save('file%s.txt' % next_number(), ContentFile(''))

That works fine. The only problem is that the FileSystemStorage's path is "hardcoded" in the Django migration. Because I use different settings for development (which changes) and production, often the manage.py makemigrations command generates a migration only because the path changed, although everything stays the same in the database.

I know there is a solution using a subclass of FileSystemStorage (see my answer below), but is there a better solution?

Upvotes: 23

Views: 4735

Answers (4)

swinters
swinters

Reputation: 153

Upgrading to Django 3.1+ fixes this: https://docs.djangoproject.com/en/3.2/releases/3.1/#file-storage

Just pass a callable into the storage argument.

from django.db import models
from django.conf import settings
from django.core.files.storage import get_storage_class


def _get_storage():
    storage_class = get_storage_class(settings.MY_STORAGE_CLASS)  # ie. 'django.core.files.storage.FileSystemStorage'
    return storage_class()

class MyModel(models.Model):
    myfile = models.FileField(max_length=255, blank=True, storage=_get_storage)

Upvotes: 9

JSTL
JSTL

Reputation: 868

My problem was related, but slightly different. The storage class used by the field can change based on settings: the default locally, remote storage in production. I implemented a subclass of FileField that ignores the storage kwarg when deconstructing the field for migration generation.

from django.db.models import FileField

class VariableStorageFileField(FileField):
    """
    Disregard the storage kwarg when creating migrations for this field
    """

    def deconstruct(self):
        name, path, args, kwargs = super(VariableStorageFileField, self).deconstruct()
        kwargs.pop('storage', None)
        return name, path, args, kwargs

It can be used like this:

class MyModel(models.Model):
    storage = get_storage_class(getattr(settings, 'LARGE_FILE_STORAGE', None))()

    file = VariableStorageFileField(blank=True, null=True, storage=storage)

Upvotes: 2

Chris Conlan
Chris Conlan

Reputation: 2962

The solution is to never run makemigrations on production. Run migrate all you want on production servers, but ignore warnings about running makemigrations if they pertain to this issue.

Think about it: makemigrations generates Python code, so running it on production would be the same as developing on that server. Depending on your server setup, your production site will likely serve those files correctly regardless of the makemigrations warning.

Upvotes: 2

geckon
geckon

Reputation: 8774

There is a solution involving a custom @deconstructible subclass of FileSystemStorage:

import os
from urlparse import urljoin

from django.conf import settings
from django.core.files.storage import FileSystemStorage
from django.utils.deconstruct import deconstructible

@deconstructible
class MyFileSystemStorage(FileSystemStorage):
    def __init__(self, subdir):
        self.subdir = subdir
        super(MyFileSystemStorage, self).__init__(location=os.path.join(settings.MEDIA_ROOT, self.subdir), base_url=urljoin(settings.MEDIA_URL, self.subdir))

    def __eq__(self, other):
        return self.subdir == other.subdir

Then I can initialize the storage like this:

import os
from urlparse import urljoin

from django.conf import settings
from django.core.files.storage import FileSystemStorage

gen_files_storage = MyFileSystemStorage('generated/')

This way Django migrations won't notice changes in my settings. Is there a better way though?

Upvotes: 11

Related Questions