CODEWITHSUNDEEP
pythondjangoamazon-web-servicesnginxamazon-s3
Aleksei Petrenko
Aleksei Petrenko

Reputation: 7188

Local filesystem as a remote storage in Django

I use Amazon S3 as a part of my webservice. The workflow is the following:

To work with files I use QueuedStorage backend. I initiate my FileFields like this:

    user_uploaded_file = models.FileField(..., storage=queued_s3storage, ...)
    result_file = models.FileField(..., storage=queued_s3storage, ...)

Where queued_s3storage is an object of class derived from ...backends.QueuedStorage and remote field is set to '...backends.s3boto.S3BotoStorage'.

Now I'm planning to deploy the whole system on one machine to run everything locally, I want to replace this '...backends.s3boto.S3BotoStorage' with something based on my local filesystem.

The first workaround was to use FakeS3 which can "emulate" S3 locally. Works, but this is not ideal, just extra unnecessary overhead.

I have Nginx server running and serving static files from particular directories. How do I create my "remote storage" class that actually stores files locally, but provides download links which lead to files served by Nginx? (something like http://myip:80/filedir/file1). Is there a standard library class for that in django?

Upvotes: 5

Views: 7392

Answers (2)

Kristian Glass
Kristian Glass

Reputation: 38530

I'm planning to deploy the whole system on one machine to run everything locally

Stop using QueuedStorage then, because "[QueuedStorage] enables having a local and a remote storage backend" and you've just said you don't want a remote.

Just use FileSystemStorage and configure nginx to serve the location / settings.MEDIA_ROOT

Upvotes: 1

pztrick
pztrick

Reputation: 3831

The default storage backend for media files is local storage.

Your settings.py defines these two environment variables:

  • MEDIA_ROOT (link to docs) -- this is the absolute path to the local file storage folder
  • MEDIA_URL (link to docs) -- this is the webserver HTTP path (e.g. '/media/' or '//%s/media' % HOSTNAME

These are used by the default storage backend to save media files. From Django's default/global settings.py:

# Default file storage mechanism that holds media.
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'

This configured default storage is used in FileFields for which no storage kwarg is provided. It can also be accessed like so: rom django.core.files.storage import default_storage.

So if you want to vary the storage for local development and production use, you can do something like this:

# file_storages.py
from django.conf import settings
from django.core.files.storage import default_storage
from whatever.backends.s3boto import S3BotoStorage

app_storage = None
if settings.DEBUG == True:
    app_storage = default_storage
else:
    app_storage = S3BotoStorage()

And in your models:

# models.py
from file_storages import app_storage

# ...
    result_file = models.FileField(..., storage=app_storage, ...)

Lastly, you want nginx to serve the files directly from your MEDIA_URL. Just make sure that the nginx URL matches the path in MEDIA_URL.

Upvotes: 14

Related Questions