CODEWITHSUNDEEP
djangodockernginxgunicorn
horse
horse

Reputation: 501

Django Download/Upload Files In Production

I have a Django project that is currently hosted. I am serving static files but do not know how to handle user file uploads / downloads in the MEDIA folder. I've read a lot about Docker, Nginx, and Gunicorn but have no idea which I need nor how to set them up. I've followed no less than 20 tutorials and watched no less than 15 YouTube videos but am still confused (I've visited every link for the first 2 pages of all my Google searches).

My question is, which of these do I need to allow users to upload/download files from a site? On top of that, I have tried getting all three working but can't figure them out, surely I'm not the only one that has so much difficulty with this, is there a good resource/tutorial which would guide me through the process (I've spent well over 40 hours reading about this stuff and trying to get it to work, I've gotten to the point where so long as it works, I don't care for understanding how it all fits together)?

Thank you.

edit - this is a stripped down version of what was requested. I haven't included the html as it's my first time doing this and I've used Ajax and things and it's a complete mess and I'm sure will just confuse you.

settings.py

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static_files')

MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'static_media')

STAT=os.path.join(BASE_DIR, 'static')

STATICFILES_DIRS = [ STAT,
                    os.path.join('static/'),
                    os.path.join('templates/static/'), # for overridden templates css
                    ]

view.py

class UploadView(View):
    def get(self, request):
        files_list = UploadedFile.objects.all()
        return render(self.request, 'users/modules.html', {'files': files_list})

    def post(self, request, *args, **kwargs):
        data = {}
        
        form = UploadedFileForm(self.request.POST, self.request.FILES)
        form.instance.data_id = self.request.POST['data_id']

        if form.is_valid():
            uploaded_file = form.save()

Thank you.

Upvotes: 2

Views: 3091

Answers (1)

Dmitry Belaventsev
Dmitry Belaventsev

Reputation: 6657

So you want to know how to make your (working in development environment) project production-ready. Let's start with what components are required

  1. Web Server (Nginx)
  2. Application Server (uWSGI)
  3. Application (Django)

Web Server serves the users' requests. It knows how to generate the correct output for a request - read a file from a filesystem, pass a request further to application server and so on. Nginx is the good choice.

Application Server is the middleman between Web Server and Application. It can spawn application instances (processes), balance the load between those instances, restart dead instances and many other things. uWSGI is good choice here.

Application - in your case it's Django project you have working in your development environment. You have everything ready here, but most likely you should adjust settings a bit. Django will communicate with Application Server through WSGI protocol.

At this point you should also understand how a web browser will load, render and display your site. All starts from a user who wants to open a page on your site, for example http://example.com/uploads. A browser will send HTTP GET request to your server and Web Server program (Nginx) will catch this request and decide what to do next.

Since that particular request isn't about some static file (static HTML file or static JPEG image and so on) - Nginx will decide to pass a request to Application Server. uWSGI will get the request and pass it forward to Django.

Django will use all the urls.py files to find the right view to generate the response for http://example.com/upload page. What your view will do?

    def get(self, request):
        files_list = UploadedFile.objects.all()
        return render(self.request, 'users/modules.html', {'files': files_list})

it will return the HTML page (rendered template). So that HTML document will be returned back to Application Server, then back to Web Server and finally to a user's web browser.

Then a browser will start parsing of that HTML document and most likely it will find some additional resources to load - css, javascript, images, fonts, ... For each resource - it will make additional GET request to Web Server. And this time Web Server will not push requests forward to Application Server. It will just read those files from file system and return back to a browser.

Those resources are not dynamic, they are static. So you basically store them under static namespace. For example:

Those files are the part of your application. Your application is shipped with those files. But also there're some files which could be uploaded to your system. You could save those files in any place - filesystem, database, ... But you must have URLs for them. Usually it's media namespace:

In both cases - those calls will be just forwarded to Web Server and then to filesystem.

Why do you see everything working on your development environment? It's likely because you run the application with python manage.py runserver. In that case - Django is your Web Server as well (and there will be no Application Server middleman). So it will manage it's own instances, it will get user's requests, it will return static files, it will return "media" files, it will return dynamically generated pages and so on.

Each component described above needs it's own configuration file. Let me show you some examples you can use for your project.

Web Server (Nginx)

sites-enabled/default.conf

upstream uwsgi {
  server uwsgi:8080;
}


server {
  listen 80 default_server;
  listen [::]:80 default_server;

  error_log /var/log/nginx/error.log;

  charset utf-8;

  location /media  {
      alias /home/web/media;
      expires 7d;
  }

  location /static {
      alias /home/web/static;
      expires 7d;
  }

  location / {
      uwsgi_pass uwsgi;
      include uwsgi_params;
  }
}

Notes:

Application Server (uWSGI)

uwsgi.conf

[uwsgi]

req-logger = file:/var/log/uwsgi-requests.log
logger = file:/var/log/uwsgi-errors.log

workers = %k
# enable-threads = true
# threads = 4

chdir = /home/web/app
module = core.wsgi
master = true
pidfile=/tmp/app.pid
socket = 0.0.0.0:8080
env = DJANGO_SETTINGS_MODULE=core.settings
memory-report = true
harakiri = 60
listen = 10240

Notes:

  • chdir = /home/web/app - it's the path to your application
  • module = core.wsgi - your application should have directory with main (core) application called core (you should see wsgi.py there)
  • pidfile=/tmp/app.pid - just a place for pid file
  • socket = 0.0.0.0:8080 - it will listen port 8080
  • env = DJANGO_SETTINGS_MODULE=core.settings - again, you need main app to be called core (it should be inside core directory) and you should have settings.py inside it)

Docker / Docker Compose

You might need Docker and Docker Compose to orchestrate all that software. But it's possible to try run everything without it as well.

docker-compose.yml

version: "2.4"

services:

  uwsgi:
    build:
      context: ./docker
      dockerfile: Dockerfile
    hostname: uwsgi
    sysctls:
      net.core.somaxconn: 10240
    environment:
      - DJANGO_SETTINGS_MODULE=${DJANGO_SETTINGS_MODULE}
      - C_FORCE_ROOT=${C_FORCE_ROOT}
    volumes:
      - ./config/uwsgi/uwsgi.conf:/uwsgi.conf
      - ../app:/home/web/app
      - ./static:/home/web/static:rw
      - ./media:/home/web/media:rw
      - ./logs:/var/log/
    restart: always
    networks:
      myapp_backend:
        aliases:
          - uwsgi

  web:
    image: nginx
    hostname: nginx
    sysctls:
      net.core.somaxconn: 10240
    depends_on:
      - uwsgi
    volumes:
      # - ./config/nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./config/nginx/sites-enabled:/etc/nginx/conf.d
      - ./media:/home/web/media:ro
      - ./static:/home/web/static:ro
      - ./logs:/var/log/nginx
    ports:
      - "80:80"
      # - "443:443"
    restart: always
    networks:
      - myapp_backend

networks:
  myapp_backend:

Dockerfile

FROM python:3.9.0

RUN export DEBIAN_FRONTEND=noninteractive
ENV DEBIAN_FRONTEND noninteractive
RUN dpkg-divert --local --rename --add /sbin/initctl

RUN apt-get install -y --fix-missing && apt-get update -y

RUN apt-get install -y python3-pip \
    python3-setuptools

COPY requirements.txt /requirements.txt
RUN pip install -r /requirements.txt
RUN pip install uwsgi

WORKDIR /home/web/app

EXPOSE 8080

CMD ["uwsgi", "--ini", "/uwsgi.conf"]

Directory Structure

You can use directory structure from this repository: https://github.com/ansysy24/GameEconomy/tree/master/deployment

Also I highly recommend this article https://andrey-borzenko.medium.com/simple-nginx-uwsgi-daphne-reactive-application-part-1-the-big-picture-20d7b9ee5b96

And don't forget to add .env file like this one https://github.com/ansysy24/GameEconomy/blob/master/deployment/.env_template (but you should rename ofc)

Upvotes: 6

Related Questions