CODEWITHSUNDEEP
pythondjangonginxgunicornnginx-config
Atom Store
Atom Store

Reputation: 1006

How to fix 502 Bad Gateway Error in production(Nginx)?

When I tried to upload a big csv file of size about 600MB in my project which is hosted in the digital ocean, it tries to upload but shows 502 Bad Gateway Error (Nginx).

The application is a data conversion application.

This works fine while working locally.

sudo tail -30 /var/log/nginx/error.log

shows

[error] 132235#132235: *239 upstream prematurely closed connection while reading response header from upstream, client: client's ip , server: ip, request: "POST /submit/ HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/submit/", host: "ip", referrer: "http://ip/"

sudo nano /etc/nginx/sites-available/myproject

shows

server {
    listen 80;
    server_name ip;
    client_max_body_size 999M;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
       alias  /root/static/;
    }
    location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn.sock;
    }
}

nginx.conf

user root;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
        # multi_accept on;
}

http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

I have also the javascript loader running while the conversion process takes place. How can I fix this?

Upvotes: 5

Views: 19397

Answers (3)

zawhtut
zawhtut

Reputation: 8541

502 error can be of anything. Check your Nginx error log as following

tail /var/log/nginx/error.log -f

In my case it is because the header is too big. So one had to increase the buffer size in /etc/nginx/sites-enabled/default as Chen.A had described.

Upvotes: 1

Chen A.
Chen A.

Reputation: 11280

This error can indicate multiple problems. The fact it works for you locally strengthen the probability the issue relies on the nginx side.

You can try to solve it by increasing the timeout thresholds (as suggested here), and the buffers size. Add this to your server's nginx.conf:

proxy_read_timeout 300s;
proxy_connect_timeout 300s;

proxy_buffer_size          128k;
proxy_buffers              4 256k;
proxy_busy_buffers_size    256k;

Upvotes: 1

Dunski
Dunski

Reputation: 672

If you are using django 3.1 or higher you can make your file processing asynchronous this way and return a response to the user while the file conversion takes place. Your view should look something like this...

    import asyncio
    from django.http import JsonResponse
    from asgiref.sync import sync_to_async
    
    
    @sync_to_async
    def crunching_stuff(my_file):
        # do your conversion here
            
    async def upload(request):
        json_payload = {
            "message": "Your file is being converted"
        }
        my_file = request.POST.get('file')
        
        asyncio.create_task(crunching_stuff(my_file))
        return JsonResponse(json_payload)

On the front end if you use Dropzone.js your user can see the file upload progress and will get a response quicker. this is a better user experience. https://www.dropzonejs.com/

Upvotes: 2

Related Questions