CODEWITHSUNDEEP
node.jsexpressxmlhttprequestmulter
Brandon
Brandon

Reputation: 79

multer/node file upload works perfectly for files <20kb, hangs on files 20kb-1mb, and returns 502 error for files >1mb

I'm working on a simple file upload site and have been having trouble consistently uploading files without error. The code I have now works perfectly when a file to be uploaded is less than ~20kb, but for files larger than that but smaller than ~1mb, only about ~20kb of the file is uploaded (thus corrupting the file).

For files larger than 1mb, the upload fails completely and returns a 502 error. I've been sure to increase the multer filesize/fieldsize limit and never receive a file too large error.

when a file between ~20kb-1mb is uploaded, there is no frontend or backend error, but multer seems to 'hang' on the upload and never reaches the upload callback function.

EDIT I've removed the frontend code since this appears to be a node/config error

app.js file (irrelevant code removed)

const express    = require('express');
const multer     = require('multer');
const path       = require('path');
const bodyParser = require('body-parser');
const glob       = require('glob');
const fs         = require('fs');

const app  = express();
const PORT = process.env.PORT || 3000;

const storage = multer.diskStorage({
    destination: function(req, file, callback) { callback(null, './public/file') },
    filename: function(req, file, callback) { callback(null, req.fileCode + '-' + file.originalname) },
});

const upload = multer({storage: storage, limits: {fileSize: 1024 * 1024 * 1024, fieldSize: 10000000000}}).single('selectedFile');



app.use(bodyParser.json({ limit: '5mb', extended: true })); //added from user suggestions
app.use(bodyParser.urlencoded({ limit: '5mb', extended: true }));

app.use(express.static(path.join(__dirname, 'public', 'media', 'styles.css')));
app.use(express.static(path.join(__dirname, 'public', 'media', 'scripts.js')));

app.get(["/","/index.html.var"], (req, res) => {
    res.sendFile(path.join(__dirname, 'public', 'index.html'));
});

app.post('/upload-file', function(req, res) {
    let fileCode = genFileCode();

    glob("**/public/file/" + fileCode + "*.*", function (err, files) {
        if(files[0])
            fileCode = genFileCode();

        req.fileCode = fileCode;

        console.log("check 1");

        upload(req, res, function(e) {
            console.log("check 2");

            if(e)
                console.log("error " + e);
            else
                console.log("no error");
        });

        res.send(fileCode);
    });  
});

enter image description here ^^exact same file (~370kb) is uploaded multiple times, all resulting in different sizes and none of which are the complete file (all are corrupted)

enter image description here

^^502 error results when any file larger than ~1mb is uploaded

Upvotes: 1

Views: 2147

Answers (4)

Ulrik
Ulrik

Reputation: 1

I see that this is a very old post, but I struggled with this for a while, and found out that it was not multer or anything that was the issue, the issue was that nginx has a limit set to 1mb for all requests. So if you use nginx, this will probably solve it:

You have to put this line in your nginx configuration file:

client_max_body_size 50M;

This sets the limit to 50mb for instance.

Upvotes: 0

Brandon
Brandon

Reputation: 79

I've figured out my error if anyone stumbles upon this and is in the same situation i was in: 

upload(req, res, function(e) {
        console.log("check 2");

        if(e)
            console.log("error " + e);
        else
            console.log("no error");
    });

    res.send(fileCode);

here it seems app.js is sending the response before the upload callback and ending the file upload prematurely. placing the res.send within the callback seems to have solved everything

Upvotes: 0

Saud Anjum
Saud Anjum

Reputation: 246

Your code seems fine.

Try to change these two lines

const upload = multer({storage: storage, limits: {fileSize: 5MB, 
fieldSize: 5MB}}).single('selectedFile');

I am changing the sugesstion.Keep the previous one as it as and try this one. Hope this will workout.

Upvotes: -1

Hardik Patel
Hardik Patel

Reputation: 370

It seems that your body parsing limit is less then 1 MB in backend. Change following Lines to your server configuration file:

app.use(bodyParser.json({limit: '5mb', extended: true}))

app.use(bodyParser.urlencoded({limit: '5mb', extended: true}))

You can set Limits according to your criteria.

Upvotes: 2

Related Questions