Using Node, Express and Multer for Upload a file to MongoDB? What should I use, Multer or GRIDfs?

Question

Hello I'm running a final project on heroku using MongoLab and Multer to upload photos seems to work fine but after a few hours or less the photos dissapear from the page but leaves the img placeholder and another field like name or date area where it was.

Is this Heroku or Mongolab issue or is it just the free account is this because of the free acounts. also it doesnt matter if there is one photo uploaded or a dozen they will still dissapear after a while.

Could you help me? Is there a hack, tricky line of code or something that I made wrong with Multer or my database? I have made a deep research in the web and Stack-Overflow I found something call GRIDfs but I don't know if that will work. Could you tell me if I'm in rigth way?

This is my code for Multer:

var express        = require("express"),
    body_parser    = require("body-parser"),
    multer         = require("multer"),
    mongoose       = require("mongoose");

    var app = express();

    //connect my local database to MongoLab database   

    var url = process.env.DATABASE || "mongodb://localhost/black_hat";

    mongoose.connect(url);

Multer configuration

var storage = multer.diskStorage({
destination: function(req, file, callback){
    callback(null, "./public/uploads");
},
filename: function(req, file, callback){
    callback(null, Date.now() + file.originalname);
}
  });

var upload = multer({storage : storage}).single("image");

My POST route

    app.post("/fotos_videos", isLoggedIn, function(req, res) {
    upload(req, res, function(err) {
        if(err){
            return res.end("Error uploading file");
        }
        // get data from form
        var newName = req.body.name,
            newCover = "/uploads/" + req.file.filename;
        // Packing into an object
        var newCollage = {
            name : newName,
            cover: newCover
        };
        //create new album
        Collage.create(newCollage, function(err, collage) {
            if(err){
                console.log(err);
            } else {
                // console.log(collage);
                res.redirect("/fotos_videos");
            }
        });
    });
});

The form where I upload the picture

    <div style="width=30%; margin: 30px auto" >
        
        <form action="/fotos_videos" method="POST" enctype="multipart/form-data">
            <div class="form-group">
                <label>Album Name</label>
                <input type="text" name="name" placeholder="Album Name" class="form-control">
            </div>
            <div class="form-group">
                <label>Choose Album Cover</label>
                <input type="file" name="image" placeholder="image url" class="form-control" required>
            </div>
            
            <div class="form-group">
                <button class="btn btn-lg btn-primary btn-block">Upload</button>
            </div>
        </form>
        
        <a href="/fotos_videos">Back</a>
        
    </div>

Finally my Model for Collage

var mongoose = require("mongoose");

// Schema 
var collageSchema = new mongoose.Schema({
    name : String,
    cover : String,
    photos: [{
        type: mongoose.Schema.Types.ObjectId,
        ref: "Photo"
    }] 
});

var Collage = mongoose.model("Collage", collageSchema);

module.exports = Collage;

Any kind of help will be appreciated.

Answer 1

if you want to save the image to the db u should not have "dest" property. also implementation of multer is wrong maybe because it is an old question and previous api was like that. here what u should u do.

configure your multer:

const multer = require("multer");
const upload = multer({
  limits: { fileSize: 5000000 },
  fileFilter(req, file, cb) {
    if (!file.originalname.match(/\.(jpg|jpeg|png)$/)) { //allowed file extensions
      return cb(new Error("please upload png,jpeg or jpg"));
    }
    cb(undefined, true);
  }
});

then create an endpoint. u already have this

app.post("/fotos_videos", isLoggedIn, function(req, res) {
   upload(req, res, function(err) { 

this iplementation wrong. upload is a middleware and if you wanna add authentication, only authenticated user should be access to this endpoint. so we need to tweak this endpoint.

Also u need to define a schema for uploads in order to save them to the db. let's say u created a model named uploads and this to it. we do not need to validate it by mongoose because multer already takes care of it. make sure u delete the dest property from multer configuration. because if u dont delete, multer will save the file to the folder so our async (req,res){} function will not be able to access to the file so we cannot save it to the db.

uploads:{type:Buffer} //include this in your uploads schema

then in this file call Uploads model.

router.post(
  "/fotos_videos/me/uploads", isLoggedIn,
  upload.single("uploads"),  //uploads is just a random name when u upload from postman, form form-data,put "uploads" in key field
  async (req, res) => {
    req.upload.uploads = req.file.buffer;//i explain down below
    await req.upload.save();
    res.send();
  },
  (err, req, res) => {
    res.status(400).send("error happned");
  }
);

We do not store our pictures in the filesystem. The reason is almost all the deployment platforms, require us to take our code and push it up to the repository on their server. So the file system gets wiped every time we deploy which means we will lose data when we deploy. So instead of storing the images on the file system, we are going to add a field to the user model to store the image of binary data in the database.

now it is time to fetch our image from the db. for this u need to have a new route

router.get("/fotos_videos/:id/uploads", async (req, res) => {
  try {
    const upload = await Uploads.findById(req.params.id);
    if (!upload || !upload.uploads) {
      throw new Error("no image found");
    }
    res.set("Content-Type", "image/png");
    res.send(upload.uploads);
  } catch (e) {
    res.status(400).send(e.message);
  }
});

Answer 2

I don't know if this will be helpful. but from my understanding multer is the stuff used to get the data from the user (like uploaded images). It doesn't do persistent storage. If you want to store the data (large files like images) you should use gridfs-stream .

You should a create a read stream to GridFs once the data comes in from the user.