CODEWITHSUNDEEP
node.jsamazon-s3multer-s3
Aatman
Aatman

Reputation: 573

How to specify upload directory in multer-S3 for AWS-S3 bucket?

I am using express + multer-s3 to upload files to AWS S3 service.

Using the following code, I was able to upload the files to S3 Bucket but directly in the bucket.

I want them to be uploaded in a folder inside the bucket.

I was not able to find the option to do so.

Here is the code

AWS.config.loadFromPath("path-to-credentials.json");
var s3 = new AWS.S3();

var cloudStorage = multerS3({
    s3: s3,
    bucket: "sample_bucket_name",
    contentType: multerS3.AUTO_CONTENT_TYPE,
    metadata: function(request, file, ab_callback) {
        ab_callback(null, {fieldname: file.fieldname});
    },
    key: function(request, file, ab_callback) {
        var newFileName = Date.now() + "-" + file.originalname;
        ab_callback(null, newFileName);
    },
});
var upload = multer({
    storage: cloudStorage
});

router.post("/upload", upload.single('myFeildName'), function(request, response) {
    var file = request.file;
    console.log(request.file);
    response.send("aatman is awesome!");
});

Upvotes: 21

Views: 17210

Answers (4)

kartik goyal
kartik goyal

Reputation: 134

Multer s3 uses a string path to add folders. Also, you can utilize req and file objects to add dynamic paths. But in the req object, you can only utilize headers, params, and query properties, not the body property as the data is not processed yet. Also, don't try to use 2 multer's in the request it will not work. Here, use the middleware and functions to preprocess using the req object and use the req object to add the customized path as the request object is a type of reference object, thus the changes in 1 middleware will reflect on its proceeding middleware.

It is the only possible solution according to me.

[EDIT 1]
Example of the above approach:
Required Path:

<s3-bucket-host-url>/user/:id/<file-name>

Middleware Code:

const preprocessMiddleware = (req, res, next) => {
let path = 'users/'; //Add static path
path = path + req.params.id;//Add id
req.processedpath = path;
next();
}

Inside Multer s3 storage function:

const storage = multerS3({
  s3: s3,
  acl: <S3ACL>,
  bucket: <S3BUCKETNAME>,
  key: (req, file, cb) => {
    let path = req.processedpath;
    const trimSplash = (str) => str.replace(/^\s*\/*\s*|\s*\/*\s*$/gm, ''); //The function is to trim all the leading and trailing slashes
    const newFileName = Date.now() + "-" + file.originalname;
    const fullPath = trimSplash (path + '/' + newFileName);
    cb(null, fullPath); //use Date.now() for unique file keys
  }
});

Router should look like this:

router.post('/uploads3', preprocessMiddleware, uploader.single('document'), someothermiddlewares);

Upvotes: 1

yubhav
yubhav

Reputation: 21

There is another nice way of doing it! After you write your bucket name, add the folder name there itself with a slash infront.

const multerS3Config = multerS3({
    s3: s3,
    bucket: process.env.AWS_BUCKET_NAME + '/<your-folder-name>', //put a slash infront
    metadata: function (req, file, cb) {
        cb(null, { fieldName: file.fieldname });
    },
    key: function (req, file, cb) {
    cb(null, Date.now() + file.originalname)
    }  
});

Upvotes: 0

iqra
iqra

Reputation: 1261

My solution to dynamic destination path. Hope this helps somebody!

const fileUpload = function upload(destinationPath) {
  return multer({
    fileFilter: (req, file, cb) => {
      const isValid = !!MIME_TYPE_MAP[file.mimetype];
      let error = isValid ? null : new Error("Invalid mime type!");
      cb(error, isValid);
    },
    storage: multerS3({
      limits: 500000,
      acl: "public-read",
      s3,
      bucket: YOUR_BUCKET_NAME,
      contentType: multerS3.AUTO_CONTENT_TYPE,
      metadata: function (req, file, cb) {
        cb(null, { fieldName: file.fieldname });
      },
      key: function (req, file, cb) {
        cb(null, destinationPath + "/" + file.originalname);
      },
    }),
  });
};



module.exports = fileUpload;

How to call:

router.patch(
  "/updateProfilePicture/:userID",
  fileUpload("user").single("profileimage"),
  usersControllers.updateProfilePicture
);

"profile image" is the key for the file passed in the body.
"user" is path to the destination folder. You can pass any path, consisting of folder and sub folders. So this puts my file in a folder called "user" inside my bucket.

Upvotes: 12

bknights
bknights

Reputation: 15447

S3 doesn't always have folders (see http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html). It will simulate folders by adding a strings separated by / to your filename.

e.g.

key: function(request, file, ab_callback) {
    var newFileName = Date.now() + "-" + file.originalname;
    var fullPath = 'firstpart/secondpart/'+ newFileName;
    ab_callback(null, fullPath);
},

Upvotes: 38

Related Questions