How to specify upload directory in multer-S3 for AWS-S3 bucket?

Question

I am using express + multer-s3 to upload files to AWS S3 service.

Using the following code, I was able to upload the files to S3 Bucket but directly in the bucket.

I want them to be uploaded in a folder inside the bucket.

I was not able to find the option to do so.

Here is the code

AWS.config.loadFromPath("path-to-credentials.json");
var s3 = new AWS.S3();

var cloudStorage = multerS3({
    s3: s3,
    bucket: "sample_bucket_name",
    contentType: multerS3.AUTO_CONTENT_TYPE,
    metadata: function(request, file, ab_callback) {
        ab_callback(null, {fieldname: file.fieldname});
    },
    key: function(request, file, ab_callback) {
        var newFileName = Date.now() + "-" + file.originalname;
        ab_callback(null, newFileName);
    },
});
var upload = multer({
    storage: cloudStorage
});

router.post("/upload", upload.single('myFeildName'), function(request, response) {
    var file = request.file;
    console.log(request.file);
    response.send("aatman is awesome!");
});

score 34 · Accepted Answer · answered May 17 '17 at 15:55

34

S3 doesn't always have folders (see http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html). It will simulate folders by adding a strings separated by / to your filename.

e.g.

key: function(request, file, ab_callback) {
    var newFileName = Date.now() + "-" + file.originalname;
    var fullPath = 'firstpart/secondpart/'+ newFileName;
    ab_callback(null, fullPath);
},

answered May 17 '17 at 15:55

bknights

11,218
2
13
25

Thank you so much!! Worked like charm... :) – Aatman May 17 '17 at 16:29
1

Is there a way to make the 'firstpart/secondpart' prefix dynamic? I have used req.params.destination but no luck – Herve Tribouilloy May 08 '19 at 15:56
@HerveTribouilloy absolutely. The firstpart/secondpart are just strings. You can generate them however you want. If you are not seeing what you expect first make sure you actually have a value in your params. – bknights May 08 '19 at 16:54
1

yes, my issue is I can't seem to send a param to the request parameter that multer/multerS3 uses. I have also read that multer does not handle params well, so I was wondering if you have more experience on this, thanks for your feedback – Herve Tribouilloy May 08 '19 at 18:47
@HerveTribouilloy the request passed to multer is not the same as what is processed by the router. You have the full path though and can parse it to get the desired param. – bknights May 08 '19 at 19:14
thanks, the problem was due to the POST request data were not being parsed. I have not found a way to parse them but instead, I found a workaround that consist in using the query parameters (the file to upload in in POST data and the other parameters are in the query). thanks for your support – Herve Tribouilloy May 09 '19 at 10:35
@bknights Hi, hope everything is good. i have a very similar question if you you please have a look at.https://stackoverflow.com/questions/65479245/nodejs-multer-aws-s3 – kd12345 Dec 29 '20 at 13:10

score 8 · Answer 2 · answered Apr 04 '20 at 14:45

My solution to dynamic destination path. Hope this helps somebody!

const fileUpload = function upload(destinationPath) {
  return multer({
    fileFilter: (req, file, cb) => {
      const isValid = !!MIME_TYPE_MAP[file.mimetype];
      let error = isValid ? null : new Error("Invalid mime type!");
      cb(error, isValid);
    },
    storage: multerS3({
      limits: 500000,
      acl: "public-read",
      s3,
      bucket: YOUR_BUCKET_NAME,
      contentType: multerS3.AUTO_CONTENT_TYPE,
      metadata: function (req, file, cb) {
        cb(null, { fieldName: file.fieldname });
      },
      key: function (req, file, cb) {
        cb(null, destinationPath + "/" + file.originalname);
      },
    }),
  });
};



module.exports = fileUpload;

How to call:

router.patch(
  "/updateProfilePicture/:userID",
  fileUpload("user").single("profileimage"),
  usersControllers.updateProfilePicture
);

"profile image" is the key for the file passed in the body.
"user" is path to the destination folder. You can pass any path, consisting of folder and sub folders. So this puts my file in a folder called "user" inside my bucket.

score 0 · Answer 3 · answered Dec 27 '20 at 02:49

There is another nice way of doing it! After you write your bucket name, add the folder name there itself with a slash infront.

const multerS3Config = multerS3({
    s3: s3,
    bucket: process.env.AWS_BUCKET_NAME + '/<your-folder-name>', //put a slash infront
    metadata: function (req, file, cb) {
        cb(null, { fieldName: file.fieldname });
    },
    key: function (req, file, cb) {
    cb(null, Date.now() + file.originalname)
    }  
});

How to specify upload directory in multer-S3 for AWS-S3 bucket?

3 Answers3