How to use multer-s3 in aws lambda handler as middleware

Question

I am trying to send a multipart/form-data form to an AWS-Lambda method. I need to be able to send files to S3, and using incoming string parameters, I need to record metadata to RDS.

Now, I can do that using express and multer-s3 as follows;

var express = require('express');
var AWS = require('aws-sdk');
var multer = require('multer')
var multerS3 = require('multer-s3')

var s3 = new AWS.S3();
const app = express();

var upload = multer({
  storage: multerS3({
    s3: s3,
    bucket: 'my-bucket-name',
    metadata: function (req, file, cb) {
      cb(null, Object.assign({}, req.body));
    },
    key: function (req, file, cb) {
      cb(null, Date.now().toString() + '.fileExtension')
    }
  })
})

app.post('/data', upload.array('file'), function(req, res, next) {
  // here using req.files, i can save metadata to RDS
})

My question is, is it possible to use multer-s3 in an AWS Lambda method? If the answer is no, or it's not recommended, could you please point me in the right direction?

Thanks..

Answer 1

I know it's been a while since the question was posted, but for the sake of people that might end up here in the future:

Short answer: It's not recommended. Why? There's some weird handling of the files sent as part of the Form Data, not sure if it's either by API Gateway or S3. I spent a whole day trying to upload images from a SPA Angular application using a similar approach as the one you mention, but I just couldn't make it work: I was able to access the files in the request, previously parsed by Multer, and effectively put each of those to an S3 Bucket, but images got corrupted. Not sure if it would work for other types of files, but this approach required more work and it felt a bit hacky. The best and easiest way of uploading files to an S3 bucket from outside of your AWS account (i.e., not using any AWS Service or EC2 instance) is, IMO, using Presigned URLs. You can check this article that might point you in the right direction.

That said, you can configure API Gateway to allow your Lambda to receive binary files. If you're using Serverless, the following is a plugin that makes things easier for that matter: https://github.com/maciejtreder/serverless-apigw-binary