Node.js - get raw request body using Express

Question

When I use Express, and my code is:

app.use(express.bodyParser());

How would I get the raw request body?

Answer 1

Edit 2: Release 1.15.2 of the body parser module introduces raw mode, which returns the body as a Buffer. By default, it also automatically handles deflate and gzip decompression. Example usage:

var bodyParser = require('body-parser');
app.use(bodyParser.raw(options));

app.get(path, function(req, res) {
  // req.body is a Buffer object
});

By default, the options object has the following default options:

var options = {
  inflate: true,
  limit: '100kb',
  type: 'application/octet-stream'
};

If you want your raw parser to parse other MIME types other than application/octet-stream, you will need to change it here. It will also support wildcard matching such as */* or */application.

---

Note: The following answer is for versions before Express 4, where middleware was still bundled with the framework. The modern equivalent is the body-parser module, which must be installed separately.

The rawBody property in Express was once available, but removed since version 1.5.1. To get the raw request body, you have to put in some middleware before using the bodyParser. You can also read a GitHub discussion about it here.

app.use(function(req, res, next) {
  req.rawBody = '';
  req.setEncoding('utf8');

  req.on('data', function(chunk) { 
    req.rawBody += chunk;
  });

  req.on('end', function() {
    next();
  });
});
app.use(express.bodyParser());

That middleware will read from the actual data stream, and store it in the rawBody property of the request. You can then access the raw body like this:

app.post('/', function(req, res) {
  // do something with req.rawBody
  // use req.body for the parsed body
});

Edit: It seems that this method and bodyParser refuse to coexist, because one will consume the request stream before the other, leading to whichever one is second to never fire end, thus never calling next(), and hanging your application.

The simplest solution would most likely be to modify the source of bodyParser, which you would find on line 57 of Connect's JSON parser. This is what the modified version would look like.

var buf = '';
req.setEncoding('utf8');
req.on('data', function(chunk){ buf += chunk });
req.on('end', function() {
  req.rawBody = buf;
  var first = buf.trim()[0];
  ...
});

You would find the file at this location:

/node_modules/express/node_modules/connect/lib/middleware/json.js.

Answer 2

I got a solution that plays nice with bodyParser, using the verify callback in bodyParser. In this code, I am using it to get a sha1 of the content and also getting the raw body.

app.use(bodyParser.json({
    verify: function(req, res, buf, encoding) {

        // sha1 content
        var hash = crypto.createHash('sha1');
        hash.update(buf);
        req.hasha = hash.digest('hex');
        console.log("hash", req.hasha);

        // get rawBody        
        req.rawBody = buf.toString();
        console.log("rawBody", req.rawBody);

    }
}));

I am new in Node.js and express.js (started yesterday, literally!) so I'd like to hear comments on this solution.

Answer 3

This solution worked for me:

var rawBodySaver = function (req, res, buf, encoding) {
  if (buf && buf.length) {
    req.rawBody = buf.toString(encoding || 'utf8');
  }
}

app.use(bodyParser.json({ verify: rawBodySaver }));
app.use(bodyParser.urlencoded({ verify: rawBodySaver, extended: true }));
app.use(bodyParser.raw({ verify: rawBodySaver, type: '*/*' }));

When I use solution with req.on('data', function(chunk) { }); it not working on chunked request body.

Answer 4

BE CAREFUL with those other answers as they will not play properly with bodyParser if you're looking to also support json, urlencoded, etc. To get it to work with bodyParser you should condition your handler to only register on the Content-Type header(s) you care about, just like bodyParser itself does.

To get the raw body content of a request with Content-Type: "text/plain" into req.rawBody you can do:

app.use(function(req, res, next) {
  var contentType = req.headers['content-type'] || ''
    , mime = contentType.split(';')[0];

  if (mime != 'text/plain') {
    return next();
  }

  var data = '';
  req.setEncoding('utf8');
  req.on('data', function(chunk) {
    data += chunk;
  });
  req.on('end', function() {
    req.rawBody = data;
    next();
  });
});

Answer 5

This is a variation on hexacyanide's answer above. This middleware also handles the 'data' event but does not wait for the data to be consumed before calling 'next'. This way both this middleware and bodyParser may coexist, consuming the stream in parallel.

app.use(function(req, res, next) {
  req.rawBody = '';
  req.setEncoding('utf8');

  req.on('data', function(chunk) { 
    req.rawBody += chunk;
  });

  next();
});
app.use(express.bodyParser());

Answer 6

// Change the way body-parser is used
const bodyParser = require('body-parser');

var rawBodySaver = function (req, res, buf, encoding) {
  if (buf && buf.length) {
    req.rawBody = buf.toString(encoding || 'utf8');
  }
}
app.use(bodyParser.json({ verify: rawBodySaver, extended: true }));




// Now we can access raw-body any where in out application as follows
request.rawBody;

Answer 7

Use body-parser Parse the body with what it will be:

app.use(bodyParser.text());

app.use(bodyParser.urlencoded());

app.use(bodyParser.raw());

app.use(bodyParser.json());

ie. If you are supposed to get raw text file, run .text().

Thats what body-parser currently supports