How do I get browsermob-proxy to record ALL response bodies

Question

I'm having trouble with browsermob-proxy and its har exporting feature. Some response bodies are not logged (the entire "text" field is missing)

My setup (using browsermob-proxy 2.1.4)

curl -X POST http://localhost:8080/proxy?port=9091

curl -X PUT "http://localhost:8080/proxy/9091/har?captureHeaders=true&captureCookies=true&captureContent=true"

Some responses are fine:

"response": { "status": 201, "statusText": "Created", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{ "name": "Cache-Control", "value": "max-age=0, no-cache, no-store" }, { "name": "Content-Type", "value": "application/json" }, { "name": "Date", "value": "Thu, 15 Feb 2018 13:07:39 GMT" }, { "name": "Location", "value": ... }, { "name": "Pragma", "value": "no-cache" }, { "name": "Render-Time", "value": "8" }, { "name": "Server", "value": "openresty" }, { "name": "Strict-Transport-Security", "value": "max-age=31536000; includeSubDomains" }, { "name": "transfer-encoding", "value": "chunked" }, { "name": "Connection", "value": "keep-alive" }], "content": { "size": 8607, "mimeType": "application/json", "text": "{ <actual json body> }", "comment": "" }, ...

But some are not (maybe it is the special content/mime type? or maybe it is the gzip content encoding?)

"response": { "status": 200, "statusText": "OK", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{ "name": "Cache-Control", "value": "max-age=0, no-cache, no-store" }, { "name": "Content-Encoding", "value": "gzip" }, { "name": "Content-Type", "value": "application/some.custom.type-v1+json" }, { "name": "Date", "value": "Thu, 15 Feb 2018 13:07:39 GMT" }, { "name": "Pragma", "value": "no-cache" }, { "name": "Render-Time", "value": "92" }, { "name": "Server", "value": "openresty" }, { "name": "Strict-Transport-Security", "value": "max-age=31536000; includeSubDomains" }, { "name": "Vary", "value": "Accept-Encoding" }, { "name": "Content-Length", "value": "1978" }, { "name": "Connection", "value": "keep-alive" }], "content": { "size": 7429, "mimeType": "application/some.custom.type-v1+json", "comment": "" < there's nothing else here!! > }, "redirectURL": "", "headersSize": 444, "bodySize": 1978, "comment": "" }, ...

Oh yea, both requests are over https, using MITM.

score 1 · Accepted Answer · answered Feb 15 '18 at 16:27

1

I think I found the answer to my own question

In BrowserMobHttpClient.java I found the following:

private boolean hasTextualContent(String contentType) {
    return contentType != null && contentType.startsWith("text/") ||
            contentType.startsWith("application/x-javascript") ||
            contentType.startsWith("application/javascript")  ||
            contentType.startsWith("application/json")  ||
            contentType.startsWith("application/xml")  ||
            contentType.startsWith("application/xhtml+xml");
}

Looks like I'll have go make a custom build of browsermob-proxy to make it work.

answered Feb 15 '18 at 16:27

Cyberwiz

7,561
3
14
29

CyberWiz, What have you changed ? I am also facing the issue that it does not record Content-Encoding header. Please let me know the change in the BrowserMobHttpClient.java. Thanks in advance – UmeshPathak Jul 15 '20 at 04:22
I just added a line for my particular content type, something like || contentType.startsWith("application/some.custom.type-v1+json") – Cyberwiz Jul 15 '20 at 13:32

How do I get browsermob-proxy to record ALL response bodies

1 Answers1