Browsermob HTTPS requests with Auth Headers

Question

I'm trying to use Browsermob proxy to capture information about requests. Have it working for HTTP requests and HTTPS requests that don't have an authorization header.

    if(product == 'Product'):
        headers = {
            'Authorization': 'Bearer %s' % accessToken()
        }
    http_proxy  = "http://localhost:" + str(self.proxy.port)
    https_proxy = "https://localhost:" + str(self.proxy.port)
    ftp_proxy   = "ftp://localhost:" + str(self.proxy.port)
    proxyDict = { 
                  "http"  : http_proxy, 
                  "ftp"   : ftp_proxy,
                  "https" : https_proxy
                }
    fullurl = baseurl
    fullurl += '/'
    fullurl += baseuri
    logger.console("fullurl: '%s'" % fullurl)
    response =   requests.request(method,fullurl,proxies=proxyDict,verify=False,data=payload,headers=headers)

The request goes through if I remove 'proxies=proxyDict'. The request seems to time out if I go through the proxy though and I get the bellow exception:

Exception: Error creating SSLEngine for connection to client to impersonate upstream host: null
    at net.lightbody.bmp.mitm.manager.ImpersonatingMitmManager.clientSslEngineFor(ImpersonatingMitmManager.java:227) ~[browsermob-dist-2.1.0.jar:?]
    at org.littleshoot.proxy.impl.ProxyToServerConnection$3.execute(ProxyToServerConnection.java:739) ~[browsermob-dist-2.1.0.jar:?]
    at org.littleshoot.proxy.impl.ConnectionFlow.doProcessCurrentStep(ConnectionFlow.java:140) ~[browsermob-dist-2.1.0.jar:?]
    at org.littleshoot.proxy.impl.ConnectionFlow.processCurrentStep(ConnectionFlow.java:128) ~[browsermob-dist-2.1.0.jar:?]
    at org.littleshoot.proxy.impl.ConnectionFlow.advance(ConnectionFlow.java:90) ~[browsermob-dist-2.1.0.jar:?]
    at org.littleshoot.proxy.impl.ConnectionFlowStep.onSuccess(ConnectionFlowStep.java:83) ~[browsermob-dist-2.1.0.jar:?]
    at org.littleshoot.proxy.impl.ConnectionFlow$2.operationComplete(ConnectionFlow.java:149) ~[browsermob-dist-2.1.0.jar:?]
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:683) [browsermob-dist-2.1.0.jar:?]
    at io.netty.util.concurrent.DefaultPromise.notifyLateListener(DefaultPromise.java:624) [browsermob-dist-2.1.0.jar:?]
    at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:139) [browsermob-dist-2.1.0.jar:?]
    at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:93) [browsermob-dist-2.1.0.jar:?]
    at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:28) [browsermob-dist-2.1.0.jar:?]
    at org.littleshoot.proxy.impl.ConnectionFlow.doProcessCurrentStep(ConnectionFlow.java:140) [browsermob-dist-2.1.0.jar:?]
    at org.littleshoot.proxy.impl.ConnectionFlow.access$000(ConnectionFlow.java:14) [browsermob-dist-2.1.0.jar:?]
    at org.littleshoot.proxy.impl.ConnectionFlow$1.run(ConnectionFlow.java:124) [browsermob-dist-2.1.0.jar:?]
    at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) [browsermob-dist-2.1.0.jar:?]
    at io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:73) [browsermob-dist-2.1.0.jar:?]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358) [browsermob-dist-2.1.0.jar:?]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:374) [browsermob-dist-2.1.0.jar:?]
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) [browsermob-dist-2.1.0.jar:?]
    at java.lang.Thread.run(Unknown Source) [?:1.8.0_91]

Caused by: java.lang.NullPointerException
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:212) ~[browsermob-dist-2.1.0.jar:?]
    at com.google.common.cache.LocalCache.get(LocalCache.java:3952) ~[browsermob-dist-2.1.0.jar:?]
    at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4790) ~[browsermob-dist-2.1.0.jar:?]
    at net.lightbody.bmp.mitm.manager.ImpersonatingMitmManager.getHostnameImpersonatingSslContext(ImpersonatingMitmManager.java:242) ~[browsermob-dist-2.1.0.jar:?]
    at net.lightbody.bmp.mitm.manager.ImpersonatingMitmManager.clientSslEngineFor(ImpersonatingMitmManager.java:223) ~[browsermob-dist-2.1.0.jar:?]
    ... 20 more

Does browsermob handle authorization like this or am I just doing it wrong? Is there another proxy solution that might work better for this? I'm looking to use the har results.

score 0 · Answer 1 · answered Sep 16 '16 at 20:31

From the error log, I'm guessing it's one of two problems:

a bug in browsermob. Try upgrading to a newer version
a mismatch between the domain browsermob thinks it's running against, and the domain you're connecting to. try switching to your explicit ip as the hostname - ie: "127.0.0.1" instead of "localhost"

This doesn't appear to be related to authorization headers.

Browsermob HTTPS requests with Auth Headers

1 Answers1