BrowserMob Proxy + Apache HttpClient: Receiving javax.net.ssl.SSLHandshakeException

Question

my goal is to use BMP to verify that Google Analytics calls are being triggered upon sending certain requests to my application (for now I'm just trying to hit yahoo.com and make a request). Preferably I'd like to do this without going thru front-end (i.e. using Selenium).

My Java code is as such:

import net.lightbody.bmp.BrowserMobProxy;
import net.lightbody.bmp.BrowserMobProxyServer;
import net.lightbody.bmp.client.ClientUtil;
import net.lightbody.bmp.core.har.Har;
import net.lightbody.bmp.core.har.HarEntry;
import net.lightbody.bmp.proxy.CaptureType;

import java.io.IOException;
import java.net.Inet4Address;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.List;

import org.apache.http.HttpHost;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

import org.junit.Test;

public class browserMob_Proxy {


@Test
public void runTest() throws ClientProtocolException, IOException, URISyntaxException, InterruptedException{

    // start the proxy
    BrowserMobProxy browserMobProxy = new BrowserMobProxyServer();
    //browserMobProxy.setTrustAllServers(true);
    Thread.sleep(10000);
    browserMobProxy.start(0);
    int port = browserMobProxy.getPort(); // get the JVM-assigned port



    CloseableHttpClient httpclient = HttpClients.createDefault();
    try {
        String hostIp = Inet4Address.getLocalHost().getHostAddress();
        HttpHost proxy = new HttpHost(hostIp, port, "http");        
        HttpHost target = new HttpHost("google.com", 443, "https");

        RequestConfig config = RequestConfig.custom().setProxy(proxy).build();
        HttpGet request = new HttpGet("/");
        request.setConfig(config);

        System.out.println("Executing request " + request.getRequestLine() + " to " + target + " via " + proxy);

        CloseableHttpResponse response = httpclient.execute(target, request);
        try {
            System.out.println("----------------------------------------");
            System.out.println(response.getStatusLine());
            EntityUtils.consume(response.getEntity());
        } finally {
            response.close();
        }
    }
    finally {
        httpclient.close();
    }

    browserMobProxy.stop();
}

}

Sending the request results in:

> javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
    at sun.security.ssl.Handshaker.process_record(Handshaker.java:914)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:275)
    at org.apache.http.impl.conn.HttpClientConnectionOperator.upgrade(HttpClientConnectionOperator.java:169)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.upgrade(PoolingHttpClientConnectionManager.java:333)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:398)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:115)
    at com.foo.www.analytics_testing.browserMob_Proxy.runTest(browserMob_Proxy.java:60)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
    at sun.security.validator.Validator.validate(Validator.java:260)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491)
    ... 41 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
    ... 47 more

How can I resolve this error?

You realize you are trying to use Http objects but you then say the site is using Https The message you are getting is from the https connection being unable to find your certs for the connection, whether you don't have them or it ignores them because you tell it to trust all. I'm not sure — Sentinel, Nov 29 '17 at 17:43
"the https connection being unable to find your certs for the connection" --I'm under the impression that browserMobProxy.setTrustAllServers(true) makes it ignore certificate checking, so this shouldn't happen. Right? However, whether I use that line or not, I still get the same error. Makes me think that that line is not doing what it's supposed to, or I'm misunderstanding what it does (more likely, I'm new to this field). — user1599401, Nov 29 '17 at 18:27
You are specifying an https connection " HttpHost target = new HttpHost("yahoo.com", 443, "https"); " https requires some sort of authentication thereby you cannot use an https connection without that level of security — Sentinel, Nov 29 '17 at 18:40
Can you give me more info on the authentication I need to provide? What authentication would I need to provide in order to hit https://www.yahoo.com? — user1599401, Nov 29 '17 at 18:52
What are you trying to accomplish exactly? this is a little overkill method if you just want to reach the site — Sentinel, Nov 29 '17 at 19:26
At this point I've resorted to using Selenium instead of HttpClient — user1599401, Nov 30 '17 at 21:01
I'm able to launch Chrome at least, but now I'm stuck here: https://stackoverflow.com/questions/47583296/browsermob-proxy-selenium-not-receiving-any-http-responses — user1599401, Dec 01 '17 at 15:05

score 0 · Answer 1 · answered Nov 29 '17 at 19:32

0

If you are just trying to reach the site, something more along the lines of this might be simpler to understand

CloseableHttpClient httpClient = HttpClients.createDefault();
URL url = new URL("http://yahoo.com");
URLConnection connection = url.openConnection();

If you want to get read from the site then then there is a little more code but I'm not sure exactly what you are asking

I'm aware this doesn't use BMP, is there a reason you are trying to use BMP?

If you can provide more detail I can try and spin up a working sample when I get back to my computer

answered Nov 29 '17 at 19:32

Sentinel

100
10

My goal is to use BMP to verify that Google Analytics calls are being fired off automatically caused by me sending certain requests to my website. I expect that it would work by me sending an HTTP request, capturing the HAR, and validating that the GA call was also placed as a result of my original request to my website. So I expect that GA call in the HAR. – user1599401 Nov 29 '17 at 19:38
It seems as though the best approach would be to use Selenium, however if you are so opposed to doing so, I can try and troubleshoot your code above Please add any code you left out so as to make it as accurately reproducible as possible – Sentinel Nov 29 '17 at 20:14
I just edited the original post to include the entirety of the code – user1599401 Nov 29 '17 at 20:25

BrowserMob Proxy + Apache HttpClient: Receiving javax.net.ssl.SSLHandshakeException

1 Answers1