Standalone vs embedded Browsermob proxy for cloud-based tests (Sauce Labs/Browserstack)

Question

I made a working test locally with embedded Browsermob proxy server. Nothing new, but still here is the sample code.

    _server = new BrowserMobProxyServer()
    _server.start();
    Proxy proxy = ClientUtil.createSeleniumProxy(_server);
    ChromeOptions options = new ChromeOptions();
    options.setCapability("proxy", proxy);
    _driver = new ChromeDriver(options);

Now we're looking into options to integrate such tests into our CI pipeline and to execute these tests in the cloud (Browserstack/Sauce Labs). I'm trying to figure out what the setup will look like in this case. Right now my understanding is that the code (which sets up the proxy and actually contains the tests) will run on our server. This means that the embedded proxy will also run on our server which is not necessarily accessible from the outside. So the questions are:

1. Will I have to switch to a standalone browsermob proxy and make it accessible?
2. If yes, then is there any actual code sample of using the standalone proxy from code? (This options doesn't look particularly appealing since we'll have to write boilerplate code to wrap the REST API)
3. If no, then am I correct assuming the remote Selenium Webdriver will connect to the website under test through the newly set up embedded proxy by means of tunnelling (Sauce Connect and alike)?
4. What is the best practice of using Browsermob with CI server with cloud-based testing platforms?

Answer 1

If the test/webdriver instance will be running on a remote machine (browserstack or sauce) in your case, it is essential that the proxy generated by your proxy server should be authenticated on the remote machine to intercept traffic. I had a similar requirement and I set it up using a standalone BrowserMob instance. Below is a working sample code for browserstack with their local testing binary:

This will need the following Dependencies:

<dependency>
    <groupId>com.browserstack</groupId>
    <artifactId>browserstack-local-java</artifactId>
    <version>1.0.0</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>net.lightbody.bmp</groupId>
    <artifactId>browsermob-core</artifactId>
    <version>2.1.5</version>
    <scope>test</scope>
</dependency> 

Code snippet:

import com.browserstack.local.Local;
import net.lightbody.bmp.BrowserMobProxy;
import net.lightbody.bmp.BrowserMobProxyServer;
import net.lightbody.bmp.client.ClientUtil;
import net.lightbody.bmp.core.har.Har;
import net.lightbody.bmp.proxy.CaptureType;
import org.openqa.selenium.Proxy;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.ie.InternetExplorerDriver;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;

import java.io.File;
import java.net.URL;
import java.util.HashMap;

public class InterceptProxy {
    public static final String USERNAME = <BrowserStack Username>;
    public static final String AUTOMATE_KEY = <BrowserStack Key>;
    public static final String URL = "https://" + USERNAME + ":" + AUTOMATE_KEY + "@hub-cloud.browserstack.com/wd/hub";

    public static void main(String[] args) throws Exception {
        BrowserMobProxy proxy = new BrowserMobProxyServer();
        proxy.start(0);
        Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
        Local browserStackLocal = new Local();
        HashMap<String, String> browserStackLocalArgs = new HashMap<String, String>();
        browserStackLocalArgs.put("key", AUTOMATE_KEY);
        browserStackLocalArgs.put("forcelocal", "true");
        browserStackLocalArgs.put("forceproxy","true");
        browserStackLocalArgs.put("force","true");
        browserStackLocalArgs.put("v", "true");
        String host=seleniumProxy.getHttpProxy().substring(0,seleniumProxy.getHttpProxy().indexOf(":"));
        String port=seleniumProxy.getHttpProxy().substring(seleniumProxy.getHttpProxy().indexOf(":")+1,seleniumProxy.getHttpProxy().length());
        browserStackLocalArgs.put("-local-proxy-host", host);
        browserStackLocalArgs.put("-local-proxy-port", port);
        browserStackLocal.start(browserStackLocalArgs);

        DesiredCapabilities caps = new DesiredCapabilities();
        caps.setCapability("browser", "Chrome");
        caps.setCapability("browser_version", "62.0");
        caps.setCapability("os", "Windows");
        //caps.setCapability(CapabilityType.PROXY, seleniumProxy);
        caps.setCapability("os_version", "10");
        caps.setCapability("browserstack.local",true);

        WebDriver driver = new RemoteWebDriver(new URL(URL), caps);
        driver.manage().deleteAllCookies();
        driver.manage().window().maximize();
        proxy.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);

        // create a new HAR with the label "yahoo.com"
        proxy.newHar("yahoo.com");

        // open yahoo.com
        driver.get("http://yahoo.com");

        // get the HAR data
        Har har = proxy.getHar();

        //Writing Har to file
        har.writeTo(new File("/Users/MyUser/Desktop/HAR.txt"));
        driver.quit();
        browserStackLocal.stop();
        proxy.stop();
    }
}