0

Trying to use PhantomJS(com.codeborne:phantomjsdriver:1.2.1) for some headless browser testing along with BrowserMob Proxy(browsermob-proxy-2.0-beta-9) to capture HAR files and Javascript execution.

It works for urls with https(eg. https://www.google.com) and I get the HAR.
However for http(eg. http://www.google.com) I get the following error in the BrowserMob logs

INFO 02/02 22:45:03 n.l.b.p.j.h.HttpSer~ - Version Jetty/5.1.x
INFO 02/02 22:45:03 n.l.b.p.j.u.Contain~ - Started HttpContext[/,/]
...
INFO 02/02 22:46:29 n.l.b.p.h.BrowserMo~ - java.net.UnknownHostException: www.google.com when requesting http://www.google.com/
INFO 02/02 22:46:54 n.l.b.p.j.u.Threade~ - Stopping Acceptor ServerSocket[addr=/0.0.0.0,localport=13000]
...

Following is how I setup PhantomJS

public RemoteWebDriver getDriverInstance() {
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setJavascriptEnabled(true);

    //code to get Proxy is below
    capabilities.setCapability(CapabilityType.PROXY, getProxyObject()); 

    capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_EXECUTABLE_PATH_PROPERTY, "./bin/phantomjs");
    capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
    capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS, new String[] {"--web-security=no", "--ssl-protocol=any", "--ignore-ssl-errors=yes"});

    WebDriver webDriver = new PhantomJSDriver(capabilities);
    return (RemoteWebDriver) webDriver;
}

public Proxy getProxyObject() {
    Proxy proxy = new Proxy();

    //publicIp is localhost for testing purposes.
    String proxyLocation = this.getPublicIp() + ":" + this.getBrowserMobProxyPort();

    proxy.setHttpProxy(proxyLocation);
    proxy.setFtpProxy(proxyLocation);
    proxy.setSslProxy(proxyLocation);
    return proxy;
}

Still looking for a solution.
Is it normal to expect such messages from BrowserMob?

I, most probably have not setup something correctly or missed a part. Would be awesome if anyone who has faced this issue, help me out or point me to a solution. I have done some searching but not found a solution that resolves this.

Also if there is additional information needed, please let me know.

Programmer
  • 165
  • 2
  • 15

1 Answers1

0

I am using a newer version of BrowserMob Proxy, but the following Scala code works for me in loading HTTP and HTTPS websites:

import java.io.File

import net.anthavio.phanbedder.Phanbedder
import net.lightbody.bmp.BrowserMobProxyServer
import net.lightbody.bmp.client.ClientUtil
import net.lightbody.bmp.core.har.HarEntry
import org.apache.commons.io.FileUtils
import org.openqa.selenium.OutputType
import org.openqa.selenium.phantomjs.PhantomJSDriver
import org.openqa.selenium.phantomjs.PhantomJSDriverService._
import org.openqa.selenium.remote.{CapabilityType, DesiredCapabilities}

import scala.collection.JavaConversions._

object PhantomJSTest {
  def main(args: Array[String]) {
    val bm = new BrowserMobProxyServer
    bm.start(0)

    val proxy = ClientUtil.createSeleniumProxy(bm)

    val phantomjs = Phanbedder.unpack()
    val capabilities = new DesiredCapabilities
    capabilities.setCapability(CapabilityType.PROXY, proxy)
    capabilities.setCapability(PHANTOMJS_EXECUTABLE_PATH_PROPERTY,
                               phantomjs.getAbsolutePath())
    capabilities.setCapability(PHANTOMJS_CLI_ARGS,
      Array[String]("--web-security=no",
                    "--ssl-protocol=any",
                    "--ignore-ssl-errors=yes"))

    val driver = new PhantomJSDriver(capabilities)

    run(bm, driver, "http://www.google.com")
    run(bm, driver, "https://www.google.com")

    driver.quit
    bm.stop
  }

  def run(bm: BrowserMobProxyServer, driver: PhantomJSDriver, url: String) {
    bm.newHar(url)
    driver.get(url)

    val har = bm.getHar
    har.getLog.getEntries.foreach { e: HarEntry =>
      println(e.getRequest.getUrl)
    }

    val file = new File(s"screenshot-${System.currentTimeMillis}.png")
    FileUtils.copyFile(driver.getScreenshotAs(OutputType.FILE), file)
    println(s"Captured loading of ${url} screenshot to ${file.getCanonicalPath}")
  }
}

Here are the libraries I am using (from my build.gradle file):

compile 'commons-io:commons-io:2.4'
compile 'org.slf4j:slf4j-simple:1.7.16'
compile 'net.lightbody.bmp:browsermob-core-littleproxy:2.1.0-beta-4'
compile 'org.seleniumhq.selenium:selenium-java:2.45.0'
compile 'com.codeborne:phantomjsdriver:1.2.1'
compile 'net.anthavio:phanbedder-2.1.1:1.0.0'

Hope this helps you debug the issue you are running in to.

Binil Thomas
  • 13,450
  • 10
  • 53
  • 68