Python BrowserMob Proxy with IE captures incorrect HAR?

Question

I am currently trying to use BrowserMob Proxy (v2.1.1) + Selenium (v2.5.3) for Python (v2.6) to test page load times and output them to HAR files. I need to test both Chrome and IE. I currently have it working perfectly for Chrome and it runs without errors on IE but it doesn't capture the correct data to the HAR file.

This image is a comparison of the two different HAR files I am getting. The first one is the result from IE and the second one is the result from Chrome. I need to be getting the same for both. I have a feeling that I am doing something wrong with the way I am setting up the proxy, but according to http://www.seleniumhq.org/docs/04_webdriver_advanced.jsp it should be basically the same thing for Chrome/IE like I have it. My thoughts are that it's not using the right proxy port or something, but I don't know how to fix it

As you can see, it also seems to be capturing what selenium is doing on the page which is not what I want. This is the code I am using:

class SeleniumObject:

    def __init__(self):
        # Start up the server
        self.server = Server(Config.BAT_PATH) #resolves to the location of browsermob-proxy-2.1.1/bin/browsermob-proxy.bat
        self.server.start()
        self.proxy = self.server.create_proxy()

    def setupDriver(self, browser):
        self.browser = browser.lower()
        PROXY = self.proxy.proxy

        # Chrome
        if self.browser == 'chrome':
            options = webdriver.ChromeOptions()
            options.add_argument("--start-maximized")
            desired_capabilities = webdriver.DesiredCapabilities.CHROME.copy()
            # Change the proxy properties of that copy.
            desired_capabilities['proxy'] = {
                "httpProxy":PROXY,
                "ftpProxy":PROXY,
                "sslProxy":PROXY,
                "noProxy":None,
                "proxyType":"MANUAL",
                "class":"org.openqa.selenium.Proxy",
                "autodetect":False
            }
            self.driver = webdriver.Chrome(chrome_options=options, desired_capabilities=desired_capabilities)

        # IE 
        if self.browser == 'ie':
            desired_capabilities = webdriver.DesiredCapabilities.HTMLUNITWITHJS.copy()
            desired_capabilities['proxy'] = {
                "httpProxy":PROXY,
                "ftpProxy":PROXY,
                "sslProxy":PROXY,
                "noProxy":None,
                "proxyType":"MANUAL",
                "class":"org.openqa.selenium.Proxy",
                "autodetect":False
            }
            self.driver = webdriver.Ie(capabilities=desired_capabilities) 

    def outputHAR(self):
            # Output the data as a HAR file
            self.har_json = json.dumps(self.proxy.har, indent=4, sort_keys=True)  # returns a HAR JSON blob
            open(self.browser + '-load-summary-' + self.sample_time + '.har', 'w').write(self.har_json)

    def setSampleTime(self):
        self.sample_time = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")

    def shutDown(self):
        self.setRegKey("ProxyEnable", 0)  # Forces the internet setting to stop using the proxy in case there was an error
        self.driver.quit()
        self.proxy.close()
        self.server.stop()

selenium = SeleniumObject()
selenium.setupDriver("chrome")
selenium.setSampleTime()
selenium.proxy.new_har("W3Schools")
selenium.driver.get("http://www.w3schools.com")
selenium.outputHAR()
selenium.shutDown()
print "Done!"

Florent B. · Accepted Answer · 2016-07-01T18:16:03.583

1

You need to ensure that the cache of IE is clean before launching the browser:

desired_capabilities  = webdriver.DesiredCapabilities.INTERNETEXPLORER
desired_capabilities ["ie.ensureCleanSession"] = True
driver = webdriver.Ie(capabilities=desired_capabilities )

Note that you are reading the metrics on the proxy. Thus you are only measuring the response times of each request and not the page load times.

edited Jul 01 '16 at 18:16

answered Jul 01 '16 at 18:06

Florent B.

37,063
6
68
92

This solved it. Thanks so much; it was starting to give me a headache. Out of curiosity, does the Chrome driver automatically clear the cache and that's why it wasn't an issue with Chrome? – Jd Francis Jul 01 '16 at 18:16
The Chrome driver creates a temporary profile at each launch, thus it always starts with a clean session. – Florent B. Jul 01 '16 at 18:18
Ahh, that makes a lot more sense now. Thanks for all the help! – Jd Francis Jul 01 '16 at 18:21
1

As an update for the latest 'iedriverserver.exe' (3.8.0), the structure of the desired capabilities Dict was modified and now nests the 'ie.ensureCleanSession' variable. In order to get around this, initialize the wrapping dict and set the variable as follows: caps = DesiredCapabilities.INTERNETEXPLORER caps['se:ieOptions'] = {} caps['se:ieOptions']['ie.ensureCleanSession'] = True caps['ignoreZoomSetting'] = True self.driver = webdriver.Ie(capabilities=caps) – chicagoCrazy Dec 11 '17 at 17:19

Python BrowserMob Proxy with IE captures incorrect HAR?

1 Answers1

Linked