How to use browsermob with python-selenium behind a corporate proxy?

Question

My test environment is under a corporate proxy ("proxy.ptbc.std.com:2538").I want to open a particular video on YoTube for a period of time (eg 200 seconds) and capture the har file for each visit, the process is repeated several times for a massive test. I have tried different examples found here but the firefox / chrome browsers do not connect to the internet because they are behind the proxy.

How can run "python-selenium + browsermobproxy" behind a corporate proxy and capture the har file for each instance.

Example code:

 from browsermobproxy import Server
    server = Server("C:\\Utility\\browsermob-proxy-2.1.4\\bin\\browsermob-proxy")
    server.start()
    proxy = server.create_proxy()

    from selenium import webdriver
    profile  = webdriver.FirefoxProfile()
    profile.set_proxy(proxy.selenium_proxy())
    driver = webdriver.Firefox(firefox_profile=profile)


    proxy.new_har("google")
    driver.get("http://www.google.co.in")
    proxy.har # returns a HAR JSON blob

    server.stop()
    driver.quit()

Any help would be appreciated

score 0 · Answer 1 · edited Jun 20 '20 at 09:12

According to browsermob-proxy documentation:

Sometimes you will want to route requests through an upstream proxy server. In this case specify your proxy server by adding the httpProxy parameter to your create proxy request:
[~]$ curl -X POST http://localhost:8080/proxy?httpProxy=yourproxyserver.com:8080
{"port":8081}

According to source code of browsermob-proxy API for Python

def create_proxy(self, params=None):
    """
    Gets a client class that allow to set all the proxy details that you
    may need to.

    :param dict params: Dictionary where you can specify params
    like httpProxy and httpsProxy
    """
    params = params if params is not None else {}
    client = Client(self.url[7:], params)
    return client

So, everything you need is to specify params in create_proxy depending on what proxy you use (http or https):

from browsermobproxy import Server
from selenium import webdriver
import json

server = Server("C:\\Utility\\browsermob-proxy-2.1.4\\bin\\browsermob-proxy")
server.start()
# httpProxy or httpsProxy
proxy = server.create_proxy(params={'httpProxy': 'proxy.ptbc.std.com:2538'})
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)

proxy.new_har("google")
driver.get("http://www.google.co.in")
result = json.dumps(proxy.har, ensure_ascii=False)
print(result)

server.stop()
driver.quit()

How to use browsermob with python-selenium behind a corporate proxy?

1 Answers1

Linked