4

I am using selenium 3.0.2 and browsermob proxy 0.7.1 to capture the network data. All I am getting is an empty JSON. My code is:

server = Server("/Users/dev/Downloads/browsermob-proxy-2.1.2/bin/browsermob-proxy")
server.start()
proxy = server.create_proxy()

if browser is None:
    profile = webdriver.FirefoxProfile()
    profile.set_preference("browser.privatebrowsing.autostart", True)

browser = webdriver.Firefox(firefox_profile=profile, proxy=proxy.selenium_proxy())
proxy.new_har("google", options={'captureHeaders': True, 'captureContent': True})

browser.get("https://google.com/")

print(proxy.har)

All I get is this empty JSON

{'log': {'pages': [{'id': 'google', 'comment': '', 'pageTimings': {'comment': ''}, 'startedDateTime': '2016-12-01T14:23:24.984-05:00', 'title': 'google'}], 'entries': [], 'version': '1.2', 'creator': {'comment': '', 'name': 'BrowserMob Proxy', 'version': '2.1.2'}, 'comment': ''}}

Sourabh Dev
  • 673
  • 7
  • 21
  • I would also really like to know the answer to this question! The HAR is empty, I think, because the proxy isn't actually being used. For http (rather than https) sites, I can get the code to work by using: `profile.set_proxy(proxy.selenium_proxy())` in place of passing the proxy object in, but the `set_proxy` method is deprecated... For HTTPS, I think this issue is also relevant: https://bugzilla.mozilla.org/show_bug.cgi?id=1103196 – user3468054 Dec 02 '16 at 15:53
  • Is the page loading successfully in the browser? Since you're getting an empty HAR, that means the proxy is started, but the proxy hasn't seen any requests. – Jason Hoetger Dec 15 '16 at 01:31

5 Answers5

0

I suspect this is caused by the same GeckoDriver bug/missing feature in this question: GeckoDriver does not currently support the "proxy" desired capability that ChromeDriver and the old FirefoxDriver supported.

Until GeckoDriver/Marionette is updated, you can work around this problem by setting the proxy values on the Firefox Profile directly. The relevant proxy fields on the profile are:

  • network.proxy.http
  • network.proxy.http_port
  • network.proxy.ssl
  • network.proxy.ssl_port
  • network.proxy.type (set to 1, "manual")

BrowserMob Proxy has a test that shows how to do this in Java. It should be easy to translate that into a Python equivalent, since you already have a firefox_profile object.

Community
  • 1
  • 1
Jason Hoetger
  • 5,650
  • 2
  • 13
  • 14
0

1.Browser mob 0.7.1 doesn't have SSL support for intercepting https sites.
2. From your question I observed you gave https URL , try using browsermob 2.1.2 and above
3. Install SSL certificates in device In which you want to intercept network(SSL certificates are available in the browsermob proxy .zip file you download ).
4. If issue didn't resolve inform me I will look deep into other things causing issues.

bharath mamillapalli
  • 51
  • 4
0

I have expected the same issue when I changed my code recently. Try to add the proxy into the profile instead of the webdriver itself. It's deprecated but it works for me.

profile.set_proxy(proxy.selenium_proxy())
browser = webdriver.Firefox(firefox_profile=profile)
Kailackous
  • 1
0

Try this cmd maybe fix your problem:

yum install -y google-chrome-stable

Wee
  • 243
  • 3
  • 9
-1

Have you tried changing

This:

server = Server("/Users/dev/Downloads/browsermob-proxy-2.1.2/bin/browsermob-proxy")

To This: 

server = Server("/Users/dev/Downloads/browsermob-proxy-2.1.2/bin/browsermob-proxy.bat")

You need to include the file extension (.bat) at the end of your BrowserMob server path for it to work :)

David
  • 9,319
  • 1
  • 34
  • 44
dan.brown
  • 139
  • 7