1

I'm using RobotFramework with Selenium library for writing automated test cases for various websites. I've encountered an issue where due to slow loading from third-party URLs, some pages take forever to load and I want to block them to speed up test execution.

However, I am stuck on implementing that solution via BrowserMob in Robot Framework. Can anyone help?

So far, I have this code: 

Start Browser
    ## Init BrowserMob Proxy
    ${BMPATH}    Join Path    ${EXECDIR}    browsermob-proxy-2.1.4    bin    browsermob-proxy.bat
    &{bmphost}    Create Dictionary    address=127.0.0.1    port=7070
    Start Local Server    ${BMPATH}
    # Create dedicated proxy on BrowserMob Proxy
    &{host}    Create Dictionary    address=127.0.0.1    port=7070
    ${BrowserMob_Proxy}=    Create Proxy    ${host}
    #Blacklist URLS
    Blacklist    https://.*\\.google.com/.*    404
    ## Configure Webdriver to use BrowserMob Proxy
    ${options}=    Evaluate    sys.modules['selenium.webdriver'].ChromeOptions()    sys, selenium.webdriver
    Call Method    ${options}    add_argument    --proxy\=127.0.0.1:7070
    #${proxy1}=    Evaluate    sys.modules['selenium.webdriver'].Proxy()    sys, selenium.webdriver
    #${proxy1.http_proxy}=    Set Variable    localhost:8888
    Create WebDriver    Chrome    chrome_options=${options}    #proxy=${BrowserMob_Proxy}
    Go to    https://www.google.com

The goal was to translate this selenium/java code: 

private WebDriver initializeDriver() throws Exception {
    // Start the server and get the selenium proxy object
    ProxyServer server = new ProxyServer(proxy_port);  // package net.lightbody.bmp.proxy

    server.start();
    server.setCaptureHeaders(true);
    // Blacklist google analytics
    server.blacklistRequests("https?://.*\\.google-analytics\\.com/.*", 410);
    // Or whitelist what you need
    server.whitelistRequests("https?://*.*.yoursite.com/.*. https://*.*.someOtherYourSite.*".split(","), 200);

    Proxy proxy = server.seleniumProxy(); // Proxy is package org.openqa.selenium.Proxy

    // configure it as a desired capability
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability(CapabilityType.PROXY, proxy);

    // start the driver   ;
    Webdriver driver = new FirefoxDriver(capabilities);

    return driver;

The code I have now, creates the server, the proxy and opens the chrome browser but fails to blacklist google.com and opens the page anyway. Tried multiple regexp expressions, all failed but I think that the error is made somewhere before that, either in the way of creating the proxy/server or the way to blacklist URL is not right.

Has anyone had success implementing this solution in RF? Can someone tell me what I'm missing?

Thanks

Stefan Mandovski
  • 21
  • 3
  • Have you tried to setup your browser manually to point to BrowserMob to see if the rules work in that scenario? – A. Kootstra Oct 08 '18 at 13:27
  • Sorry for the delayed answer @A.Kootstra but priorities changed and this was forgotten up until now. I managed to solve the problem with your suggestion, the proxy setup was not working. Thanks! Will upload the correct code for anyone who might need this – Stefan Mandovski Feb 19 '19 at 09:41

1 Answers1

1

For anyone who might need this, the solution is below: 

Start Browser
    [Documentation]    Start browser
    Set Selenium Implicit Wait    10
    ${BMPATH}    Join Path    ${EXECDIR}    browsermob-proxy-2.1.4    bin    browsermob-proxy.bat
    Start Local Server    ${BMPATH}
    ## Create dedicated proxy on BrowserMob Proxy
    &{host}    Create Dictionary    port=7070  
    ${BrowserMob_Proxy}=    Create Proxy    ${host}
    Blacklist    .*\/\/.*google.*    200
    ## Configure Webdriver to use BrowserMob Proxy
    ${options}=    Evaluate    sys.modules['selenium.webdriver'].ChromeOptions()    sys, selenium.webdriver
    Call Method    ${options}    add_argument    --start-maximized
    Call Method    ${options}    add_argument    --proxy-server\=localhost:7070
    Create WebDriver    Chrome    chrome_options=${options}

This will start the chrome browser maximized and route traffic via BMP. The regex in the Blacklist line means it will block all URL containing 'google' and return status 200 (this can be changed at your will).

Stefan Mandovski
  • 21
  • 3