Browsermob Proxy + Watir not capturing traffic continuously

Question

I have the BrowserMob Proxy set up correctly with Watir and it is capturing traffic and saving the HAR file; however, what it's not doing is that it's not capturing the traffic continuously. So following is what I'm trying to achieve:

1. Go to homepage
2. Click on a link to go to another page where I need to wait for some events to happen
3. Once on the second page, start capturing traffic after the event happens and wait for a specific call to occur and capture its contents.

What I'm noticing however, is that it's following all of the above steps, but on step 3 the proxy stops capturing traffic before that call is even made on that page. The HAR that is returned doesn't have that call in it hence the test fails before it even does its job. Following is how the code looks like.

class BMP
attr_accessor :server, :proxy, :net_har, :sel_proxy

 def initialize
    bm_path = File.path(Support::Paths.cucumber_root + "/browsermob- 
    proxy-2.1.4/bin/browsermob-proxy")
    @server = BrowserMob::Proxy::Server.new(bm_path, {:port => 9999, 
       :log => false, :use_little_proxy => true, :timeout => 100})
    @server.start
    @proxy = @server.create_proxy
    @sel_proxy = @proxy.selenium_proxy

    @proxy.timeouts(:read => 50000, :request => 50000, :dns_cache => 
      50000)

    @net_har = @proxy.new_har("new_har", :capture_binary_content => 
      true, :capture_headers => true, :capture_content => true)
end

def fetch_har_entries(target_url)

  har_logs = File.join(Support::Paths.har_logs, "har_file # . 
  {Time.now.strftime("%m%d%y_%H%M%S")} .har")
  @net_har.save_to har_logs

  index = 0
  while (@net_har.entries.count > index) do

    if @net_har.entries[index].request.url.include?(target_url) && 
    entry.request.method.eql?("GET")
      logs = JSON.parse(entry.response.content.text) if not 
          entry.response.content.text.nil?
      har_logs = File.join(Support::Paths.har_logs, "json_file_# . 
          {Time.now.strftime("%m%d%y_%H%M%S")}.json")
      File.open(har_logs, "w") do |json|
         json.write(logs)
      end
      break
    end 
  index += 1
  end
 end
end

In my test file I have following

Then("I navigate to the homepage") do
 visit(HomePage) do |page|
  page.element.click
 end
end

And("I should wait for event to capture traffic") do 
 visit(SecondPage) do |page|
  page.wait_until{page.element2.present?)
  BMP.fetch_har_entries("target/url")
 end
end

What am I missing that is causing the proxy to not capture traffic in its entirety?

Answer 1

In case anyone gets here from a google search, I figured out how to resolve this on my own (thanks stackoverflow community for nothing, lol). So to resolve the issue, i used a custom retriable loop called eventually method.

 logs = nil
eventually(timeout: 110, interval: 1) do
  @net_har = @proxy.new_har("har", capture_binary_content: true, capture_headers: true, capture_content: true)
  @net_har.entries.each do |entry|
    begin
      break if @net_har.entries.index entry == @net_har.entries.count
      next unless entry.request.url.include?(target_url) &&
                  entry.request.post_data.text.include?(target_body_text)

      logs = entry.request.post_data.text
      break
    rescue TypeError
      fail("Response body for the network call came back empty")
    end
  end
  raise EOFError if logs_hash.nil?
end
logs
end

Basically I'm assuming what was happening was the BMP would only cache or capture 30 seconds worth of har logs, and if my network event didn't occur during those 30 secs, i was SOL. So the what above code is doing is that's it's waiting for the logs variable to be not nil, if it is, it raises an EOFError and goes back to the loop initializes the har again and looks for the network call again. It keeps on doing that until it find the call or 110 seconds are up. Following is the eventually method I'm using

def eventually(options = {})
  timeout = options[:timeout] || 30
  interval = options[:interval] || 0.1
  time_limit = Time.now + timeout
  loop do
    begin
     yield 
    rescue EOFError => error
  end
  return if error.nil?
  raise error if Time.now >= time_limit

  sleep interval
 end
end