Ruby: Phantom.js blocked on specific site?

Question

I am using capybara poltergeist to automate a small script on tumblr.com

My script works fine with my chrome driver.. And my poltergeist driver loads all other websites just fine, but for some reason throws a Capybara::Poltergeist::StatusFailError when I try to load tumblr.

Steps for reproduction:

$ brew install phantomjs
$ gem install capybara
$ gem install poltergeist
$ gem install selenium-webdriver
$ irb


require 'capybara/poltergeist'

module Drivers
  class Poltergeist < Capybara::Poltergeist::Driver
    def needs_server?
      false
    end
  end
end

Capybara.register_driver :poltergeist_errorless do |app|
  Drivers::Poltergeist.new(app, js_errors: false, timeout: 10000, phantomjs_options: ['--load-images=no', '--ignore-ssl-errors=yes'])
end

session = Capybara::Session.new(:poltergeist_errorless)
session.visit('https://google.com') # This works fine
session.visit('https://tumblr.com') # This does not work?

I tried to set all of my headers to look my google chrome's request, but that also does not seem to fix it. Does anyone have any suggestions?

Stumped. Pretty hard to debug poltergeist but can't figure out why visiting tumblr results in a `{'status' => 'fail' }`. Works fine in selenium... — Anthony, Sep 16 '14 at 01:05
Thanks for looking into it, I really appreciate your time. I can't understand why this is such a hard problem to solve? — BananaNeil, Sep 16 '14 at 03:52

score 16 · Accepted Answer · edited Jun 01 '17 at 10:37

16

The problem is related to phantomjs SSL handshake fail. You can take my gist and run with phantomjs, you will see:

[cut]
= onResourceError()
  - unable to load url: "https://www.tumblr.com/"
  - error code: 6, description: SSL handshake failed
= onResourceReceived()
  id: 3, stage: "end", response: {"contentType":null,"headers":[],"id":3,"redirectURL":null,"stage":"end","status":null,"statusText":null,"time":"2014-09-16T12:06:05.547Z","url":"https://www.tumblr.com/"}
= onLoadFinished()
  status: fail
DONE WITH  fail WebPage(name = "WebPage")

Checking a little bit a workaround is to use --ssl-protocol=any in phantom, so your code will become:

Capybara.register_driver :poltergeist_errorless do |app|
  Drivers::Poltergeist.new(app, js_errors: false, timeout: 10000, phantomjs_options: ['--load-images=no', '--ignore-ssl-errors=yes', '--ssl-protocol=any'])
end

To work.

References:

edited Jun 01 '17 at 10:37

ndnenkov

33,260
9
67
97

answered Sep 16 '14 at 12:12

Enrico Carlesso

6,510
4
32
41

Thanks for this Enrico! – etusm Apr 24 '15 at 20:21
I still face this problem on my laptop, but on production. It's a random failure. – Chamnap Jul 18 '15 at 01:52
@Chamnap what do you mean by "on production"? Have you initialized Poltergeist with '--ignore-ssl-errors=yes' and '--ssl-protocol=any'? – Enrico Carlesso Jul 18 '15 at 07:46
I mean it's a random failure. Sometimes work on my laptop but not on production and vice versa. Yeah, I initialised with those options already, and also check with `ps aux | grep phantomjs`. There is an issue on poltergeist too, https://github.com/EFForg/phantom-of-the-capitol/issues/47. – Chamnap Jul 18 '15 at 12:44

Ruby: Phantom.js blocked on specific site?

1 Answers1