Protractor tests run instable ( ETIMEDOUT )

Question

We are working to make our Protractor E2E tests stable, so our development and deployment flow can depend on it. However, until now we are not able to have a fully stable run of E2E tests.

Sometimes, inconsistent which, some test somewhere is failing due a ETIMEDOUT in ClientRequest in selenium-webdriver.

Failed: ETIMEDOUT connect ETIMEDOUT 5.255.93.10:80
   Stack:
     Error: ETIMEDOUT connect ETIMEDOUT 5.255.93.10:80
         at ClientRequest.<anonymous> (/var/lib/jenkins/workspace/HPS UI E2E Tests/node_modules/grunt-protractor-runner/node_modules/selenium-webdriver/http/index.js:365:15)
         at emitOne (events.js:90:13)
         at ClientRequest.emit (events.js:182:7)
         at Socket.socketErrorListener (_http_client.js:306:9)
         at emitOne (events.js:90:13)
         at Socket.emit (events.js:182:7)
         at emitErrorNT (net.js:1265:8)
         at _combinedTickCallback (internal/process/next_tick.js:74:11)
         at process._tickCallback (internal/process/next_tick.js:98:9)
     From: Task: WebDriver.findElements(By(css selector, *[id="chart_doubleSlider_brush"]))
         at WebDriver.schedule (/var/lib/jenkins/workspace/HPS UI E2E Tests/node_modules/grunt-protractor-runner/node_modules/selenium-webdriver/lib/webdriver.js:344:17)
         at WebDriver.findElements (/var/lib/jenkins/workspace/HPS UI E2E Tests/node_modules/grunt-protractor-runner/node_modules/selenium-webdriver/lib/webdriver.js:897:22)
         at /var/lib/jenkins/workspace/HPS UI E2E Tests/node_modules/grunt-protractor-runner/node_modules/protractor/built/element.js:141:44
         at Promise.invokeCallback_ (/var/lib/jenkins/workspace/HPS UI E2E Tests/node_modules/grunt-protractor-runner/node_modules/selenium-webdriver/lib/promise.js:1329:14)
         at TaskQueue.execute_ (/var/lib/jenkins/workspace/HPS UI E2E Tests/node_modules/grunt-protractor-runner/node_modules/selenium-webdriver/lib/promise.js:2790:14)
         at TaskQueue.executeNext_ (/var/lib/jenkins/workspace/HPS UI E2E Tests/node_modules/grunt-protractor-runner/node_modules/selenium-webdriver/lib/promise.js:2773:21)
         at /var/lib/jenkins/workspace/HPS UI E2E Tests/node_modules/grunt-protractor-runner/node_modules/selenium-webdriver/lib/promise.js:2652:27
         at /var/lib/jenkins/workspace/HPS UI E2E Tests/node_modules/grunt-protractor-runner/node_modules/selenium-webdriver/lib/promise.js:639:7
         at process._tickCallback (internal/process/next_tick.js:103:7)

We would like to have a way to stabilise this test, for example to have some kind of retry mechanism in the Protractor run or on low level a retry mechanism of the failed http request. Right now, we have not found any solution for it.

More information about our setup:

• Our test-scripts are executed by a Jenkins machine.
• We use grunt-protractor-runner to start our protractor tests.
• The tests are executed on BrowserStack.
• We set up a BrowserStack tunnel between BrowserStack and our Jenkins machine before the test start up. Since the application we test is on a private environment. ( we use https://www.npmjs.com/package/browserstacktunnel-wrapper )
• We test on IE11, Chrome ( latest ), FireFox ( latest, 38, 31 ). However there is no relation between timeouts and the browser where testing on.
• The execution time of our tests runs are 30 minutes for each browser.
• We have set options.agent to http.Agent({keepAlive: true,keepAliveMsecs: 30 * 1000, timeout: 10 * 30 * 1000} of our http request, to keep the sockets open.
  • In the capabilities of Protactor we set 'browserstack.local': true and 'browserstack.debug': true.

We would like to hear your advice on how to stabilise our Protractor tests.