2

So -- I can, from my laptop do the following: testcafe "saucelabs:Chrome@70.0:Windows 10" tests/settings/users.js

This will, connect to SauceLabs, to launch the test (or multiple tests, works either way), to hit back against our development server. (IP's have been whitelisted, it all works).

I can also, of course, login to sauce labs, open a "live testing" browser and connect to this environment/login etc.

Thus, the "SauceLabs =-> My environment" is fine.

If I try to run a build on circleci to do exactly this it fails.

In my role I have the following code:

const getPageHTML = ClientFunction(() => document.documentElement.outerHTML);
let value = await getPageHTML();
console.log(value);
const pageLink = ClientFunction (() => window.location.href);
value = await pageLink();
console.log(value);

This will print out that the entire outerHTML is only 

<html><head></head><body></body></html>

It will print out that the current url is "about:blank"

So, my question.. when running tests, using saucelabs, FROM CIRCLE, would I need to whitelist CIRCLE?

I can't seem to figure out what the functional/execution difference is between running the tests from my laptop vs circle when we are USING saucelabs browsers.

Is the "IP from which the tests are being run" (whitelist ip) circleCI at this point, or saucelabs? I, of course, have been presuming it is saucelabs.

NOTE: I actually was able to just take the testcafe "example.js" test, and get it to run using circle/saucelabs. So this literally has something to do with just running the tests back towards my environment. I'd love to learn more about what the "source ip range" would be if circle is running tests "using" saucelab browsers.

Any advice would be greatly appreciated!

(just having a very hard time trying to figure out how to debug/triage the issue)

Alex Skorkin
  • 4,034
  • 3
  • 21
  • 44
Roger Studner
  • 108
  • 1
  • 7

2 Answers2

3

For SauceLabs to exercise an application running within CircleCI containers, you will need to use Sauce's Connect Tunnel since services running inside a CircleCI container are not internet accessible.

CircleCI and SauceLabs partnered to create an "orb" that simplifies the setup of a tunnel. I'm not familiar with TestCafe, but it should solve the connectivity issues.

You can see an example of SauceLabs (though using Maven/Java) in CircleCI here, https://github.com/eddiewebb/demo-blueskygreenbuilds/blob/master/.circleci/config.yml#L17, but basically

  - saucelabs/with_proxy:
      name: "Chrome Tests"
      requires:
        - "Unit Test"
      tunnel_identifier: chrome
      steps:
        - maven/with_cache:
            steps:
              - run: ./mvnw verify -B -Dspring.profiles.active=it -Dsauce.tunnel="chrome"
Eddie
  • 8,370
  • 3
  • 41
  • 54
  • So, the "application" is not running in Circle.. all circle has, is the javascript test code (testcafe code). The actual website I want to test is https://. – Roger Studner Dec 06 '18 at 16:44
  • @RogerStudner, Testcafe is also an application that starts its own proxy server. Therefore, you need to use Sauce's Connect Tunnel for connection with CircleCI, as Eddie mentioned in his answer. – Artem Dec 07 '18 at 11:24
  • Artem - I'd love to be able to chat about this with you outside SO. I can: 1) go to saucelabs and open a live session to hit my companies testing site 2) run tests from my laptop, using a saucelabs browser, to hit my companies test site. 3) run tests, from circleci, using a saucelabs browser, to test PUBLIC FACING websites 4) I cannot, run tests, from circleci, using a saucelabs browser, to test my companies testing site (which is whitelisted to saucelabs / the above 2 OTHER scenarios work). – Roger Studner Dec 07 '18 at 14:27
  • @RogerStudner, Testcafe proxy server should have access from CircleCI to your site. Could you please clarify if you used the Sauce's Connect Tunnel when SauceLabs Live session started as mentioned in the first statement? If yes, probably, the path of the tested site is local and it cannot be accessed. You can try to make a corporate VPN and connect to it from the CircleCI container. If no, the path of the tested site is protected by proxy settings and SauceLabs is whitelisted. Then solution is to add CircleCI to your company white list. – Artem Dec 14 '18 at 10:52
  • @Artem - Ahh, you said something there that might be the key. I could not find clarity on how the testcafe proxy server worked in regards to controlling a saucelabs browser. I guess I was always under the impression that from my servers perspective, the incoming IP address would have been saucelabs and NOT circleci (since whitelisting circle is "all of EC2" basically). When I turn off my firewall this all works, which would make sense given your statement. So basically testcafe gets ahold of a saucelabs browser, but is routing that browser THROUGH circle back to my site? – Roger Studner Dec 15 '18 at 13:54
2

You need to use Sauce's Connect Tunnel for connection with CircleCI, as @Eddie mentioned in his answer.

The tested site is protected by proxy settings and SauceLabs is whitelisted. The solution is to add CircleCI to your company's white list.

SauceLab browser ===tunnel(ok)===> Testcafe proxy server (CircleCI) ===firewall(fail)===> Site on your machine

Artem
  • 856
  • 4
  • 11