2

Does anyone know if Xamarin Test Cloud, when you choose more than one device, executes the tests in parallel? I mean, parallel between devices, not between tests. Or sequentially?

Thanks

Sebastian Ferrari
  • 769
  • 1
  • 10
  • 18

2 Answers2

2

It depends on what you mean by running in parallel.

Opt-in to use single-device parallelization

The Xamarin Test Cloud has a feature called "parallelization", which you can only opt in to when you select a single device in the Test Cloud. With this approach, the Test Cloud can run multiple copies of identical devices of the same model & OS version; so that your tests return results faster. The maximum number of such devices in a run is still limited by device availability & your account's concurrency.

Automatic concurrent execution on multiple devices

However, with multiple devices selected; that form of "parallelization" cannot be selected. Yet Test Cloud will automatically run on multiple different devices at the same time, if those devices are available and you're not at your concurrency limit. For each individual device, the tests are run sequentially (though order is not guaranteed); but between separate devices they can run in parallel.

In either case, the Test Cloud does not at any point guarantee that the execution between devices will be actually "parallel" by syncing them up or anything like that; it just means it will run them on as many devices as it is able to within your license & technical limits, and that the test runs may overlap (which is the "parallel" aspect.)

user62171
  • 641
  • 7
  • 16
1

It is depends on your subscription plan. If your plan have more than 1 concurrent devices, then it will run parallel between devices. For example if one is having "Small Startup" plan which is having 3 concurrent devices, then your test will run in 3 devices parallel.

lowleetak
  • 1,312
  • 1
  • 8
  • 19
  • So, you are saying that it depends of the subscription plan and is not a configuration thing, right? – Sebastian Ferrari Aug 20 '17 at 17:31
  • 1
    It is combination of many factors. Even your plan enable you to have more than 1 concurrent devices, you will still bound to whether the devices you selected is currently free to run. All test is queue up to run in the actual devices in test cloud. If you are submitting your test by command line, you can also configure the parallel running using: --test-chunk (to run tests in parallel by method), --fixture-chunk (to run tests in parallel by fixture). I think there is also an option in Xamarin test cloud admin panel to configure this. – lowleetak Aug 21 '17 at 02:36
  • Thanks for your time @lowleetak, have a nice week! – Sebastian Ferrari Aug 21 '17 at 23:21