Specify order of Specs2 Specification execution with SBT

Question

I am trying to specify the order in which Spec2 Specifications are run, I know about the sequential keyword that makes sure they run one after another but this corresponds to the tests within a Specification (and doesn't actually guarantee any order)

I found this SO question: https://stackoverflow.com/a/15832297/1757402 which looked promising but again, seems to just sort the tests within a Specification

I am assuming SBT/Specs runs the Specifications in the order in which the JVM returns the classes, is there any way to change this? Or any way to guarantee an order?

So say I have the following Specifications:

CApplicationSpec.scala

@RunWith(classOf[JUnitRunner])
class CApplicationSpec extends Specification {
  "CApplicationSpec" should {
    "be OK" in new WithApplication{
      OK must equalTo(OK)
    }
  }
}

BApplicationSpec

@RunWith(classOf[JUnitRunner])
class BApplicationSpec extends Specification {
  "BApplicationSpec" should {
    "be OK" in new WithApplication{
      OK must equalTo(OK)
    }
  }
}

At the moment if I test these, the order of execution can change each time, I want a way to be able to guarantee that BApplication (or any other Spec) will always run first, maybe by sorting them alphabetically?

@Łukasz When specific actions that occur in other tests are run before specific tests they cause the tests to fail due to thread executor exceptions — RichyHBM, Jan 17 '16 at 17:47

score 1 · Answer 1 · answered Jan 18 '16 at 04:06

You can create a specification which will "link" other specifications and run them in a specified order:

object AllSpecifications extends Specification { def is = s2"""
  ${"run BSpecification" ~ BSpecification}
  ${"run CSpecification" ~ CSpecification}
 """
}

score 1 · Answer 2 · answered Jan 18 '16 at 13:28

I ended up doing it via SBT with testGrouping

//Put all tests into a single Group so that SBT reports correct number of tests, but order them by test name
testGrouping in Test <<= definedTests in Test map { tests => {
    val sortedTests = tests map {
      test => new Tests.Group(test.name, Seq(test), Tests.InProcess)
    } sortBy (_.name.toLowerCase) flatMap {
      _.tests
    }
    Seq(new Tests.Group("Tests", sortedTests, Tests.InProcess))
  }
}

Which orders all the tests alphabetically (including package name) I then have all specs I want to run first in a specific package

Specify order of Specs2 Specification execution with SBT

2 Answers2