IntelliJ running tests one by one

Question

To be short - I' just wrote a singleton class. Like the all singletons consists my singleton of a private constructor and a static field containing a predefined amount of objects of the class.

The case is - the singleton class has to be initialized using a JSON file and the expected behavior is it throwing an exception when no files from a given list are present.

The problem is that apparently IntelliJ (or maybe just JUnit) is loading all test classes and then executing them one by one in one program in a random order. This means that by the time the class which test the wrong initialization has been reached, the singleton already has been initialized. This means that every time I run all the tests at once I'll have one test which did not pass and when I run them one by one manually it'll result in all the tests passing.

Is there a way to force IntelliJ to run all the test classes separated?

Edit

I don't know how this could change anything, but lets assume it will. Here is a sample code with the exact same behavior. It's an IDE question, not a java speciffic one.

public class Singleton {
    private static Singleton singleton = new Singleton();

    public static Singleton getInstance() {
        return singleton;
    }

    private Singleton() {
        if (!new File("maybeIExist.txt").exists())
            throw new ExceptionInInitializerError();
    }

    public void doStuff() {
        System.out.println("stuff");
    }
}

And of course the tests:

public class SingletonTestOne {
    @Test(expected = ExceptionInInitializerError.class)
    public void test() throws Exception {
        Singleton.getInstance();
    }
}

And the second one:

public class SingletonTestTwo {
    @Before
    public void before() throws Exception {
        //code creating file
    }
    @Test
    public void test() throws Exception {
        Singleton.getInstance();
        doStuff();
    }
    @After
    public void after() throws Exception {
        //code deleting file
    }
}

I added it. I don't know how the code added anything to the question, but here you have it. — Adrian Jałoszewski, Sep 02 '16 at 13:33
You should try [this](https://github.com/junit-team/junit4/wiki/test-execution-order) — jumb0jet, Sep 02 '16 at 13:41
@ahrytsiuk I have the methods grouped in different classes, because they check different behavior. Is there something like that, but for classes? I need to use the singleton in another test case and it is initialized there, before it enters the wrong initialization test. — Adrian Jałoszewski, Sep 02 '16 at 13:46
Theoretically, unit tests should be independent and this is perhaps why [JUnit does not guarantee their order](https://github.com/junit-team/junit4/wiki/test-execution-order). There is a [similar thread](http://stackoverflow.com/questions/3693626/how-to-run-test-methods-in-specific-order-in-junit4) where execution order is discussed, but I don't think it will be useful. Under the circumstances perhaps [using a new classloader](http://stackoverflow.com/questions/42102/using-different-classloaders-for-different-junit-tests) to test your specific singleton happy/sad path may be the way to go — Morfic, Sep 02 '16 at 13:59
I don't need them to be in an specific order. They only have to execute separated instead of being run in one go. — Adrian Jałoszewski, Sep 02 '16 at 15:29

score 1 · Accepted Answer · answered Sep 03 '16 at 14:36

This is not an IntelliJ problem, you would encounter the same when running mvn test. You can get the tests running with the following changes:

First change the Singleton class, so that it's singleton field is lazily created in the getInstance() call, notice that this method now needs to be synchronized. You need this to be able to reset the field.

public final class Singleton {
  private static Singleton singleton = null;

  public static synchronized Singleton getInstance() {
    if (singleton == null) {
        singleton = new Singleton();
    }
    return singleton;
  }

  private Singleton() {
    if (!new File("maybeIExist.txt").exists()) {
        throw new ExceptionInInitializerError();
    }
  }

  public void doStuff() {
    System.out.println("stuff");
  }
}

Next, in your test class you can reset the singleton field in a @Before method to null using reflection:

import org.junit.Before;
import org.junit.Test;

import java.lang.reflect.Field;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import static org.junit.Assert.fail;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.core.IsNull.notNullValue;

public class SingletonTest {

  @Before
  public void resetSingleton() throws NoSuchFieldException, IllegalAccessException {
    Field instance = Singleton.class.getDeclaredField("singleton");
    instance.setAccessible(true);
    instance.set(null, null);
  }

  @Test
  public void singletonCanFindFile() throws Exception {
    final Path path = Paths.get("maybeIExist.txt");

    Files.createFile(path);
    final Singleton instance = Singleton.getInstance();
    assertThat(instance, notNullValue());
    Files.deleteIfExists(path);
  }

  @Test(expected = ExceptionInInitializerError.class)
  public void singletonCannotFindFile() throws Exception {
      final Singleton instance = Singleton.getInstance();
      fail("no exception thrown!");
  }
}

IntelliJ running tests one by one

1 Answers1