Mockito: Is validation of framework usage enabled without @RunWith(MockitoJUnitRunner.class)

Question

We are using Testng 6.8.8 + Mockito 1.10.19 and, obviously we can't use MockitoJUnitRunner, but the validation framework still works! In this thread I read that it should not be the case. Can someone explain? Is this because we still have @Before* callbacks and MockitoAnnotations.initMocks(this)?

My code:

public class MealTransformerTest {

MealTransformer mealTransformer = new MealTransformer();

@Test(expectedExceptions = NotImplementedException.class)
public void shouldThrowException() throws NotImplementedException {
    mealTransformer.transform(any(),
            any(),
            any(),
            any());
}

} Nothing fails in this specific test, but when I run the suite, Mockito will tell me about incorrect use of matchers.

I can also do something like:

public class MealTransformerTest {

MealTransformer mealTransformer = new MealTransformer();

//I don't need it in the tests. But I need at least 1 @Mock or @Spy to trigger framework validation
@Mock
private CloneUtils cloneUtils;

@BeforeMethod
void setUp() {
    MockitoAnnotations.initMocks(this);
}


@Test(expectedExceptions = NotImplementedException.class)
public void shouldThrowException() throws NotImplementedException {
    mealTransformer.transform(any(),
            any(),
            any(),
            any());
}

@Test(expectedExceptions = NotImplementedException.class)
public void shouldThrowException123() throws NotImplementedException {
    mealTransformer.transform(any(),
            any(),
            any(),
            any());
}
}

I receive:

org.mockito.exceptions.misusing.InvalidUseOfMatchersException: 
Misplaced argument matcher detected here:
....

Don't get me wrong, I really like how it works, but I was surprised to see it without @RunWith(MockitoJUnitRunner.class).

You can put a breakpoint in the argument matcher and see why this exception is still being generated even without MockitoJUnitRunner. Even without the runner, the Mockito classes will still throw exceptions if they are used incorrectly. The runner simply provides some 'integration-level' validation on top of that. — David Rawson, Dec 08 '16 at 19:25
Please also look into http://stackoverflow.com/questions/10806345/runwithmockitojunitrunner-class-vs-mockitoannotations-initmocksthis?rq=1 and http://stackoverflow.com/questions/23273230/misplaced-argument-matcher-detected-here-you-cannot-use-argument-matchers-outsi?rq=1 .These might help — Naman, Dec 08 '16 at 19:42
`MockitoJUnitRunner` is for JUnit. TestNG will ignore it. So, having it or not doesn't change with testng tests. — juherr, Dec 08 '16 at 20:03
@JulienHerr, for sure. I have no doubts about that, I've been using both testNG and JUnit for a while now. — yuranos, Dec 08 '16 at 23:17
@nullpointer providing a link to the very same thread I mentioned myself in the question isn't very helpful, but thanks. — yuranos, Dec 08 '16 at 23:18

score 3 · Accepted Answer · edited May 23 '17 at 12:08

Matchers work via side-effects and static global state, so you can break this call into pieces to see what's happening.

MockitoAnnotations.initMocks(this);
// in @Before [1]

mealTransformer.transform(any(), any(), any(), any());
//              [6]       [2]    [3]    [4]    [5]

After init [1], Java sees four calls to any [2-5] and one call to transform [6]. Mockito, however, only sees the four calls to any; Because mealTransformer is not a mock, Mockito can't see its calls. If you ran that test in isolation, Mockito would be left with four recorded matchers that are not consumed. The JVM would tear down the test framework, and the test would pass—unless you have a call to validateMockitoUsage in an @After method, which would catch exactly that case.

When you have two tests, however, they stack up like this:

MockitoAnnotations.initMocks(this);
// [1]
mealTransformer.transform(any(), any(), any(), any());
//              [6]       [2]    [3]    [4]    [5]
// test passes! now the next test
MockitoAnnotations.initMocks(this);
// [7]
mealTransformer.transform(any(), any(), any(), any());
//              [12]      [8]    [9]    [10]   [11]

Here, step 7 happens in the same test execution in the same JVM as steps 1-6, which means that initMocks is called when there are 4 unconsumed matchers on the stack. This should never happen, so initMocks takes its first opportunity to catch that error. One symptom would be if the misused matcher exception corresponds to a matcher outside of the test that fails: testB failing due to the misuse of a matcher in testA. With the use of validateMockitoUsage in an @After message, you'd have the appropriate test fail instead.

thanks for the excellent answer. I also mentioned(a comment in the second snippet) that I need to use @Mock/@Spy to make it happen. Without test doubles, validation is not triggered. Did you have a chance to look at http://stackoverflow.com/questions/10806345/runwithmockitojunitrunner-class-vs-mockitoannotations-initmocksthis/10812752#10812752 ? I was just thinking: maybe the fact that testNG doesn't create a new test class instance for every test is the reason why validation works by default? JUnit will re-create the test class over and over again, so... — yuranos, Dec 08 '16 at 23:38
[CONTINUE] maybe that's why without @RunWith(MockitoJUnitRunner.class) Mockito will not trigger validation usage for JUnit tests, simply because it's not able to store its state. — yuranos, Dec 08 '16 at 23:38
@yuranos87 I suspect that it's simpler than that: In both cases you'll get validation only when you interact with Mockito next, which means single-test runs won't ever be validated. Though Mockito is relatively good at proactively validating, I'll bet a call to `initMocks` with no mocks to create is just an edge case that doesn't happen to call `mock` or `spy` in a way that actually validates. Finally, remember that mocking state is global but tied to the thread that invokes it, so if tests occur on separate threads they may not get the passive validation you're asking about. — Jeff Bowman, Dec 08 '16 at 23:55
'just an edge case' - oh, man, strangely as it sounds, taking into consideration Mockito's maturity now, it has a few edge cases. Thanks! — yuranos, Dec 09 '16 at 00:08

Mockito: Is validation of framework usage enabled without @RunWith(MockitoJUnitRunner.class)

1 Answers1

Linked