How to deal with Mockito's UnnecessaryStubbingException when using @ParameterizedTest

Question

I'm moving some tests to newer Mockito versions and I'm running into a wall when it comes to @ParameterizedTest tests and Mockito's UnnecessaryStubbingsException.

The issue is that the test in question sometimes needs to have a service mocked, depending on the parameters of the test. For some parameters, the code will not execute to the line where the mocked service is called, while for other parameters it will.

This results in Mockito throwing the UnnecessaryStubbingsException for the cases where the mock is unused. I can't remove the stub because then the test will fail for parameters where the code actually executes to the point where the service needs to be mocked.

To illustrate, let's say I have this dummy method I'm testing:

public boolean process(String flag) {
    if (Objects.equals(flag, "flag1")) {
        throw new IllegalArgumentException("Oh no, exception!");
    }
    boolean result = someService.execute(flag);
    if (result) {
        throw new IllegalArgumentException("Oh no, exception!");
    }
    return result;
}

And then the parameterised test to check multiple flags:

@ParameterizedTest
@MethodSource("getFlags")
void shouldTestIfFlagWorks(String someFlag) {
    // Given
    Mockito.doReturn(true).when(someService).execute(someFlag);

    // When
    Throwable thrown = Assertions.catchThrowable(() -> serviceUnderTest.process(someFlag));

    // Then
    Assertions.assertThat(thrown).hasMessage("Oh no, exception!");
}

private static Stream<Arguments> getFlags() {
    return Stream.of(
        Arguments.of("flag1"),
        Arguments.of("flag2")
    );
}

The example is a bit contrived, but this will fail with UnnecessaryStubbingsException because the first parameter the test runs with doesn't need the mock. If I remove the stub, the test with the first parameter will work, while it will fail once it runs for the second time with the next parameter.

Is there any way to circumvent this? One option I know would solve this is to use Mockito.lenient(). The other option is to refactor and move the parameters that need the mock to a separate test.

I'm curious if there's any different / better approach, or something else that I'm missing here.

Answer 1

if you are using mockito with scala, here's a good way to handle a library upgrade that is suddenly enforcing a tighter strictness against legacy unit tests. In my case I went from a direct use of the mockito core library to using the lowest version of mockito-scala with mockito-scala-test (which themselves were dependent on a high mockito-core 2.x that introduced a more strict enforcement). I immediately had many hundreds of legacy unit tests failing due to the change in strictness that the newer version of mockito stared enforcing.

I created a simple trait:

trait LenientMockitoSugar extends MockitoSugar {
   override val strictness: Strictness = Strictness.Lenient
}

For every class that had test failures, I replaced its extention of MockitoSugar with this trait. That handled more than 95% of my test failures.

This approach will allow for a later migration of unit tests to a more strict usage, one test class at a time.

Answer 2

With junit 5.x and mockito 4.x the UnnecessaryStubbingsException will only show up, when you add @ExtendWith(MockitoExtension.class) to your test class like this:

@ExtendWith(MockitoExtension.class)
class MainTest {
  private Service someService;
  private MainService serviceUnderTest;

  @BeforeEach
  void setup() {
    this.someService = Mockito.mock(Service.class);
    this.serviceUnderTest = new MainService(someService);
  }
//...see full code example below
}

But that @ExtendWith(MockitoExtension.class) is actually not necessary for this type of test. Mockitos JavaDoc says: "Extension that initializes mocks and handles strict stubbings." So, as long as you don't use the @Mock annotation in your tests, you ain't gonna need it.

But if you need to use @ExtendWith(MockitoExtension.class) then you have several options:

You can replace
Mockito.doReturn(true).when(someService).execute(someFlag);
by
Mockito.lenient().doReturn(true).when(someService).execute(someFlag); or alternatively add above the test class:
@MockitoSettings(strictness = Strictness.LENIENT)
and your exception goes away again. Or another option would be to use:

@Mock(strictness = Mock.Strictness.LENIENT)
private Service someService;

You might also be able to use this old, deprecated annotation option.

@Mock(lenient = true) //deprecated!
Service someService;

As Andreas Siegel suggested in the other answer above, but as I said @Mock(lenient = true) is actually deprecated.

You can get and run a complete code example using:

git clone --depth 1 --branch SO74233801 git@github.com:bodote/playground.git
cd backend
gradle test --tests '*MainTest*' --console plain

The code without any annotations and therefore without the UnnecessaryStubbingsException showing up looks like:

public interface Service {
  public boolean execute(String flag) ;
}

and

public class MainService {
  private Service someService;

  public MainService(Service service) {
    this.someService = service;
  }

  public boolean process(String flag) {
    if (Objects.equals(flag, "flag1")) {
      throw new IllegalArgumentException("Oh no, exception!");
    }
   boolean result = someService.execute(flag);
    if (result) {
      throw new IllegalArgumentException("Oh no, exception!");
    }
    return result;
  }
}

when the Test is:

class MainTest {

  private Service someService;
  private MainService serviceUnderTest;
  @BeforeEach
  void setup(){
    this.someService = Mockito.mock(Service.class);
    this.serviceUnderTest = new MainService(someService);
  }

  @ParameterizedTest
  @MethodSource("getFlags")
  void shouldTestIfFlagWorks(String someFlag) {
    // Given
    Mockito.doReturn(true).when(someService).execute(someFlag);

    // When
    Throwable thrown = Assertions.catchThrowable(() -> 
       serviceUnderTest.process(someFlag));

    // Then
    Assertions.assertThat(thrown).hasMessage("Oh no, exception!");
  }

  private static Stream<Arguments> getFlags() {
    return Stream.of(
            Arguments.of("flag1"),
            Arguments.of("flag2")
    );
  }
}

the test run shoes no UnnecessaryStubbingsException:

BTW I'm using these dependencies:

testImplementation 'org.mockito:mockito-core:4.8.1'
testImplementation 'org.mockito:mockito-junit-jupiter:4.8.1'
testImplementation group: 'org.junit.jupiter', name: 'junit-jupiter', version: '5.9.1'

Now try adding @ExtendWith(MockitoExtension.class) and run the test again. The UnnecessaryStubbingsException should show up now, until you either replace
Mockito.doReturn(true).when(someService).execute(someFlag);
by
Mockito.lenient().doReturn(true).when(someService).execute(someFlag); or alternatively add above the test class:
@MockitoSettings(strictness = Strictness.LENIENT)
and your exception goes away again.

I just also checked with testImplementation group: 'org.mockito', name: 'mockito-all', version: '2.0.2-beta' but each time with junit-jupiter 5.9.1: same result. Junit 4 however does have different annotations and behaves quite differently.

Answer 3

I don't think you're missing something. All you mentioned is correct.

The way I understand it is that UnnecessaryStubbingsException is Mockito's way of telling you:

Hey, you defined some mock behavior for your test that is not necessary because we never get to that point. You should better remove it to keep your test clean.

And if you are fine with unnecessary definitions of mock behavior because some of the arguments of the parameterized test will lead to the mock being called while others won't, lenient would be the way to go. So the exception goes away if you define your mock like this:

@Mock(lenient = true)
Service someService;

So my actual personal answer is: It's good to have that exception to think about the code and the test and the exception again, and then decide that this is exactly what I want to do and set the mock to lenient.