JUnit 5 – A First Look at the Next Generation of JUnit

Home  >>  JUnit  >>  JUnit 5 – A First Look at the Next Generation of JUnit

JUnit 5 – A First Look at the Next Generation of JUnit

On February 18, 2016, Posted by , In JUnit,Spotlight, By , , With 3 Comments

In the beginning of February, the JUnit 5 (aka JUnit Lambda) team has published an alpha release. Since JUnit 4 is among the most used items in my toolbox I thought it might be worth to have a look at the next major release.

I took the latest build for a spin and noted down the changes that I found noteworthy here.

Installing JUnit 5

It is probably needless to say that a project titled JUnit Lambda requires Java 1.8 or later. If that is given then including the library is straightforward. The latest revision of the current alpha release channel is available from Sonatype’s snapshots repository at
https://oss.sonatype.org/content/repositories/snapshots/org/junit/

The artifacts can be consumed with Maven and Gradle. If you prefer to manually maintain dependencies, there is also a zip distribution available that contains everything to compile and run JUnit 5.

At development time it is sufficient to depend on the org.junit:junit5-api module.

Note that when specifying the snapshot repository it should be configured to never cache artifacts so that always the latest version is used.

Cutting Loose from JUnit 4

As far as I can see, the new version is a complete rewrite of the library with no dependencies whatsoever on older versions. Thus, you can enjoy legacy free testing (for a while at least;-).

But of course there is a migration path that allows both versions to coexist and enables you to maintain the existing test code base while writing new tests with JUnit 5. More on this later.

Same but Different

But let’s finally look at how JUnit 5 tests look like. At first sight, not much has changed. A simple test class …

class FirstTests {
  @Test
  void firstTest() {
    fail();
  }
}

… is barely distinguishable from a JUnit 4 test.

But did you spot the little difference? Right, tests don’t need to be public anymore, but if you prefer still can be of course.

Though annotations are still used to identify methods to set up and tear down the test environment, their names have changed. What was @BeforeClass/AfterClass is now @BeforeAll/AfterAll and @Before/After are now named @BeforeEach/AfterEach.

Ignoring tests is also still possible with the @Disabled annotation.

@Test vs. @Test

As you have seen already, tests are still tagged with the @Test annotation. But be careful if you happen to also have JUnit 4 on your class path. JUnit 5 brings its own @Test annotation, thus make sure to import org.junit.gen5.api.Test which is the right one. Otherwise the JUnit 5 test runner won’t find your tests.

Another thing to note is that the new @Test annotation does not offer other services. If you were used to use timeout or expected from time to time, you will need to replace them in JUnit 5.

Running Tests with JUnit 5

It’s no wonder that there is no IDE support yet to run JUnit 5 tests. Therefore I used the ConsoleRunner to execute my experiments. Three more modules are required to run tests this way:

  • org.junit:junit5-engine
  • org.junit:junit-launcher
  • org.junit:junit-console

My IDE of choice is Eclipse, and in order to run tests with the ConsoleRunner from there I had to manually extend the Classpath of the launch configuration. Only after adding the test-classes output folder that contains the compiled tests, they would be picked up. But this quirk may as well be due to my meager Maven knowledge or due to a particularity in the Eclipse Maven integration.

The JUnit 5 team also provides basic plug-ins to execute tests in Maven and Gradle builds. See the Build Support chapter if you want to given them a try.

Test Method Parameters

In JUnit 5, methods are now permitted to have parameters. This allows injecting dependencies at the method level.

See also  JUnit in a Nutshell: Unit Test Assertion

In order to provide a parameter, a so-called resolver is necessary, an extension that implements MethodParameterResolver. Like with all other extensions, to use a resolver for a given method or class it needs to be declared with @ExtendWith. There are also two built-in resolver that don’t need to be explicitly declared. They supply parameters of type TestInfo and TestReporter.

For example:

class MethodParametersTest {

  @Test
  // implicitly uses TestInfoParameterResolver to provide testInfo
  void testWithBuiltIntParameterResolver( TestInfo testInfo ) {
    // ...
  }

  @Test
  @ExtendWith( CustomEnvironmentParameterResolver.class )
  // explicit resolver declared, could also be placed at class level
  void testWithCustomParameterResolver( CustomEnvironment environment ) {
    // ...
  }
}

If no matching parameter resolver can be found at runtime, the engine fails the test with a corresponding message.

The documentation states that there are plans to provide additional extensions, also one for dynamic test registration among them. With this extension in place, it would be possible to have parameterized tests. And given that test methods already accept parameters it seems likely that parameterized tests will also work at the method level.

Assertions

At first sight, assertions haven’t changed much, except that they are now homed in the org.junit.gen5.api.Assertions class.

But a closer look reveals that assertThat() is gone, and with it the unfortunate dependency on Hamcrest. These methods duplicated the API provided by MatcherAssert and tied previous versions of JUnit to the Hamcrest library. This dependency occasionally led to class resolution conflicts. In particular, when used with other libraries, that – even worse – include a copy of Hamcrest on their own.

Another change is the new assertAll() method that is meant to group assertions. For example

assertAll( "names", () -> {
  assertEquals( "John", person.getFirstName() );
  assertEquals( "Doe", person.getLastName() );
} );

will report a MultipleFailuresError containing all failed assertions within the group.

It is then the test executors responsibility to display this failure in a suitable way. The current ConsoleRunner implementation, however, doesn’t yet regard grouped failures and just reports the first one:

Finished:    testNames [junit5:com...GroupAssertionsTest#testNames()]
             => Exception: names (1 failure)
             expected: <John> but was: <Mary>

My first, unfiltered thought was that if grouping assertions were needed, it might be a sign to divide the code into multiple tests instead. But I haven’t used grouped assertions yet for real, and there may as well be places where they perfectly make sense.

 Testing with JUnit

Testing with JUnit Book

Testing with JUnit is one of the most valuable skills a Java developer can learn. No matter what your specific background, whether you’re simply interested in building up a safety net to reduce regressions of your desktop application or in improving your server-side reliability based on robust and reusable components, unit testing is the way to go.

Frank has written a book that gives a profound entry point in the essentials of testing with JUnit and prepares you for test-related daily work challenges.

  Get It Now! 

Testing Exceptions

Testing exceptions has been unified. To replace expected and ExpectedException there is now an expectThrows assertion that evaluates a lambda expression and verifies that it throws an exception of the given type.

For example,

@Test
void testException() {
  Foo foo = new Foo();

  Throwable exception = expectThrows( IllegalStateException.class, foo::bar );
    
  assertEquals( "some message", exception.getMessage() );
}

… will fail if calling foo::bar() does not throw an IllegalStateException. Otherwise the thrown exception will be returned and can be further verified. If the thrown exception is of no interest, there is also an assertThrows() method that returns void.

Goodbye Runner, Rule and ClassRule

JUnit 5 doesn’t know runners, rules, or class rules anymore. These partially competing concepts have been replaced by a single consistent extension model.

See also  JUnit in a Nutshell: Hello World

Extensions can be used declaratively by annotating a test class or test method with @ExtendWith. For example a test that wishes to have some fields initialized with mocked instances could use a Mockito extension like this:

@ExtendWith(MockitoExtension.class)
class MockedTest {

  @Mock
  Person person;
  
  // ...
  
}

If you are interested in more on this topic, you may want to read the separate post about extensions and how to migrate existing rules to custom extensions.

Backwards Compatibility

To bridge the gap until IDEs support JUnit 5 natively there is a JUnit 4 Runner that is able to execute tests written for JUnit 5. Use the @RunWith(JUnit5.class) annotation to run test classes and test suites.

Through this runner, is is possible to run JUnit 4 and 5 tests classes side by side. What is certainly out of scope is mixing old and new concepts in a single test, for example having @Rules coexist with @ExtendWith or the like.

Test utilities like Mockito and AssertJ will continue to work with the new version without change. They interact with JUnit by raising an exception which is still considered a test failure, even in JUnit 5 ;-)

Open Test Alliance for the JVM

The JUnit Lambda team has also started the Open Test Alliance for the JVM with the goal to establish a standard that facilitates the interaction between test frameworks, assertion libraries, mock libraries, build tools, and IDEs.

The primary goal is to provide a library that defines a standard set of exceptions to be used by testing frameworks (e.g. JUnit, TestNG, Spock, etc.) as well as assertion libraries. Build tools and IDEs would also benefit in that they could rely on the same set of types regardless of the test framework.

A draft implementation is available in the form of the org.opentest4j library, which is – you guess it – used by JUnit 5.

Outlook

My impression is that basic concepts of the new version are established. Things like @Test, set up and tear down annotations, the concept of a single extension model will probably remain in their current shape.

But many details seems to be unresolved yet, and APIs are likely to change, which I think is quite understandable at this stage in the development cycle. Each part of the API is tagged with an @API annotation that indicates how stable it is.

If this post caught your interest and you may want to browse the documentation for more, there is plenty more to explore, for example:

The first milestone is planned to be due by the end of Q1 2016. A tentative list of items to be addressed in this release is available here.

Rüdiger Herrmann
Follow me
Latest posts by Rüdiger Herrmann (see all)

3 Comments so far:

  1. Reinier says:

    Great post. I’m learning in deep JUnit4 and I’m glad to listen about all theses changes in JUnit5

  2. John says:

    Having used JUnit a lot through my practice, I struggle to see what is it so useful that this update brings? If one asks my humble opinion – JUnit 4 works just well for 99%, having the remainder covered by the custom runners. What I really miss in JUnit is the AssertJ-style assertions for collections. And ironically, we’re not getting them with this update.

    • Rüdiger Herrmann says:

      Thanks, John, for sharing your insights. I agree that JUnit 4 is very useful and there is not much missing. However, I find JUnit 5 removes some of the small pain points of JUnit 4. What I tike in particular is that extensions can be applied to single methods where rules always apply to all methods in a test class.

      While I like the AssertJ style very much and use it whenever possible, the assertion style may also be a matter of taste. The good news is that during the development of JUnit 5, the exception types to signal assertion violations were split into a separate library (see https://github.com/ota4j-team/opentest4j). I think this is a good step towards separating assertion libraries from test runners.