JUnit Rules

Home  >>  JUnit  >>  JUnit Rules

JUnit Rules

On September 24, 2012, Posted by , In JUnit, By ,,, , With 20 Comments

The first time I stumbled over a JUnit @Rule annotation I was a bit irritated of the concept. Having a public field in a test case seemed somewhat odd and so I was reluctant to use it regularly. But after a while I got used to that and it turned out that rules can ease writing tests in many ways. This post gives a quick introduction of the concept and some short examples of what rules are good for.

What are JUnit Rules?

Let’s start with a look at a JUnit out-of-the-box rule. The TemporaryFolder is a test helper that can be used to create files and folders located under the file system directory for temporary content1. The interesting thing with the TemporaryFolder is that it guarantees to delete its files and folders when the test method finishes2. To work as expected the temporary folder instance must be assigned to an @Rule annotated field that must be public, not static, and a subtype of TestRule:

public class MyTest {
	
  @Rule
  public TemporaryFolder temporaryFolder = new TemporaryFolder();

  @Test
  public void testRun() throws IOException {
    assertTrue( temporaryFolder.newFolder().exists() );
  }
}

How does it work?

Rules provide a possibility to intercept test method calls similar as an AOP framework would do. Comparable to an around advice in AspectJ you can do useful things before and/or after the actual test execution3. Although this sounds complicated it is quite easy to achieve.

 Testing with JUnit

Testing with JUnit Book

Testing with JUnit is one of the most valuable skills a Java developer can learn. No matter what your specific background, whether you’re simply interested in building up a safety net to reduce regressions of your desktop application or in improving your server-side reliability based on robust and reusable components, unit testing is the way to go.

Frank has written a book that gives a profound entry point in the essentials of testing with JUnit and prepares you for test-related daily work challenges.

  Get It Now! 

The API part of a rule definition has to implement TestRule. The only method of this interface called apply returns a Statement. Statements represent – simply spoken – your tests within the JUnit runtime and Statement#evaluate() executes them. Now the basic idea is to provide wrapper extensions of Statement that can do the actual contributions by overriding Statement#evaluate():

public class MyRule implements TestRule {

  @Override
  public Statement apply( Statement base, Description description ) {
    return new MyStatement( base );
  }
}

public class MyStatement extends Statement {

  private final Statement base;

  public MyStatement( Statement base ) {
    this.base = base;
  }

  @Override
  public void evaluate() throws Throwable {
    System.out.println( "before" );
    try {
      base.evaluate();
    } finally {
      System.out.println( "after" );
    }
  }
}

MyStatement is implemented as wrapper that is used in MyRule#apply(Statement,Destination) to wrap the original statement given as argument. It is easy to see that the wrapper overrides Statement#evaluate() to do something before and after the actual evaluation of the test4.

The next snippet shows how MyRule can be used exactly the same way as the TemporaryFolder above:

public class MyTest {

  @Rule
  public MyRule myRule = new MyRule();
	
  @Test
  public void testRun() {
    System.out.println( "during" );
  }
}

Launching the test case leads to the following console output which proves that our example rule works as expected. The test execution gets intercepted and modified by our rule to print ‘before’ and ‘after’ around the ‘during’ of the test:

before
during
after

Now that the basics are understood let’s have a look at slightly more useful things you could do with rules.

See also  Testing with JUnit Book Release

Test Fixtures

Quoted from the according wikipedia section a test fixture ‘is all the things that must be in place in order to run a test and expect a particular outcome. Frequently fixtures are created by handling setUp() and tearDown() events of the unit testing framework’.

With JUnit this often looks somewhat like this:

public class MyTest {

  private MyFixture myFixture;

  @Test
  public void testRun1() {
    myFixture.configure1();
    // do some testing here
  }
	
  @Test
  public void testRun2() {
    myFixture.configure2();
    // do some testing here
  }
	
  @Before
  public void setUp() {
    myFixture = new MyFixture();
  }

  @After
  public void tearDown() {
    myFixture.dispose();
  }
}

Consider you use a particular fixture the way shown above in many of your tests. In that case it could be nice to get rid of the setUp() and tearDown() methods. Given the sections above we now know that this can be done by changing MyFixture to implement TestRule. An appropriate Statement implementation would have to ensure that it calls MyFixture#dispose() and could look like this:

public class MyFixtureStatement extends Statement {

  private final Statement base;
  private final MyFixture fixture;

  public MyFixtureStatement( Statement base, MyFixture fixture ) {
    this.base = base;
    this.fixture = fixture;
  }

  @Override
  public void evaluate() throws Throwable {
    try {
      base.evaluate();
    } finally {
      fixture.dispose();
    }
  }
}

With this in place the test above can be rewritten as:

public class MyTest {

  @Rule
  public MyFixture myFixture = new MyFixture();

  @Test
  public void testRun1() {
    myFixture.configure1();
    // do some testing here
  }
	
  @Test
  public void testRun2() {
    myFixture.configure2();
    // do some testing here
  }
}

I come to appreciate the more compact form of writing tests using rules in a lot of cases, but surely this is also a question of taste and what you consider better to read5.

Fixture Configuration with Method Annotations

So far I have silently ignored the Description argument of TestRule#apply(Statement,Description). In general a Description describes a test which is about to run or has been run. But it also allows access to some reflective information about the underlying java method. Among others it is possible to read the annotations attached to such a method. This enables us to combine rules with method annotations for convenience configuration of a TestRule.

Consider this annotation type:

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD})
public @interface Configuration {
  String value();
}

Combined with the following snippet inside MyFixture#apply(Statement,Destination) that reads the configuration value annotated to a certain test method…

Configuration annotation
  = description.getAnnotation( Configuration.class );
String value = annotation.value();
// do something useful with value

… the test case above that demonstrates the usage of the MyFixture rule can be rewritten to:

public class MyTest {

  @Rule
  public MyFixture myFixture = new MyFixture();

  @Test
  @Configuration( value = "configuration1" )
  public void testRun1() {
    // do some testing here
  }
	
  @Test
  @Configuration( value = "configuration2" )
  public void testRun2() {
    // do some testing here
  }
}

Of course there are limitations to the latter approach due to the fact that annotations only allow Enums, Classes or String literals as parameters. But there are use cases where this is completely sufficient. A nice example using rules combined with method annotations is provided by the restfuse library. If you are interested in a real world example you should have a look at the library’s implementation of the Destination rule6.

Comming to the end the only thing left to say is that I would love to hear from you about other useful examples of JUnit rules you might use to ease your daily testing work :-)

It would be remiss not to tell you that chapter 6, Reducing Boilerplate with JUnit Rules, of my book Testing with JUnit is available as a free reading sample at https://www.packtpub.com/packtlib/book/Application%20Development/9781782166603/6. So, in case you're not tired of my scribblings yet, boldly go ahead and take the opportunity to delve deeper into the world of JUnit rules...

Update 2014/09/01: Rule Ordering:

Sometimes you use more than one rule in a test case. If for some reason you have to define an order of how these rules are nested at runtime, JUnit offers the type RuleChain. This class provides the methods outerRule and around to configure an execution chain of TestRules in a fluent interface like style. Please have a look at the API for more information.

See also  JUnit in a Nutshell: Test Structure

Update 2014/09/01: Rule Resources:

1) JUnit provides a set of rule implementations that are described in the Rules section of the tool’s online documentation.
2) Thanks to Benny for hinting me at the System Rules library, which provides a collection of rules for testing code that uses java.lang.System.
3) Last but not least we have described a couple of custom rules on our blog.

Update 2014/09/23: JUnit in a Nutshell:

I integrated this post as the fifth chapter into my multi-part tutorial about JUnit testing essentials. So if you are interested in further reading on that topic, here is the chapter navigation:

Chapter Navigation
 Prev | Table of Content | Next 
Title Image: © Depositphotos.com/alexmillos
  1. The directory which is in general returned by System.getProperty( "java.io.tmpdir" );
  2. Looking at the implementation of TemporaryFolder I must note that it does not check if the deletion of a file is successful. This might be a weak point in case of open file handles
  3. And for what it’s worth you even could replace the complete test method by something else
  4. The delegation to the wrapped statement is put into a try...finally block to ensure that the functionality after the test gets executed, even if the test would fail. In that case an AssertionError would be thrown and all statements that are not in the finally block would be skipped
  5. You probably noted that the TemporaryFolder example at the beginning is also nothing else but a fixture use case
  6. Note that restfuse’s Destination class implements MethodRule instead of TestRule. This post is based on the latest JUnit version where MethodRule has been marked as @Deprecated. TestRule is the replacement for MethodRule. But given the knowledge of this post it should be nevertheless easily possible to understand the implementation
Frank Appel
Follow me
Latest posts by Frank Appel (see all)

20 Comments so far:

  1. Zemian Deng says:

    Hello Frank. Nice post!

    So were you able to find out what’s the reason behind why JUnit requires @Rule field to be declare public? What will it happen if you mark it private?

    Thanks,
    Zemian

    • Frank Appel says:

      Zemian,

      glad you like the post :-)

      If you annotate a private field with @Rule the JUnit runtime throws an java.lang.Exception that states ‘The @Rule ‘myRule’ must be public’. I guess this might have s.th. to do with the reflective nature of the approach.

      To gain access to a private/default/protected field from a class outside of MyTest’s package you would have to use the Field#setAccessible(boolean), which may not be possible in every case due to policy restrictions. However the latter is probably only rarely the case in testing environment.

      On the other hand it could be problem if you extend your test class from another class that uses @Rule. There would not be an API notion that tells you that using the super class manipulates your tests. And this could lead to some unpredictable behavior of your tests.

      But the latter is just wild guessing…

  2. Big thanks for the article
    You have explained JUnit @Rule very clear with good code snippets
    Respect

  3. Frank Gerhardt says:

    Nice post!

    How would you handle a resource which is an external service if that service is expensive to create?

    I thought I would cache the service statically in my rule. It would work but it feels a bit weird.

    • Frank Appel says:

      Frank,

      I know what you mean. JUnit tests should not take long for execution, because you have to run them often – at least I have to ;-). Dealing with external services can spoil that, since you need time for connection establishment, authentication or whatsoever. But in that case the test tends to be actually more a kind of integration test.

      In general I prefer having both: a fast JUnit Test which uses mocks to simulate the external service and an integration test that takes some time but assures that the real thing works as expected. The first one gets executed within a fast plain JUnit test suite, which I run quite often while working. The second one runs within an “occasionally” executed integration test suite (e.g. before pushing code to origin/master or during a server side build process).

      Buffering an external resource globally feels a bit weird because this might lead to side effects and/or interdependencies of several test methods based on that global reference. It is a bit like this: testA writes a record foo to some external service and testB works only if the record foo exists…

      Said all this I think it depends on how expensive it is to create the external resource. If the tests gets executed by an server side integration build only, you might be able to afford resource creation per test method. If it blocks your daily work, buffering might be the better solution, but you have to take more care writing the tests…

  4. Dan says:

    I am new to @Rule annotation which seems to be introduced in JUnit 4.7. But when researching on the topic of testing exceptions, I also came across this guy who is saying that his magic test library can allow for testing exceptions in pre 4.7 versions too. See the blog post: magic test, specially the table shown by him on that page. Would like to know your comments on the same. Also since I am beginner on JUnit, I also see that @Rule annotation can be applied for timeout of unit tests. So it means that @Rule is a generic annotation and we can use it for multiple purposes. right!

    • Frank Appel says:

      1) Maybe I miss the point with magic test, but the example test method does not seems to be not very expressive. Would I grasp at once what the test is all about if I read ConfigTest#readValue()? I usualy use one test method per check with the expected parameter of the @Test annotation.

      2) You are right, @Rule can be used for multiple purposes.

  5. Harshad says:

    Aweseome post.
    Thank you.

  6. Kim says:

    Thank you so much for the explanation! I was looking for a way to create directories and reliably delete them, but not for all of my tests. I found your article and within an hour, I had all of my tests written and working! I had never heard about Rules before. Good job!

  7. Nitz says:

    Hi frank, i have a scenario where i need to test a java REST client. I need to set up a rule that the tests should be executed only if the REST server is up and running. Could you guide me to solve this?

    • Frank Appel says:

      Not sure if this is a good approach, but you may consider to use a ping or health check service to ensure that the server is up and running. The check could take place before the rule statement actually delegates test execution to the original statement. If the check is not successful the latter can be skipped. Configuration data like server ip etc could be set as constructor params of the Rule.

    • Rüdiger Herrmann says:

      I share Frank’s doubts, Your REST server might be down unwantedly and none of your tests will be silently skipped – sounds scary to me.

      If you still want to skip the tests, you may want to have a look at the Conditional Ignore Rule.

  8. Stefan Liebig says:

    The @Rule is very powerful but also alluring.
    There is a micro benchmark tool JUnitBenchmarks (http://labs.carrotsearch.com/junit-benchmarks.html) which uses @Rules to do there measurements. We use it to track the performance of certain functionality over different versions of a product. From one version to the other we measured a huge regression which we could not explain. It turned out that the cause was in the @Before of the test. There was code that now took longer. And unfortunately, the time in @Before and @After is also “counted”.
    See the issue: http://issues.carrot2.org/browse/JUNITBENCH-54

    • Frank Appel says:

      Hi Stefan,

      I had a look at the benchmark tool and read the issue. My first impression is, that TestRule might not be the best choice for benchmark testing after all. Although it seems reasonable or alluring (as you called it) on first sight.

      But setup and teardown are per definition part of a unit test (see Test Structure). And despite of being an expert on benchmark testing I agree with Christian Campo’s assessment, that these phases should not add up to the measured execution time of a certain functionality.

      Because of this I tend to the opinion, that a particular TestRunner might have been a better architectural choice. And this would probably not lack the problem of the workaround you have mentionend. So I cannot follow David Weiss’s argument on the disadvantage of using @RunWith instead of @Rule, except that runners cannot be combined as far as I understand it – but one death you have to die…

      So alltogether it is important to understand how @Rule works and what are its implications to see if it fits for a certain task.

      Frank

  9. Achim says:

    Hi Frank, thx for the good explanation of TestRules…
    i got my rule to run – which runs the test multiple times… but I’m failing
    to understand how to set the Description… Running the test in Eclipse
    only shows the original test method name. Is there any known way to
    set the Description of the test -other than writing my own TestRunner?

    • Frank Appel says:

      Hi Achim, glad you like the post. AFAIK, there is no API to change the description with Rules. This is because the description is created by the runner with factory methods, injected to the method rule and the description API provides no setter to change the displayName field, for example.

  10. Achim says:

    Thx a lot – even though it was the answer I was expecting . At least I know now that I was not unable to find the right search term in Google … 8-)))
    Keep up the good work!