Getting JUnit Test Names Right

Home  >>  JUnit  >>  Getting JUnit Test Names Right

Getting JUnit Test Names Right

Finding good names is one of the challenges of crafting software. And you need to find them all the time and for everything – classes, methods, variables, just to name a few. But what makes a name a good name? To quote Uncle Bob: ‘Three things: Readability, readability, and readability!’ Which he defines later one by clarity, simplicity, and density of expression1.

Though this makes sense to me, I watch myself struggling in particular with test method naming a bit. To better understand of what I am talking about, one needs to know that I write my code test driven. And doing this for a while I changed my focus of work gradually from the unit under test more to the test itself. This is probably because I like to think of a test case as a living specification and quality assurance in one piece and hence that it is vitally important2.

So whenever a test breaks, ideally I would be able to recognize at a glance what specification was broken and why. And the best way to achieve this seems to be by finding an expressive test name, because this is the first information displayed in the reporting view:

specification

Seen from this angle I am not always happy with what shows up in this view and so I spent a bit of time on research to see what school of thought might be helpful. Unfortunately, most of the results I found were somewhat dated and – less surprising – the opinions on this topic are divided. This post represents my reflections based on those findings and a bit of personal experience.

 Testing with JUnit

Testing with JUnit Book

Testing with JUnit is one of the most valuable skills a Java developer can learn. No matter what your specific background, whether you’re simply interested in building up a safety net to reduce regressions of your desktop application or in improving your server-side reliability based on robust and reusable components, unit testing is the way to go.

Frank has written a book that gives a profound entry point in the essentials of testing with JUnit and prepares you for test-related daily work challenges.

  Get It Now! 

Tests per Method or Behavior Related JUnit Test Names?

In its pure form, the tests per method approach is often provided by tools that e.g. generate a single test stub after the fact. In case you have a class Foo with the method bar the generated method would be called testBar. I was always skeptical about the usefulness of such a development style or naming convention and would have argued like this quote from an old JavaRanch thread: ‘you shouldn’t think about it as testing methods at all, you should think about it as testing behavior of the class. Consequently, I like my test method names to communicate the expected behavior’3.

Interestingly enough I am about to change my opinion a bit on that one. The idea of communicating the ‘behavior’ as stated above requires finding a concise name that expresses this ‘behavior’ comprehensively. But then the term behavior implies a transition from one state to another conducted by an action or denoted in BDD terms, for example, a Given-When-Then pattern. Honestly, I do not think that it is, in general, a good idea to put all this information in a single name4:

@Test
public void
  givenIsVisibleAndEnabledWhenClickThenListenerIsNotified() {}
@Test
public void
  givenIsVisibleAndNotEnabledWhenClickThenListenerIsNotNotified() {}
@Test
public void
  givenIsNotVisibleAndEnabledWhenClickThenListenerIsNotNotified() {}

Maybe its just a question of taste but from my experience, this approach often lacks readability due to the absence of simplicity and/or clarity no matter what kind of format style I chose. Furthermore, such overloaded names tend to have the same problem as comments – the names get easily out of date as the content evolves5. Because of this, I would rather like to go with the BUILD-OPERATE-CHECK6 pattern instead. This would allow to split up the phases into separate sub-method names placed within a single test:

@Test
public void testNameHasStillToBeFound() {
  // do what is needed to match precondition
  givenIsVisibleAndEnabled();

  // execute the transition
  whenClick();

  // verify the expected outcome
  thenListenerIsNotified();
}

Unfortunately, this leads us to where we started. But if you take a closer look at the examples above, all the methods group around a common denominator. They all belong to the same action that fires the transition. In our case the click event. Considering that from the development process point of view I regard a test case more important than the unit under test, one could interpret this as a sign to reflect the action by an appropriate method name in the unit under development7.

See also  Testing with JUnit Book Announcement

So for the sake of example assume we have a ClickAction that wraps around a UI control. And introducing a method called ClickAction#execute() might seem appropriate to us, given the situation above. As simplicity matters we could use that name also for the test method that represents the transition from the default state of the ClickAction – control construct via ClickAction#execute():


class ClickActionTest {

  @Test
  public void execute() {
    Control control = mock( Control.class );
    ClickAction clickAction = new ClickAction( control );
  
    clickAction.execute();

    verify( control ).notifyListeners(...)
  }
}

To keep things simple the next test name may mention only the state information that is important as it differs from the default and leads to another outcome:

class ClickActionTest {

  [...]

  @Test
  public void executeOnDisabledControl() {
    Control control = mock( Control.class );
    when( control.isEnabled() ).thenReturn( false );
    ClickAction clickAction = new ClickAction( control );
  
    clickAction.execute();

    verify( control, never() ).notifyListeners(...)
  }

  @Test
  public void executeOnInvisibleControl() {
  [...]
}

As you can see, this approach results in a set of test names that technically spoken represents a variety of the ‘tests per method’ pattern – but not for completely bad reasons as I think. Given the context I consider this naming pattern is simple, clear and expressive up to one point:

The expected test outcome is still not mentioned at all. On first glance, this looks unsatisfactory, but from my current point of view, I am willing to accept this as a sound trade-off. Especially as the cause for a failing test is usually also shown in the JUnit reporting view. Because of this, that problem can be handled by providing meaningful test failures8.

Update 2015/10/01: During the work on my book Testing with JUnit I refined and elaborated my thoughts on this topic a bit. Now, I determine behavior precisely as the outcome of a component’s functionality applied under given circumstances or preconditions. This matches self-evidently the Structure of a Well Written Test (setup the preconditions, execute the functionality, and verify the outcome), since we intend to test behavior. Given these terms, an expressive and unique test name can be derived from its functionality and the given preconditions, which exactly leads to the naming pattern I’ve described with the last listing.

Conclusion

Actually, I am using the test naming pattern described above for some time now. So far it works out not too bad. In particular, when working with pretty small units as I usually do, there is little room for misinterpretation. However, this approach does not match all cases and sometimes it simply feels better and is still readable enough to mention the outcome. I will not be harping on about principles here and maybe I am getting it all wrong. So I would be happy for any pointers to more elaborated approaches that you might be aware of to broaden my point of view.

See also  OSGi Testsuite: Introducing Classname Filters


  1. Robert C. Martin about clean tests, Clean Code, Chapter 9 Unit Tests
  2. What would be worse: losing the unit under test or the test case? With a good test case restoring the unit should be most of the time straightforward, however vice versa you could easily miss out one of the corner cases that were specified in the lost test case
  3. Naming convention for methods with JUnit, Naming convention for methods with JUnit
  4. To prevent misunderstandings: BDD does nothing the like and comes with its own testing framework. I just mentioned it here as the term ‘behavior’ seems to suggest it and the term ‘givenWhenThen’ floats around in many discussions about test names. However you find actually proposals like Roy Osherove’s naming conventions labelled ‘UnitOfWork_StateUnderTest_ExpectedBehavior’ that still seem to be well accepted albeit the post has seen most of the days of the last decade
  5. A very popular naming pattern looks in principle like this: ‘shouldExpectedBehaviorWhenUnitOfWorkOnStateUnderTest’. However, I don’t consider this an improvement either, since it lacks readability for the same reasons.
  6. Robert C. Martin, Clean Code, Chapter 9, Clean Tests
  7. Or even to extract the whole functionality into a separate class. But this case is described in my post More Units with MoreUnit
  8. Which is probably a topic of its own and as I have to come to an end I leave it at that way :D
Frank Appel
Follow me
Latest posts by Frank Appel (see all)

9 Comments so far:

  1. Stefan Röck says:

    No post without a bunch of footers :-)
    I fully agree, naming is very important and I remember the long discussions we had in the past… I still prefer the loquacious method name approach, e.g. persistUserWithLoginNameButWithoutEmail(). Sticking with the concrete method names of the class under test potentially leads to some refactoring effort if you change the persist() method name to save() or similar. Hence I find some level of abstraction in the test method names very useful. And as screen sizes seem to be increasing more in the horizontal than in the vertical I don’t expect to run into problems in the near future anyway :-)
    Additionally, message texts in JUnit assertions can be very helpful to provide additional information for failing tests. Something like “User doesn’t seem to be persisted because id is still null” is definitely more meaningful than “Assertion failed. Expected: True, Was: False”.

    • Frank Appel says:

      Hi Stefan, long time no see ;-) So you prefer to spend the time to find two names – one for the test and one for the method under test? Well, being somewhat ungifted I am most of the time happy if I manage to find one… Just kidding ;-) . But regarding the renaming problem, there is a tooling solution I mentioned in one of the footers, called MoreUnit.

      One question regarding the persistUserWithLoginNameButWithoutEmail() example: is ‘UserWithLoginNameButWithoutEmail’ precondition, postcondition – or both? Or is it persistUser-WithLogin…? Sorry, but I am bad in reading these camelCase sentences… know what I mean?

  2. Frank Neblung says:

    My testing efforts are mainly influenced by The Art of Unit Testing and Writing Maintainable Automated Acceptance Tests. From the former I like the test method naming pattern, from the latter the advise to keep your tests free of incidental details. In my opinion Mockito is an incidental detail which one should avoid to repeat.

    I would have written your last test probably like that:

    @Test
    public void click_whenDisabled_shouldNotNotify() {
    enabled = false;

    click();

    assertThat(notified).isFalse();
    }

    At first it might look like an odd coding practice.
    Constructing the object under test goes from setUp() to actionUnderTest().
    But the test communicates the distilled intention, and the intention is reflected by the test’s name.
    The test methods itself won’t be touched, even if you decide to replace Mockito by self shunt or hand rolled mocks.

  3. […] => Getting JUnit Test Names Right […]

  4. Adam says:

    I’ve independently arrived at exactly the same style as you from finding full BDD style names are just too long and add no value if they just paraphrase the given/when/then steps they call. We just need a catchy name for this approach… :)

    • Frank Appel says:

      Hi Adam, glad you agree. A catchy name would be great, but getting catchy names right is also somewhat difficult ;-)

  5. Hi, Frank!

    Excellent post! Though I’ve somewhat stumbled on the concept of best practices recently, a little research has shown the topic is way broader than I initially thought.

    I used to name my tests as “testMethodSuccess” and “testMethodFailure”. Now I try to go like “shouldCallDecoratedImport” or “shouldNotCallDecoratedImportIfFileCantBeLocked”. Ok, not perfect, I agree. It’s a middle ground between your article closing approach and the initial Given-When-Then.

    I guess it started when I wrote in a Facebook group that your application is as good as your requirements. Somehow that stuck with me and Ive tried to bring that to testing. So far, I’m happy with the results.

    Last month or so I came across a post that seemed to go the BDD direction when discussing testing best practices. Now, in trying to recover that post, I found yours. And I’m glad I did.

    Thank you so much. Until next time!

    • Frank Appel says:

      Hi Luiz, glad you liked the post. Indeed, finding a good test naming convention is more difficult than it seems on first sight. I‘m happy if my thoughts could help you a bit finding yours.