Sunday 14 October 2012

Handling multiple failures in single test in Junit 4


In my last post we have seen handling multiple assertion failures in single test using junit 3.x. In this post we’ll see how to overcome the same problem but in junit 4 style.

Junit 4 addresses the problem of handling multiple assertion failures pretty well. It has an ErrorCollector class that collects all failures in a test case and reports them at the end of test case. The trick lies in usage of error collector which is also not much difficult.
All you have to do is follow the below mentioned changes and update your base class accordingly.

  1. Let’s assume you have a Base class which is extended by all your test classes. In my example it’s the TestBase.java. In the base class create an ErrorCollector object and (most important) add @Rule annotation to it.
  2. Now create a Boolean variable “fatal” and add setter and getter methods for it. This will help you specify which test cases you want to make fatal and which you don’t.
  3. Create an assertEquals(String, Object, Object) method with public access. We will be using this method in all our test classes that will extend the TestBase.java. This method will be the decision maker on how the test execution should go ahead (whether fatal or non-fatal).

Your code should look like the one mentioned below:

public class TestBase {

      @Rule
      public ErrorCollector collector = new ErrorCollector();
     
      private boolean fatal;
     
      public TestBase() {
            fatal=true;
      }
     
      public void assertEquals(String msg, Object expected, Object actual) {
            if(getFatal()) {
                  Assert.assertEquals(msg,expected, actual);
            } else {
                  collector.checkThat(msg, actual, CoreMatchers.equalTo(expected));
            }
      }
     
      public void setFatal(boolean fatalFlag) {
            fatal = fatalFlag;
      }
     
      public boolean getFatal() {
            return fatal;
      }
     
}


That’s all!!!
You are done with all the implementation part. Now, all you have to do is to use it, and that is demonstrated below. The test class TestSample.java extends the TestBase.java to make use of the code implemented in it.

public class TestSample extends TestBase{
     
      @Test
      public void one() {
            assertEquals("Comparing 1 and 2", 1, 2);
            assertEquals("Comparing A and B", "A", "B");
            assertEquals("Comparing 10.2 and 10.2", 10.2, 10.2);
      }
     
      @Test
      public void two() {
            setFatal(false);
            assertEquals("Comparing 1 and 2", 1, 2);
            assertEquals("Comparing A and B", "A", "B");
            assertEquals("Comparing 10.2 and 10.2", 10.2, 10.2);
      }
}


The output will be: Runs 2; Failures:3; Error:0
  1. For test case “one”:
    java.lang.AssertionError: Comparing 1 and 2 expected:<1> but was:<2>

  1. For test case “two”:
    java.lang.AssertionError: Comparing 1 and 2
    Expected: <1> 
         got: <2>
java.lang.AssertionError: Comparing A and B
Expected: "A"
     got: "B"

As you can see that for test case “one” which is by default fatal (default junit behabiour), the test case exited after the first assertion failure.
Whereas, for test case “two” which is explicitly marked as non-fatal by setting the fatal flag to false, all the assertions within the test case were executed even though the first assertion failed.

So, by doing few changes in your base class of your test framework, you can add an important feature of handling multiple assertion failures to your framework.
That’s all for this post.
I’ll keep posting up some interesting junit problems on this blog, as and when I encounter (and solve) them.
Feel free to comment if you like the post or have any good suggestions.

1 comment: