Sep 18, 2013

The Benefits of Using assertThat over other Assert Methods in Unit Tests

Introduction

When Junit 4 was released a number of things changed, most notably the use of the test annotation so that there is no longer a need to extend TestCase and name methods starting with “test”.  There were several other improvements as well. With the released of version 4.4 one improvement that often gets overlooked is the addition of the the assertThat method.  This new method incorporates the use of the hamcrest library and is a much improved way to write assertions. It uses what’s called matchers which are self-contained classes which have static methods that get used with the assertThat method. These static methods can be chained which gives a lot of flexibility over using the old assert methods.  Below we’ll go over some of the benefits of using assertThat over the old methods.

Readability

The first benefit is that assertThat is more readable than the other assert methods.  For example take a look at the following assertEquals method:

assertEquals(expected, actual);

We’ve all seen this many times, the expected value is passed in first and actual second, but it’s very cryptic until you get used to it. Now let’s look at the same assertion with assertThat:

assertThat(actual, is(equalTo(expected)));

The first thing to notice is that it’s the other way around (actual first, expected second), which is the biggest hurdle to get over.  It also reads more like a sentence: “Assert that the actual value is equal to the expected value.”  As another, better example of readability, compare how to check for not equals, first the old way:

assertFalse(expected.equals(actual));

Since there is no “assertNotEquals” (unless it’s custom coded) we have to use assertFalse and do an equals on the two variables. Here’s the much more readable new way with assertThat:

assertThat(actual, is(not(equalTo(expected))));

What’s cool about the “not” method is that it can surround any other method, which makes it a negate for any matcher.  Also as seen above, the matcher methods can be chained to create any number of possible assertions.  Another cool thing is that there’s an equivalent short-hand version of the above equality methods which saves on typing:

assertThat(actual, is(expected));
assertThat(actual, is(not(expected)));

Better Failure Messages

Another benefit to using assertThat are much better error messages.  Let’s look at a common example of the use of assertTrue and it’s failure message.

assertTrue(expected.contains(actual));
java.lang.AssertionError at ...

First of all there’s no “assertStringContains” method (unless it’s custom coded once again), so we have to use assertTrue to test this.  The problem here is that the assertion error doesn’t report the values for expected and actual.  Granted the expected value is easy to find, but a debugging session will probably be needed to figure out the actual value. Now let’s look at the same test using assertThat:

assertThat(actual, containsString(expected));
java.lang.AssertionError:
Expected: a string containing "abc"
got: "def"

As you can see, both values are returned in the error message.  This is much better since in many cases a developer can look at the error message and figure out right away what they did wrong rather than having to debug to find the answer.  That saves developer time and hassle.

Type Safety

Another benefit to assertThat is it’s generic and type-safe.  Look at the below example of assertEquals:

assertEquals("abc", 123); //compiles, but fails

The assertThat method does not allow this as it typed, so the following would not compile:

assertThat(123, is("abc")); //does not compile

This is very handy as it does not allow comparing of different types.  The are some drawbacks to this, but overall it’s a positive change.

Flexibility

As mentioned in the introduction, matchers can be chained, such as is(not()) for example. In addition to that, hamcrest offers logical matchers, “not” being one of them. There are two more logical matchers worth noting; “anyOf” and “allOf” which will be shown below:

assertThat("test", anyOf(is("test2"), containsString("te")));

The following passing example shows how anyOf works. Similar to a logical “or”, it passes if any or all of the matchers given to it also pass. Only if none of them pass would the following error message be given:

assertThat("test", anyOf(is("test2"), containsString("ca")));
java.lang.AssertionError: 
Expected: (is "test2" or a string containing "ca")
     got: "test"

The “allOf” matcher works the same but is a logical “and” so all of the matchars given to it would have to pass before it did. The “not” matcher also fits into the logical category.

Portability

An additional benefit to hamcrest library is its portability.   It can be used with both JUnit and TestNG.  JUnit has the assertThat method, but hamcrest has its own assertThat method that does the same thing.  It can be used in any test framework and it looks the same as the above methods, just a different static import.

Custom Matchers

In the same way that custom assert methods can be written, custom matcher can also be written. Here’s an example of a simple string equals matcher:

import org.hamcrest.Description;
import org.hamcrest.TypeSafeMatcher;

public class TestMatcher extends TypeSafeMatcher<String> {
   private String expected;

   private TestMatcher(String expected) {
      this.expected = expected;
   }

   @Override
   public void describeTo(Description description) {
      description.appendValue(expected);
   }

   @Override
   protected boolean matchesSafely(String item) {
      return item.equals(expected);
   }

   public static TestMatcher myEquals(String expected) {
      return new TestMatcher(expected);
   }
}

Here’s an example of how to use this matcher along with the error message it returns:

assertThat("test", myEquals("test2"));
java.lang.AssertionError: 
Expected: "test2"
     got: "test"

Cross Reference

There is a fair amount of information on the internet about this new matching mechanism, but what appears to be missing is a good cross reference of the old vs the new. I will attempt to create cross reference for many common assert methods. Below that is a list of all the matchers in the hamcrest library as of version 1.3. Any of the below matchers may have an “is” matcher around them for greater readability, but this is optional. (i.e. assertThat(1, is(greaterThan(3))); instead of assertThat(1, greaterThan(3));)

Notes:1) Most of the examples below will be failure examples. 2) The static import for assertThat is: org.junit.Assert.assertThat 3) The following will be need to be added to your classpath as most of the useful matchers are in this dependency: http://mvnrepository.com/artifact/org.hamcrest/hamcrest-library

Old Assert Method Equivalent with assertThat Static Imports Notes
assertEquals(“expected”, “actual”); assertThat(“actual”, is(“expected”)); org.hamcrest.core.Is.is “is” is short hand for is(equalTo())
assertArrayEquals(new String[] {“test1”, “test2”}, new String[] {“test3”, “test4”}); assertThat(new String[] {“test3”, “test4”}, is(new String[] {“test1”, “test2”})); org.hamcrest.core.Is.is The error message looks like this: java.lang.AssertionError:
Expected: is [“test1”, “test2”]
got: [“test3”, “test4”]
assertTrue(value); or assertFalse(value); assertThat(actual, is(true));
assertThat(actual, is(false));
org.hamcrest.core.Is.is The error message looks like this (depending on the values): java.lang.AssertionError:
Expected: is true
got: false
assertNull(value); or assertNotNull(value); assertThat(actual, nullValue());
assertThat(actual, notNullValue());
org.hamcrest.core.IsNull.
notNullValue
org.hamcrest.core.IsNull.
nullValue;
The error message looks like this (depending on the values):
java.lang.AssertionError:
Expected: not null
got: null
Also both matchers can be typed as such:
assertThat(actual, nullValue(String.class)); which means the actual argument must be a string.
assertSame(expected, actual); or assertNotSame(expected, actual); assertThat(actual, sameInstance(expected));
assertThat(actual, not(sameInstance(expected)));
org.hamcrest.core.IsNot.not
org.hamcrest.core.IsSame.
sameInstance
The error message looks like this (depending on the values): java.lang.AssertionError:
Expected: sameInstance(“test”)
got: “test”
using: String actual = new String(“test”);
String expected = new String(“test”);
assertTrue(1 > 3); assertThat(1, greaterThan(3)); org.hamcrest.number.
OrderingComparison.
greaterThan
The error message is similar to the pattern above rather than “java.lang.AssertionError” OrderingComparison also contains: “comparesEqualTo”, “greaterThanOrEqualTo”, “lessThan” and “lessThanOrEqualTo”
assertTrue(“abc”.contains(“d”)); assertThat(“abc”, containsString(“d”)); oorg.hamcrest.core.
StringContains.containsString
The error message is similar to the pattern above.
assertTrue(“abc”.contains(“d”)); assertThat(“abc”, containsString(“d”)); oorg.hamcrest.core.
StringContains.containsString
The error message is similar to the pattern above. See also in the same package: StringStartsWith, StringEndsWith

There are many more that could be posted here, but instead here is a hierarchy of classes to look at. Most of these are self-explanatory, but take a look at them to see if they fit whatever requirements you may have.

  • org.hamcrest.beans
    • HasProperty
    • HasPropertyWithValue
    • SamePropertyValuesAs
  • org.hamcrest.collection
    • IsArray
    • IsArrayContaining
    • IsArrayContainingInAnyOrder
    • IsArrayContainingInOrder
    • IsArrayContainingWithSize
    • IsCollectionWithSize
    • IsEmptyCollection
    • IsEmptyIterable
    • IsIn
    • IsIterableContainingInAnyOrder
    • IsIterableContainingInOrder
    • IsIterableWithSize
    • IsMapContaining
  • org.hamcrest.number
    • BigDecimalCloseTo
    • IsCloseTo
    • OrderingComparison
  • org.hamcrest.object
    • HasToString
    • IsCompatibleType
    • IsEventFrom
  • org.hamcrest.text
    • IsEmptyString
    • IsEqualIgnoringCase
    • IsEqualIgnoringWhiteSpace
    • StringContainsInOrder
  • org.hamcrest.xml
    • HasXPath

Conclusion

As can be seen JUnit’s new assertThat method has much better functionality than the old assert methods. I’ve found it very useful in my coding experiences but it seems very few developers know about it. I hope this post helps enlighten you to what you can do with this new way of writing assertions.

About the Author

Object Partners profile.

One thought on “The Benefits of Using assertThat over other Assert Methods in Unit Tests

  1. chandiprasad says:

    Good one !

  2. CoL says:

    maybe few developers know about it because there’s so much IMPORT’ing to do to use each matcher!

    1. Mark says:

      Agreed! Eclipse cannot even auto- figure it out.

      1. Le Stephane says:

        To enable auto-completion of Hamcrest core matchers imports in Eclipse with Ctrl-Space
        * Press Ctrl-3
        * Sear for **Preferences *Favorites* – Java / Editor / Content Assist**)
        * Click **New Type…**
        * Add **org.hamcrest.CoreMatchers**

  3. Mohammed Hossain says:

    Very good article!

    We’ve switched to using assertThat few years ago and was looking for an article like this to review. Thanks. 🙂

  4. Sonali Mishra says:

    It is a very useful piece of article.

  5. Cyper says:

    Clearly addressed and very useful Thank you!

  6. Bennet Schulz says:

    Very good article about the benefits of hamcrest. Good job!

  7. Naveenkumar says:

    Very nice article.

  8. deepika says:

    Very information article.

  9. shadeven says:

    Thanks for this article. I use this as my cheatsheet when writing tests.

  10. Asma Khan says:

    very useful article.

  11. Dominik says:

    One caveat though, which we just realized:

    In Eclipse with JUnit, when using assertThat(stringResult, equalTo(expectedResult)), there is no way to easily compare the two strings in a comparison view.

    Using assertEquals(expectedResult, stringResult) makes it possible to double click on the org.junit.ComparisonFailure in the JUnit view to get a diff-like viewer.

    1. turberg says:

      I agree, it would be nice if the Eclipse plugin added that so that it worked that way for both methods (assertThat and assertEquals).

  12. Amedee Van Gasse says:

    After reading this, I googled a bit more on assertThat and found AssertJ, which is taking it to the next level: http://joel-costigliola.github.io/assertj/index.html
    What I like most about assertThat (either Hamcrest or AssertJ), is that your asserts are now read as SUBJECT-VERB-OBJECT, while in the old days they were VERB-OBJECT-SUBJECT. For example:

    VERB-OBJECT-SUBJECT: assertEquals(expected, actual);
    OBJECT-VERB-SUBJECT: assertThat(actual).isEqualTo(expected);

    I gave a code example to a non-developer, and she immediately understood what was going on in the second example. It reads more like natural language.

    1. Amedee Van Gasse says:

      sorry, the second line should obviously be:

      SUBJECT-VERB-OBJECT: assertThat(actual).isEqualTo(expected);

  13. Yury Siachko says:

    Thanks for brilliant post!

    Indeed, assertThat() brings more readable failure messages. Even when tests are run in command line via Maven.

    For example, in case of assertEquals() you will get:

    org.junit.ComparisonFailure: expected: but was:

    In case of assertThat() you will get:

    java.lang.AssertionError:
    Expected: is “Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.”
    but: was “-Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.”

    Perhaps, it’s possible to print the whole text in case of assertEquals() by adjusting some Maven’s or JUnit’s settings. But Hamcrest’s assertions print the whole text out-of-the-box.

  14. Gabriel Matossian says:

    I found it very useful, thanks for sharing!

  15. Rudy Vissers says:

    Please correct the following:oorg.hamcrest.core.StringContains.containsString.
    oorg does not exists!

  16. Dilini says:

    Great post Tim! A very comprehensive article on assertThat. This is very useful to me and I refer to it time and time again. I’ve also added a link to this on my blog post https://dilinirajapaksha.github.io/DeveloperJournal/test-driven-development. I absolutely love the table in the Cross Reference section. Thanks for putting this together.

  17. somebody says:

    Nice article…

  18. Cezar Augusto says:

    Nice article, I found it very useful. Thanks for sharing

  19. Manas says:

    So gooodd! Explain is such a simple way.. it helped me to learn in easy way

  20. Trejkaz says:

    And then in JUnit 5 they took assertThat back out again and left in all the legacy-style ones.

    Really hard to tell what the JUnit 5 team were thinking. :/

  21. TP says:

    No, man. No. If I wanted excessive amount of parenthesis I’d go straight to Lisp.

    🙂

  22. Manjusha says:

    good article. But for me it’s giving some type signature error and test fails even it is true. I have added hamcrest-all1.3 jar in my eclipse project

  23. Klitos says:

    Regarding the section on type safety, assertThat(123, is(“abc”)); does not fail to compile.

Leave a Reply to chandiprasad Cancel reply

Your email address will not be published.

Related Blog Posts
Natively Compiled Java on Google App Engine
Google App Engine is a platform-as-a-service product that is marketed as a way to get your applications into the cloud without necessarily knowing all of the infrastructure bits and pieces to do so. Google App […]
Building Better Data Visualization Experiences: Part 2 of 2
If you don't have a Ph.D. in data science, the raw data might be difficult to comprehend. This is where data visualization comes in.
Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll […]
A security model for developers
Software security is more important than ever, but developing secure applications is more confusing than ever. TLS, mTLS, RBAC, SAML, OAUTH, OWASP, GDPR, SASL, RSA, JWT, cookie, attack vector, DDoS, firewall, VPN, security groups, exploit, […]