JUnit Assertions and Reading Failure Messages

An assertion is only as useful as the message it prints when it fails. This lesson runs each assertion against real bugs and reads what comes back. It uses the Maven project from Your First JUnit 6 Test with Maven and the same mvn alias:

alias mvn='docker run --rm -v "$PWD":/w -v m2:/root/.m2 -w /w maven:3.10.0-eclipse-temurin-21-noble mvn'

assertEquals, assertTrue and assertThrows are covered in Unit Testing with JUnit 5 and Mockito; here the focus is the rest of the Assertions class and what each failure looks like.

One failure hides the next: assertAll

The class under test parses a CSV line into an Order. The spec says ids and statuses are stored in upper case.

// File: src/main/java/Order.java
public record Order(String id, int total, String status) {
    public static Order fromCsv(String line) {
        String[] f = line.split(",");
        return new Order(f[0].trim(), Integer.parseInt(f[1].trim()), f[2].trim());
    }
}

// File: src/test/java/OrderTest.java
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

class OrderTest {
    Order order = Order.fromCsv("a-1, 250, paid");

    @Test
    void oneByOne() {
        assertEquals("A-1", order.id());
        assertEquals(250, order.total());
        assertEquals("PAID", order.status());
    }

    @Test
    void grouped() {
        assertAll("order",
            () -> assertEquals("A-1", order.id()),
            () -> assertEquals(250, order.total()),
            () -> assertEquals("PAID", order.status()));
    }
}

$ mvn -B -ntp test -Dtest=OrderTest | sed -n '/^\[ERROR\] Failures:/,/^\[ERROR\] Tests run:/p'
[ERROR] Failures: 
[ERROR]   OrderTest.grouped:17 order (2 failures)
	org.opentest4j.AssertionFailedError: expected: <A-1> but was: <a-1>
	org.opentest4j.AssertionFailedError: expected: <PAID> but was: <paid>
[ERROR]   OrderTest.oneByOne:10 expected: <A-1> but was: <a-1>
[INFO] 
[ERROR] Tests run: 2, Failures: 2, Errors: 0, Skipped: 0

Two methods check the same three fields against the same buggy parser. oneByOne reports only the id: the first assertEquals threw, so the status check never ran, and you would fix the id, rerun, and only then discover the status bug. grouped wraps the checks in assertAll, which runs every lambda, collects the failures, and throws one MultipleFailuresError headed order (2 failures) with each failure listed under it. The total check passed, so it doesn’t appear. (The sed range prints only Surefire’s failure summary.)

Use assertAll when several assertions describe one result, such as the fields of a parsed record. Keep separate tests when the checks are about different behaviours.

Lists: assertIterableEquals

A tag parser that should lower-case tags and drop duplicates. Comparing its list with the expected one:

// File: src/main/java/Tags.java
import java.util.ArrayList;
import java.util.List;

public class Tags {
    public static List<String> parse(String text) {
        List<String> tags = new ArrayList<>();
        for (String part : text.split(",")) {
            tags.add(part.trim().toLowerCase());
        }
        return tags;
    }
}

// File: src/test/java/TagsTest.java
import org.junit.jupiter.api.Test;
import java.util.List;
import static org.junit.jupiter.api.Assertions.*;

class TagsTest {
    @Test
    void duplicatesAreDropped() {
        assertIterableEquals(List.of("java", "maven", "junit"),
                Tags.parse("Java, Maven, java, JUnit"));
    }
}

$ mvn -B -ntp test -Dtest=TagsTest | grep -E "^\[ERROR\]   "
[ERROR]   TagsTest.duplicatesAreDropped:9 iterable contents differ at index [2], expected: <junit> but was: <java>

assertIterableEquals walks both sequences together and stops at the first difference: index 2, where the expected junit met an actual java. That is the duplicate the parser failed to drop. assertEquals on the two lists would also fail, but its message prints both lists whole and leaves the comparing to you. assertIterableEquals also works across types: an expected List matches an actual Set or any other Iterable with the same elements in the same order.

Text output: assertLinesMatch

A report renderer that includes today’s date, so no fixed string can match its second line.

// File: src/main/java/Report.java
import java.util.List;

public class Report {
    public static List<String> render(Order order) {
        return List.of(
            "ORDER " + order.id(),
            "generated " + java.time.LocalDate.now(),
            "total: " + order.total(),
            "status: " + order.status());
    }
}

// File: src/test/java/ReportTest.java
import org.junit.jupiter.api.Test;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertLinesMatch;

class ReportTest {
    @Test
    void reportLayout() {
        Order order = new Order("A-1", 250, "PAID");
        assertLinesMatch(List.of(
                "ORDER A-1",
                "generated \\d{4}-\\d{2}-\\d{2}",
                "total: 250.00",
                "status: PAID"),
            Report.render(order));
    }
}

$ mvn -B -ntp test -Dtest=ReportTest | sed -n '/<<< FAILURE!$/,/^\tat /p'
[ERROR] ReportTest.reportLayout -- Time elapsed: 0.255 s <<< FAILURE!
org.opentest4j.AssertionFailedError: 
expected line #3 doesn't match actual line #3
	expected: `total: 250.00`
	  actual: `total: 250`
	at org.junit.jupiter.api.Assertions.assertLinesMatch(Assertions.java:1678)

assertLinesMatch compares line by line, and each expected line may be a literal or a regular expression. Line 2, generated \d{4}-\d{2}-\d{2}, matched whatever today’s date is, so the test isn’t tied to the day it runs. Line 3 failed because the expected total: 250.00 is neither equal to total: 250 nor a regex that matches it; the message names the line number and shows both lines. The sed range starts at Surefire’s <<< FAILURE! line and stops at the first stack frame, which drops the rest of the trace.

An expected line of >> skip >> (or >> 3 >> for exactly three) fast-forwards past lines you don’t care about, such as log noise.

Messages: a string, or a supplier

Every assertion takes an optional last argument: a message, either as a String or as a Supplier<String> built only when needed.

// File: src/test/java/MessageTest.java
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

class MessageTest {
    static String describe(Order o) {
        System.out.println("building message for " + o.id());
        return "order " + o.id() + " parsed from CSV";
    }

    @Test
    void passes() {
        Order o = Order.fromCsv("B-2, 90, NEW");
        assertEquals(90, o.total(), () -> describe(o));
    }

    @Test
    void fails() {
        Order o = Order.fromCsv("c-3, 40, new");
        assertEquals("NEW", o.status(), () -> describe(o));
    }
}

$ mvn -B -ntp test -Dtest=MessageTest | grep -E "building|^\[ERROR\]   "
building message for c-3
[ERROR]   MessageTest.fails:20 order c-3 parsed from CSV ==> expected: <NEW> but was: <new>

passes and fails both pass a lambda, but building message for ... was printed only once, for c-3. JUnit calls the supplier only when the assertion fails, so an expensive message (one that formats a large object or queries something) costs nothing on the passing path. A plain String message is built before assertEquals even runs. Either way, the message comes first in the failure line, followed by ==> and JUnit’s own expected: <..> but was: <..>.

The same bug through AssertJ

Many projects use AssertJ instead of JUnit’s Assertions: one entry point, assertThat, followed by methods that read as a sentence. Add it to pom.xml:

<!-- pom.xml: inside <dependencies> -->
    <dependency>
      <groupId>org.assertj</groupId>
      <artifactId>assertj-core</artifactId>
      <version>3.27.7</version>
      <scope>test</scope>
    </dependency>

The same duplicate-tag bug as the TagsTest above, asserted the AssertJ way:

// File: src/test/java/TagsAssertJTest.java
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;

class TagsAssertJTest {
    @Test
    void duplicatesAreDropped() {
        assertThat(Tags.parse("Java, Maven, java, JUnit"))
            .containsExactly("java", "maven", "junit");
    }
}

$ mvn -B -ntp test -Dtest=TagsAssertJTest | sed -n '/<<< FAILURE!$/,/^\tat /p'
[ERROR] TagsAssertJTest.duplicatesAreDropped -- Time elapsed: 0.525 s <<< FAILURE!
org.opentest4j.AssertionFailedError: 

Expecting actual:
  ["java", "maven", "java", "junit"]
to contain exactly (and in same order):
  ["java", "maven", "junit"]
but some elements were not expected:
  ["java"]

	at TagsAssertJTest.duplicatesAreDropped(TagsAssertJTest.java:9)

Compare this with TagsTest’s one line. AssertJ printed the whole actual list, the expected list, and then the diagnosis: every expected element was found, but one "java" was not expected. That points straight at the missing de-duplication, where JUnit’s message pointed at index 2. It runs on the JUnit Platform like any other test: AssertJ only replaces the assertion calls, and its failures are the same AssertionFailedError type, so Surefire and IDEs report them the same way.

AssertJ also has containsExactlyInAnyOrder, extracting(...) for fields of objects, and assertThatThrownBy for exceptions. Pick one style per project; mixing both assertThat and assertEquals imports in one class gets confusing.

Practice

1. Fix Tags.parse so it drops duplicates while keeping first-seen order, then run both tag tests with one -Dtest pattern.

// File: src/main/java/Tags.java
import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;

public class Tags {
    public static List<String> parse(String text) {
        Set<String> tags = new LinkedHashSet<>();
        for (String part : text.split(",")) {
            tags.add(part.trim().toLowerCase());
        }
        return new ArrayList<>(tags);
    }
}

$ mvn -B -ntp test -Dtest='Tags*' | grep -E "Tests run:|BUILD"
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.729 s -- in TagsAssertJTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.022 s -- in TagsTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS

2. Write a ReportSkipTest that checks only the first and last lines of Report.render, using >> skip >> for everything between them.

// File: src/test/java/ReportSkipTest.java
import org.junit.jupiter.api.Test;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertLinesMatch;

class ReportSkipTest {
    @Test
    void firstAndLastLine() {
        Order order = new Order("A-1", 250, "PAID");
        assertLinesMatch(List.of("ORDER A-1", ">> skip >>", "status: PAID"),
            Report.render(order));
    }
}

$ mvn -B -ntp test -Dtest=ReportSkipTest | grep -E "Tests run:|BUILD"
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.300 s -- in ReportSkipTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS

3. Check the parsed Order’s id and status with AssertJ’s extracting, and compare its failure message with the assertAll one.

// File: src/test/java/OrderAssertJTest.java
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;

class OrderAssertJTest {
    @Test
    void idAndStatusAreUpperCase() {
        Order order = Order.fromCsv("a-1, 250, paid");
        assertThat(order)
            .extracting(Order::id, Order::status)
            .containsExactly("A-1", "PAID");
    }
}

$ mvn -B -ntp test -Dtest=OrderAssertJTest | sed -n '/<<< FAILURE!$/,/^\tat /p'
[ERROR] OrderAssertJTest.idAndStatusAreUpperCase -- Time elapsed: 0.779 s <<< FAILURE!
org.opentest4j.AssertionFailedError: 

Expecting actual:
  ["a-1", "paid"]
to contain exactly (and in same order):
  ["A-1", "PAID"]
but some elements were not found:
  ["A-1", "PAID"]
and others were not expected:
  ["a-1", "paid"]

	at OrderAssertJTest.idAndStatusAreUpperCase(OrderAssertJTest.java:11)