JUnit Timeouts and Parallel Test Execution

A test that hangs blocks the whole build, and a suite that runs one test at a time wastes the cores it runs on. JUnit 6 handles both: timeouts that fail a test which runs too long, and parallel execution that runs tests at the same time. Parallelism also exposes tests that share state, and this lesson shows that bug too. It uses the project and mvn alias from Your First JUnit 6 Test with Maven:

alias mvn='docker run --rm -v "$PWD":/w -v m2:/root/.m2 -w /w maven:3.10.0-eclipse-temurin-21-noble mvn'

Timings in this lesson’s output (Time elapsed, returned after) vary from run to run and machine to machine; the explanations rely only on their rough size.

@Timeout fails a test that runs too long

@Timeout(1) gives a test one second (the unit defaults to seconds). This one sleeps for three.

// File: src/test/java/ImportTest.java
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Timeout;

class ImportTest {
    @Test
    @Timeout(1)
    void importsQuickly() throws InterruptedException {
        Thread.sleep(3000);
        System.out.println("import done");
    }
}

$ mvn -B -ntp test -Dtest=ImportTest | grep -E "import done|Tests run:|^\[ERROR\]   "
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.225 s <<< FAILURE! -- in ImportTest
[ERROR]   ImportTest.importsQuickly » Timeout importsQuickly() timed out after 1 second
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

import done never printed, and the class finished in about a second, not three. When the time runs out, JUnit interrupts the test thread; Thread.sleep reacts to the interrupt by throwing, and JUnit reports a TimeoutException: importsQuickly() timed out after 1 second. Surefire counts it under Errors, not Failures, because it is an exception rather than a failed assertion.

Interrupting only works if the code checks for it. A busy loop that never sleeps, waits or checks Thread.interrupted() keeps running, and JUnit then reports the timeout only when the test finally ends. For code that can’t be interrupted, add threadMode = SEPARATE_THREAD to @Timeout, which runs the test on another thread and stops waiting at the deadline.

assertTimeout waits, assertTimeoutPreemptively doesn’t

The two assertion forms put a timeout around one block of code instead of the whole test. Both get the same 3-second task and a 1-second limit.

// File: src/test/java/TimeoutStyleTest.java
import org.junit.jupiter.api.Test;
import java.time.Duration;
import static org.junit.jupiter.api.Assertions.*;

class TimeoutStyleTest {
    static void slowTask(String who) throws InterruptedException {
        Thread.sleep(3000);
        System.out.println(who + ": task finished");
    }

    @Test void waits() {
        long start = System.nanoTime();
        try { assertTimeout(Duration.ofSeconds(1), () -> slowTask("waits")); }
        finally { System.out.printf("waits: returned after %.0f s%n", (System.nanoTime() - start) / 1e9); }
    }

    @Test void aborts() {
        long start = System.nanoTime();
        try { assertTimeoutPreemptively(Duration.ofSeconds(1), () -> slowTask("aborts")); }
        finally { System.out.printf("aborts: returned after %.0f s%n", (System.nanoTime() - start) / 1e9); }
    }
}

$ mvn -B -ntp test -Dtest=TimeoutStyleTest | grep -E "^(waits|aborts)|^\[ERROR\]   " | sed -E 's/by [0-9]+ ms/by N ms/'
aborts: returned after 1 s
waits: task finished
waits: returned after 3 s
[ERROR]   TimeoutStyleTest.aborts:20 execution timed out after 1000 ms
[ERROR]   TimeoutStyleTest.waits:14 execution exceeded timeout of 1000 ms by N ms

assertTimeout ran the task to the end on the test’s own thread: waits: task finished printed, the call returned after 3 seconds, and only then did it fail with execution exceeded timeout of 1000 ms by N ms. (The overrun is around 2000 ms and changes every run, so the sed replaces it with N.) It tells you the code is too slow but doesn’t save any time.

assertTimeoutPreemptively ran the task on a separate thread and gave up after 1 second: aborts: task finished never printed, and the failure is execution timed out after 1000 ms. The catch is that separate thread. Anything bound to the test thread through a ThreadLocal, such as a Spring transaction, isn’t visible inside the block, so preemptive timeouts can break code that works fine elsewhere.

Running tests in parallel

Four tests that each take a second and report which thread ran them.

// File: src/test/java/ParallelTest.java
import org.junit.jupiter.api.Test;

class ParallelTest {
    static void work(String name) throws InterruptedException {
        Thread.sleep(1000);
        System.out.println(name + " on " + Thread.currentThread().getName());
    }

    @Test void a() throws InterruptedException { work("a"); }
    @Test void b() throws InterruptedException { work("b"); }
    @Test void c() throws InterruptedException { work("c"); }
    @Test void d() throws InterruptedException { work("d"); }
}

$ mvn -B -ntp test -Dtest=ParallelTest | grep -E " on |Tests run:.*-- in"
a on main
b on main
c on main
d on main
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.217 s -- in ParallelTest

By default JUnit runs tests one after another on the main thread, so four one-second tests take just over four seconds. Parallel execution is switched on with configuration parameters in junit-platform.properties:

# File: src/test/resources/junit-platform.properties
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.config.strategy = fixed
junit.jupiter.execution.parallel.config.fixed.parallelism = 4

$ mvn -B -ntp test -Dtest=ParallelTest | grep -E " on |Tests run:.*-- in" | sed -E 's/^[a-d] on //' | sort
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.225 s -- in ParallelTest
ForkJoinPool-1-worker-1
ForkJoinPool-1-worker-2
ForkJoinPool-1-worker-3
ForkJoinPool-1-worker-4

The same class now finishes in just over one second, and four different ForkJoinPool worker threads ran the tests (the sed strips the test letters, because which worker picks up which test changes every run). The four properties:

• parallel.enabled = true turns the feature on; nothing runs in parallel without it.
• mode.default = concurrent makes test methods run in parallel. The default, same_thread, would only allow parallelism where a class or method opts in with @Execution(CONCURRENT).
• strategy = fixed with parallelism = 4 uses exactly four threads. The default dynamic strategy uses one thread per available processor core, so the count changes by machine.

junit.jupiter.execution.parallel.mode.classes.default controls whether different classes run in parallel with each other, separately from methods.

Shared state races

Now the cost. Four tests each add 1 to a shared balance 100 times, with a 1 ms pause between reading the value and writing it back, as a slow database round trip would.

// File: src/test/java/BalanceTest.java
import org.junit.jupiter.api.*;

class BalanceTest {
    static int balance = 0;

    static void deposit100() throws InterruptedException {
        for (int i = 0; i < 100; i++) {
            int read = balance;
            Thread.sleep(1);
            balance = read + 1;
        }
    }

    @Test void first()  throws InterruptedException { deposit100(); }
    @Test void second() throws InterruptedException { deposit100(); }
    @Test void third()  throws InterruptedException { deposit100(); }
    @Test void fourth() throws InterruptedException { deposit100(); }

    @AfterAll static void report() {
        System.out.println("balance is 400: " + (balance == 400));
    }
}

$ mvn -B -ntp test -Dtest=BalanceTest | grep "balance is"
balance is 400: false

The balance should be 400, and it isn’t. Two tests running at once both read, say, 57, and both write 58: one deposit is lost. With parallel execution on, every test that touches balance races every other. The test only checks balance == 400, because the actual total is different on every run; that changing number is what a flaky test looks like in CI.

@ResourceLock tells JUnit that tests use a shared resource, so it never runs two of them at the same time:

// File: src/test/java/BalanceTest.java
import org.junit.jupiter.api.*;
import org.junit.jupiter.api.parallel.ResourceLock;

@ResourceLock("balance")
class BalanceTest {
    static int balance = 0;

    static void deposit100() throws InterruptedException {
        for (int i = 0; i < 100; i++) {
            int read = balance;
            Thread.sleep(1);
            balance = read + 1;
        }
    }

    @Test void first()  throws InterruptedException { deposit100(); }
    @Test void second() throws InterruptedException { deposit100(); }
    @Test void third()  throws InterruptedException { deposit100(); }
    @Test void fourth() throws InterruptedException { deposit100(); }

    @AfterAll static void report() {
        System.out.println("balance is 400: " + (balance == 400) + " (" + balance + ")");
    }
}

$ mvn -B -ntp test -Dtest=BalanceTest | grep -E "balance is|Tests run:.*-- in"
balance is 400: true (400)
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.742 s -- in BalanceTest

With @ResourceLock("balance") on the class, its tests take turns: the balance is exactly 400 on every run, and the class takes around 0.5 seconds of sequential work instead of overlapping. The lock is just a name; any other class annotated @ResourceLock("balance") would also wait for it, while classes without the lock still run in parallel with everything. Add mode = ResourceAccessMode.READ for tests that only read, so readers can share. For a test that must run with nothing else at all in parallel (it changes a system property, say), use @Isolated.

The better fix is not to share mutable static state between tests. But when the shared thing is real (a file, a port, an external database), @ResourceLock is how to keep parallel runs correct.

Practice

1. Give every test in the project a 2-second limit without annotating any of them, then check it with a test that sleeps for 3 seconds.

# File: src/test/resources/junit-platform.properties
junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.config.strategy = fixed
junit.jupiter.execution.parallel.config.fixed.parallelism = 4
junit.jupiter.execution.timeout.default = 2 s

// File: src/test/java/ReportTest.java
import org.junit.jupiter.api.Test;

class ReportTest {
    @Test
    void rendersReport() throws InterruptedException {
        Thread.sleep(3000);
    }
}

$ mvn -B -ntp test -Dtest=ReportTest | grep -E "^\[ERROR\]   "
[ERROR]   ReportTest.rendersReport » Timeout rendersReport() timed out after 2 seconds

2. Keep ParallelTest’s four tests on one thread even though parallel execution is enabled, and confirm it from the output.

// File: src/test/java/SequentialTest.java
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.parallel.Execution;
import org.junit.jupiter.api.parallel.ExecutionMode;

@Execution(ExecutionMode.SAME_THREAD)
class SequentialTest {
    static void work(String name) throws InterruptedException {
        Thread.sleep(1000);
        System.out.println(name + " on " + Thread.currentThread().getName());
    }

    @Test void a() throws InterruptedException { work("a"); }
    @Test void b() throws InterruptedException { work("b"); }
    @Test void c() throws InterruptedException { work("c"); }
    @Test void d() throws InterruptedException { work("d"); }
}

$ mvn -B -ntp test -Dtest=SequentialTest | grep -E " on |Tests run:.*-- in" | sed -E 's/^[a-d] on //' | sort | uniq -c
      1 [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.256 s -- in SequentialTest
      4 ForkJoinPool-1-worker-1

3. What happens to the race if parallelism is 1? Predict, then run BalanceTest’s unlocked version with -Djunit.jupiter.execution.parallel.config.fixed.parallelism=1.

// File: src/test/java/BalanceTest.java
import org.junit.jupiter.api.*;

class BalanceTest {
    static int balance = 0;

    static void deposit100() throws InterruptedException {
        for (int i = 0; i < 100; i++) {
            int read = balance;
            Thread.sleep(1);
            balance = read + 1;
        }
    }

    @Test void first()  throws InterruptedException { deposit100(); }
    @Test void second() throws InterruptedException { deposit100(); }
    @Test void third()  throws InterruptedException { deposit100(); }
    @Test void fourth() throws InterruptedException { deposit100(); }

    @AfterAll static void report() {
        System.out.println("balance is 400: " + (balance == 400));
    }
}

$ mvn -B -ntp test -Dtest=BalanceTest -Djunit.jupiter.execution.parallel.config.fixed.parallelism=1 | grep "balance is"
balance is 400: true