Testing with MUnit and ScalaCheck

Tests run with the same one-line tooling as everything else in this track: a using directive for the dependency and scala-cli test. No build file.

The code under test

// shipping.scala
case class Order(id: Int, country: String, status: String, amount: BigDecimal)

object Shipping:
  def cost(weightKg: Double, express: Boolean = false): BigDecimal =
    require(weightKg > 0, s"weight must be positive, got $weightKg")
    val base = if weightKg < 2 then BigDecimal("3.99") else BigDecimal("6.99")
    val adjusted = if weightKg >= 10 && !express then BigDecimal(0) else base
    (adjusted * (if express then BigDecimal("2.5") else BigDecimal(1)))
      .setScale(2, BigDecimal.RoundingMode.HALF_UP)

  def revenueByCountry(orders: List[Order]): Map[String, BigDecimal] =
    orders.filter(_.status == "completed").groupMapReduce(_.country)(_.amount)(_ + _)

A first suite

// shipping.test.scala
//> using dep org.scalameta::munit::1.0.3

class ShippingSuite extends munit.FunSuite:

  test("light parcels cost the base rate"):
    assertEquals(Shipping.cost(1.5), BigDecimal("3.99"))

  test("heavy standard parcels ship free"):
    assertEquals(Shipping.cost(12), BigDecimal("0.00"))

  test("express multiplies the base rate"):
    assertEquals(Shipping.cost(1.5, express = true), BigDecimal("9.98"))

  test("negative weight is rejected"):
    val e = intercept[IllegalArgumentException](Shipping.cost(-1))
    assert(e.getMessage.contains("must be positive"))

scala-cli test .

Compiling project (test, Scala 3.6.2, JVM (21))
Compiled project (test, Scala 3.6.2, JVM (21))
ShippingSuite:
  + light parcels cost the base rate 0.021s
  + heavy standard parcels ship free 0.001s
  + express multiplies the base rate 0.001s
  + negative weight is rejected 0.004s
Execution took 0.03s
4 tests, 4 passed

assertEquals is type-checked — comparing a BigDecimal with a String will not compile, which rules out a whole class of test that passes for the wrong reason.

Reading a failure

  test("express multiplies the base rate"):
    assertEquals(Shipping.cost(1.5, express = true), BigDecimal("9.99"))

ShippingSuite:
  + light parcels cost the base rate 0.019s
==> X ShippingSuite.express multiplies the base rate  0.014s munit.ComparisonFailException: shipping.test.scala:12
11:  test("express multiplies the base rate"):
12:    assertEquals(Shipping.cost(1.5, express = true), BigDecimal("9.99"))
13:
values are not the same
=> Obtained
9.98
=> Diff (- obtained, + expected)
-9.98
+9.99
4 tests, 3 passed, 1 failed

The source line, the obtained and expected values, and a diff. On collections the diff is where MUnit earns its place:

  test("revenue is grouped by country"):
    val orders = List(
      Order(1001, "GB", "completed", BigDecimal("25.50")),
      Order(1002, "US", "completed", BigDecimal("12.00")),
      Order(1003, "GB", "returned",  BigDecimal("40.00")),
    )
    assertEquals(
      Shipping.revenueByCountry(orders),
      Map("GB" -> BigDecimal("65.50"), "US" -> BigDecimal("12.00"))
    )

==> X ShippingSuite.revenue is grouped by country  0.031s munit.ComparisonFailException
values are not the same
=> Diff (- obtained, + expected)
 Map(
-  "GB" -> 25.50,
+  "GB" -> 65.50,
   "US" -> 12.00
 )
1 test, 0 passed, 1 failed

The test was wrong, not the code — the returned order should not count. A diff that names the differing key is the difference between a two-second fix and reading both maps by eye.

Add a clue to an assertion that would otherwise be cryptic:

    assert(total > 0, s"expected positive total for $orders")

Fixtures

class OrdersSuite extends munit.FunSuite:

  val tempFile = FunFixture[java.nio.file.Path](
    setup = _ => java.nio.file.Files.createTempFile("orders", ".csv"),
    teardown = f => java.nio.file.Files.deleteIfExists(f),
  )

  tempFile.test("writes a header row"): path =>
    java.nio.file.Files.writeString(path, "id,amount\n1001,25.50\n")
    val lines = java.nio.file.Files.readAllLines(path)
    assertEquals(lines.get(0), "id,amount")
    assertEquals(lines.size, 2)

OrdersSuite:
  + writes a header row 0.042s
1 test, 1 passed

teardown runs whether the test passes or fails, so a failing test still cleans up. For something expensive and shared, override beforeAll / afterAll instead — and keep it read-only, because MUnit runs suites in parallel by default.

Async tests

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

class AsyncSuite extends munit.FunSuite:

  def fetchTotal(customerId: Int): Future[BigDecimal] =
    Future(BigDecimal("77.50"))

  test("returns the customer total"):
    fetchTotal(1).map: total =>
      assertEquals(total, BigDecimal("77.50"))

  test("fails for an unknown customer"):
    val f = Future(throw new NoSuchElementException("no customer 99"))
    f.failed.map: e =>
      assertEquals(e.getMessage, "no customer 99")

AsyncSuite:
  + returns the customer total 0.104s
  + fails for an unknown customer 0.008s
2 tests, 2 passed

Return the Future and MUnit waits for it. No Await, and a failed future fails the test rather than passing silently — which is what happens if you forget the return and the test body evaluates to Unit.

Raise the limit for a genuinely slow test:

  override val munitTimeout = scala.concurrent.duration.Duration(60, "s")

Property-based tests

Example tests check the cases you thought of. Property tests check the ones you did not:

//> using dep org.scalameta::munit-scalacheck::1.0.0

import org.scalacheck.Prop.*
import org.scalacheck.Gen

class ShippingPropSuite extends munit.ScalaCheckSuite:

  property("cost is never negative"):
    forAll(Gen.posNum[Double]): w =>
      Shipping.cost(w) >= BigDecimal(0)

  property("express never costs less than standard"):
    forAll(Gen.choose(0.1, 9.9)): w =>
      Shipping.cost(w, express = true) >= Shipping.cost(w)

  property("revenue equals the sum of completed orders"):
    val genOrder = for
      id      <- Gen.posNum[Int]
      country <- Gen.oneOf("GB", "US", "NL")
      status  <- Gen.oneOf("completed", "returned", "pending")
      amount  <- Gen.choose(1, 10000).map(c => BigDecimal(c) / 100)
    yield Order(id, country, status, amount)

    forAll(Gen.listOf(genOrder)): orders =>
      val expected = orders.filter(_.status == "completed").map(_.amount).sum
      Shipping.revenueByCountry(orders).values.sum == expected

ShippingPropSuite:
  + cost is never negative 0.182s
  + express never costs less than standard 0.094s
  + revenue equals the sum of completed orders 0.311s
3 tests, 3 passed

Each property ran 100 generated cases. When one fails, ScalaCheck shrinks the input to the smallest example that still fails:

==> X ShippingPropSuite.express never costs less than standard  0.121s
Failing seed: 9kL2mQx8vT1nR4pY7wZ3aB6cD5eF0gH2jK8lM4nP1qS
falsified after 12 successful property evaluations.
> ARG_0: 10.0
> ARG_0_ORIGINAL: 7823.4419

The original counterexample was 7823.44; the shrunk one is 10.0 — the exact boundary where free shipping starts. That is the value a human would have written a test for, found automatically.

The failing seed reproduces the run:

scala-cli test . -- -Dscalacheck.seed=9kL2mQx8vT1nR4pY7wZ3aB6cD5eF0gH2jK8lM4nP1qS

Running a subset

scala-cli test . --test-only 'ShippingSuite'

ShippingSuite:
  + light parcels cost the base rate 0.020s
  + heavy standard parcels ship free 0.001s
4 tests, 4 passed

  test("known broken".ignore) { ??? }
  test("slow".tag(munit.Slow)) { ??? }
  test("only this one".only) { ??? }

  + light parcels cost the base rate 0.019s
  - known broken (ignored)

.only is convenient and dangerous — committed, it silently reduces CI to one test. MUnit fails the suite if .only is present and the munitFlakyOK-style CI flag is set; simplest is a lint rule or a grep in the pipeline.

ScalaTest, which you will also meet

//> using dep org.scalatest::scalatest::3.2.19

import org.scalatest.funsuite.AnyFunSuite
import org.scalatest.matchers.should.Matchers

class ShippingSpec extends AnyFunSuite with Matchers:
  test("light parcels cost the base rate") {
    Shipping.cost(1.5) shouldBe BigDecimal("3.99")
  }
  test("negative weight is rejected") {
    an[IllegalArgumentException] should be thrownBy Shipping.cost(-1)
  }

ShippingSpec:
- light parcels cost the base rate
- negative weight is rejected
Run completed in 312 milliseconds.
Total number of tests run: 2
All tests passed.

Same tests, a matcher DSL, and several suite styles to choose from. Prefer MUnit for new work and read ScalaTest fluently, because most existing Scala codebases use it.

Practice

=> Diff (- obtained, + expected)
 Map(
-  "GB" -> 25.50,
+  "GB" -> 65.50,
   "US" -> 12.00
 )

test("returns the customer total"):
  fetchTotal(1).map(total => assertEquals(total, BigDecimal("77.50")))

+ returns the customer total 0.104s

falsified after 12 successful property evaluations.
> ARG_0: 10.0
> ARG_0_ORIGINAL: 7823.4419

==> X OrdersSuite.writes a header row  0.038s
1 test, 0 passed, 1 failed

ls /tmp/orders*.csv

ls: cannot access '/tmp/orders*.csv': No such file or directory

Next: the type system — generics, variance, and opaque types.