Parametrized Tests in Pytest
Run one test function against many inputs with parametrize, name the cases so failures stay readable, and parametrize a fixture to rerun a whole file.
A loop inside a test stops at the first failing value and reports one failure. parametrize
turns the same data into separate tests, so you learn about all of them in one run.
The loop that hides failures
# test_shipping.py
from shipping import shipping_cost
def test_standard_rates():
for weight, expected in [(0.5, 3.99), (2.0, 6.99), (10.0, 0.0)]:
assert shipping_cost(weight) == expected$ pytest -q
. [100%]
1 passed in 0.01sOne test, one dot. Break the middle case and the third never runs — the assertion raises and
the function exits. You fix 2.0, run again, and only then discover 10.0 was broken too.
One test per case
import pytest
from shipping import shipping_cost
@pytest.mark.parametrize(
"weight, expected",
[
(0.5, 3.99),
(1.99, 3.99),
(2.0, 6.99),
(9.99, 6.99),
(10.0, 0.0),
],
)
def test_standard_rates(weight, expected):
assert shipping_cost(weight) == expected$ pytest -v
========================= test session starts =========================
collected 5 items
test_shipping.py::test_standard_rates[0.5-3.99] PASSED [ 20%]
test_shipping.py::test_standard_rates[1.99-3.99] PASSED [ 40%]
test_shipping.py::test_standard_rates[2.0-6.99] PASSED [ 60%]
test_shipping.py::test_standard_rates[9.99-6.99] PASSED [ 80%]
test_shipping.py::test_standard_rates[10.0-0.0] PASSED [100%]
========================== 5 passed in 0.02s ==========================Five collected items from one function. The first argument is a comma-separated string of parameter names, the second a list of tuples matching them in order. pytest builds the bracketed test ID from the values.
Those IDs are addresses — run a single case with one:
pytest "test_shipping.py::test_standard_rates[10.0-0.0]"collected 1 item
test_shipping.py . [100%]
1 passed in 0.01sQuote it, because the brackets are glob characters in most shells.
What a failure looks like
Change the boundary case so 2.0kg is expected to be cheap:
(2.0, 3.99),$ pytest
collected 5 items
test_shipping.py ..F.. [100%]
============================== FAILURES ===============================
_____________________ test_standard_rates[2.0-3.99] ___________________
weight = 2.0, expected = 3.99
@pytest.mark.parametrize(
"weight, expected",
[
(0.5, 3.99),
...
def test_standard_rates(weight, expected):
> assert shipping_cost(weight) == expected
E assert 6.99 == 3.99
E + where 6.99 = shipping_cost(2.0)
test_shipping.py:16: AssertionError
======================= short test summary info =======================
FAILED test_shipping.py::test_standard_rates[2.0-3.99] - assert 6.99 == 3.99
================== 1 failed, 4 passed in 0.03s ========================The other four still ran and still passed. The header line weight = 2.0, expected = 3.99 is
pytest printing the arguments for the failing case, which is usually all you need to see.
Naming the cases
Value-derived IDs stop being readable the moment the values are not numbers:
@pytest.mark.parametrize(
"address, valid",
[
({"line1": "1 High St", "postcode": "SW1A 1AA"}, True),
({"line1": "1 High St", "postcode": ""}, False),
],
)
def test_address_validation(address, valid):
assert is_valid(address) is validtest_addresses.py::test_address_validation[address0-True] PASSED [ 50%]
test_addresses.py::test_address_validation[address1-False] PASSED [100%]address0 tells you nothing. Supply ids:
@pytest.mark.parametrize(
"address, valid",
[
({"line1": "1 High St", "postcode": "SW1A 1AA"}, True),
({"line1": "1 High St", "postcode": ""}, False),
],
ids=["complete", "missing-postcode"],
)
def test_address_validation(address, valid):
assert is_valid(address) is validtest_addresses.py::test_address_validation[complete] PASSED [ 50%]
test_addresses.py::test_address_validation[missing-postcode] PASSED [100%]Now the CI failure line reads FAILED …[missing-postcode] and you know what broke before
opening the file. ids must be the same length as the case list.
Marking one case
pytest.param wraps a single case so it can carry its own ID or marks:
@pytest.mark.parametrize(
"weight, expected",
[
(1.0, 3.99),
pytest.param(0.0, 0.0, marks=pytest.mark.xfail(raises=ValueError,
reason="zero is rejected")),
pytest.param(1_000_000.0, 0.0, id="absurd-weight"),
],
)
def test_rates(weight, expected):
assert shipping_cost(weight) == expected$ pytest -v
collected 3 items
test_shipping.py::test_rates[1.0-3.99] PASSED [ 33%]
test_shipping.py::test_rates[0.0-0.0] XFAIL [ 66%]
test_shipping.py::test_rates[absurd-weight] PASSED [100%]
=================== 2 passed, 1 xfailed in 0.02s ======================The zero case raises ValueError, which is what the mark says to expect, so it reports
XFAIL instead of failing. Marking one case beats deleting it: the expectation stays written
down, and if the behaviour ever changes pytest reports XPASS and you find out.
Stacking decorators
Two parametrize decorators on the same function produce every combination:
@pytest.mark.parametrize("express", [False, True])
@pytest.mark.parametrize("weight", [1.0, 5.0, 12.0])
def test_cost_is_never_negative(weight, express):
assert shipping_cost(weight, express) >= 0$ pytest -q
...... [100%]
6 passed in 0.02sThree weights times two flags is six tests. This is the cheapest way to turn a rule that should hold everywhere — “cost is never negative” — into broad coverage. The count multiplies, though: three stacked decorators of five values each is 125 tests.
Parametrizing a fixture
Give a fixture params and every test that requests it runs once per value:
# conftest.py
import pytest
from store import DictStore, FileStore
@pytest.fixture(params=["memory", "file"])
def store(request, tmp_path):
if request.param == "memory":
yield DictStore()
else:
yield FileStore(tmp_path / "orders.json")# test_store.py
def test_round_trip(store):
store.put("A1", {"qty": 2})
assert store.get("A1") == {"qty": 2}
def test_missing_key_returns_none(store):
assert store.get("nope") is None$ pytest -v
collected 4 items
test_store.py::test_round_trip[memory] PASSED [ 25%]
test_store.py::test_round_trip[file] PASSED [ 50%]
test_store.py::test_missing_key_returns_none[memory] PASSED [ 75%]
test_store.py::test_missing_key_returns_none[file] PASSED [100%]Two tests became four, and neither test knows it happened. request.param holds the current
value; request is the built-in fixture exposing the requesting context. Add "sqlite" to
that list and the whole file runs against SQLite too — this is how one suite stays honest
across several backends.
Selecting by ID
pytest -k memory # every case whose ID contains "memory"
pytest -k "file and not missing" # boolean expressions work$ pytest -k memory -q
.. [100%]
2 passed, 2 deselected in 0.02sdeselected, not skipped — the tests were collected and then filtered out. That is why
-k is fine for a quick loop and useless as a permanent way to disable a test: nothing
records that the filtered cases were meant to run.
Practice
1. Parametrize a test over the express rate for 1kg, 5kg and 12kg.
@pytest.mark.parametrize(
"weight, expected",
[(1.0, 9.98), (5.0, 17.48), (12.0, 17.48)],
ids=["light", "medium", "heavy"],
)
def test_express_rates(weight, expected):
assert shipping_cost(weight, express=True) == pytest.approx(expected, abs=0.01)test_shipping.py::test_express_rates[light] PASSED [ 33%]
test_shipping.py::test_express_rates[medium] PASSED [ 66%]
test_shipping.py::test_express_rates[heavy] PASSED [100%]12kg costs the same as 5kg because the free-over-10kg rule does not apply to express. The table makes that rule visible at a glance, which prose in a docstring never does.
2. Give one case a wrong expectation and run with --tb=line.
$ pytest --tb=line -q
.F. [100%]
/home/you/shipping/test_shipping.py:18: assert 17.48 == 9.98
1 failed, 2 passed in 0.02sOne line per failure instead of a full traceback. When twenty parametrized cases fail for the
same reason, --tb=line fits them all on one screen.
3. Stack two parametrize decorators with 4 and 3 values, then count the tests.
$ pytest --collect-only -q | tail -3
12 tests collected in 0.01sTwelve, the product. --collect-only shows what would run without running it — the fastest
check that a parametrize list expands the way you meant.
4. Add a third value to the store fixture's params and rerun.
collected 6 items
test_store.py::test_round_trip[memory] PASSED [ 16%]
test_store.py::test_round_trip[file] PASSED [ 33%]
test_store.py::test_round_trip[sqlite] PASSED [ 50%]
test_store.py::test_missing_key_returns_none[memory] PASSED [ 66%]
test_store.py::test_missing_key_returns_none[file] PASSED [ 83%]
test_store.py::test_missing_key_returns_none[sqlite] PASSED [100%]Every test in the file gained a case without being edited. That is the argument for putting the backend choice in a fixture rather than in each test: coverage grows from one line.
Next: marks — skipping tests, expecting failures, and selecting groups by name.