Testing Python Code

Tests are small programs that run your code and check the result. pytest finds them, runs them and reports exactly what went wrong. Output below is from Python 3.13 with pytest 9.1, pytest-cov 7.1 and pytest-asyncio 1.4 (pip install pytest pytest-cov pytest-asyncio); the run time at the end of each report varies.

A first test — and a first failure

Put the code in one file and tests in a file named test_*.py. Each test_ function is a test.

# calc.py
def add(a, b):
    return a + b

def average(values):
    return sum(values) / len(values)

# test_calc.py
from calc import add, average

def test_add():
    assert add(2, 3) == 5

def test_add_floats():
    assert add(0.1, 0.2) == 0.3

def test_average():
    assert average([1, 2, 3, 4]) == 2.5

$ pytest test_calc.py
============================= test session starts ==============================
platform linux -- Python 3.13.16, pytest-9.1.1, pluggy-1.6.0
rootdir: /home/user/py-demo/testing
plugins: asyncio-1.4.0, cov-7.1.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 3 items

test_calc.py .F.                                                         [100%]

=================================== FAILURES ===================================
_______________________________ test_add_floats ________________________________

    def test_add_floats():
>       assert add(0.1, 0.2) == 0.3
E       assert 0.30000000000000004 == 0.3
E        +  where 0.30000000000000004 = add(0.1, 0.2)

test_calc.py:8: AssertionError
=========================== short test summary info ============================
FAILED test_calc.py::test_add_floats - assert 0.30000000000000004 == 0.3
========================= 1 failed, 2 passed in 0.02s ==========================

Each . is a pass and F a failure. pytest rewrote the plain assert to show both sides and where the left value came from — no assertEqual needed. The bug is in the test, not add: floats aren’t exact. pytest.approx compares with a tolerance:

# test_calc.py
import pytest
from calc import add, average

def test_add():
    assert add(2, 3) == 5

def test_add_floats():
    assert add(0.1, 0.2) == pytest.approx(0.3)

def test_average():
    assert average([1, 2, 3, 4]) == 2.5

$ pytest -q test_calc.py
...                                                                      [100%]
3 passed in 0.01s

-q trims the report to the essentials. Other everyday flags: -v (one line per test), -x (stop at the first failure), -k name (run tests whose names match).

Testing exceptions

pytest.raises passes only if the block raises the given exception, and match checks its message.

# test_errors.py
import pytest
from calc import average

def test_average_empty():
    with pytest.raises(ZeroDivisionError, match="division by zero"):
        average([])

def test_average_wrong_error():
    with pytest.raises(ValueError):
        average([])

$ pytest -q test_errors.py
.F                                                                       [100%]
=================================== FAILURES ===================================
___________________________ test_average_wrong_error ___________________________

    def test_average_wrong_error():
        with pytest.raises(ValueError):
>           average([])

test_errors.py:11: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

values = []

    def average(values):
>       return sum(values) / len(values)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
E       ZeroDivisionError: division by zero

calc.py:6: ZeroDivisionError
=========================== short test summary info ============================
FAILED test_errors.py::test_average_wrong_error - ZeroDivisionError: division...
1 failed, 1 passed in 0.02s

The second test expected ValueError, so the real ZeroDivisionError escaped pytest.raises and failed the test — showing the local variable values = [] in the frame where it happened. That second test documents a design question: should average([]) raise something clearer?

Fixtures: setup and teardown

A fixture is a function that prepares something a test needs; tests receive it by naming it as a parameter.

# test_db.py
import sqlite3
import pytest

@pytest.fixture
def db():
    conn = sqlite3.connect(":memory:")
    conn.execute("CREATE TABLE users (name TEXT)")
    print("\n  setup: fresh database")
    yield conn
    conn.close()
    print("  teardown: closed")

def test_insert(db):
    db.execute("INSERT INTO users VALUES ('alice')")
    assert db.execute("SELECT COUNT(*) FROM users").fetchone() == (1,)

def test_starts_empty(db):
    assert db.execute("SELECT COUNT(*) FROM users").fetchone() == (0,)

$ pytest -q -s test_db.py

  setup: fresh database
.  teardown: closed

  setup: fresh database
.  teardown: closed

2 passed in 0.01s

-s lets the print calls through. Each test got its own database: the row inserted by test_insert was gone by test_starts_empty, because the fixture’s default scope is one call per test. Code after yield is teardown and runs even if the test fails. @pytest.fixture(scope="module") or "session" shares one instance across many tests — faster, but tests can then leak state into each other. Fixtures in a conftest.py file are available to every test in that directory without importing.

Parametrize: one test, many cases

# test_params.py
import pytest

@pytest.mark.parametrize("text, expected", [
    ("hello", "HELLO"),
    ("", ""),
    ("straße", "STRASSE"),
    ("Python 3", "PYTHON 3"),
])
def test_upper(text, expected):
    assert text.upper() == expected

$ pytest -v --no-header test_params.py
============================= test session starts ==============================
collecting ... collected 4 items

test_params.py::test_upper[hello-HELLO] PASSED                           [ 25%]
test_params.py::test_upper[-] PASSED                                     [ 50%]
test_params.py::test_upper[stra\xdfe-STRASSE] PASSED                     [ 75%]
test_params.py::test_upper[Python 3-PYTHON 3] PASSED                     [100%]

============================== 4 passed in 0.01s ===============================

--no-header dropped the platform, rootdir and plugin lines from the header. Each tuple became a separately reported test with its parameters in the ID, so a failure tells you exactly which case broke. The German ß upper-cases to two letters — the kind of edge case a parameter table makes cheap to pin down.

Mocking dependencies

A Mock stands in for a collaborator and records how it was used.

# notify.py
def send_welcome(user: dict, mailer) -> bool:
    if not user.get("email"):
        return False
    mailer.send(user["email"], f"Welcome, {user['name']}!")
    return True

# test_notify.py
from unittest.mock import Mock
from notify import send_welcome

def test_sends_email():
    mailer = Mock()
    assert send_welcome({"name": "Ana", "email": "ana@example.com"}, mailer)
    mailer.send.assert_called_once_with("ana@example.com", "Welcome, Ana!")

def test_skips_missing_email():
    mailer = Mock()
    assert not send_welcome({"name": "Bo"}, mailer)
    mailer.send.assert_not_called()

def test_wrong_expectation():
    mailer = Mock()
    send_welcome({"name": "Cy", "email": "cy@example.com"}, mailer)
    mailer.send.assert_called_once_with("cy@example.com", "Hi Cy")

$ pytest -q --tb=line test_notify.py
..F                                                                      [100%]
=================================== FAILURES ===================================
E   AssertionError: expected call not found.
    Expected: send('cy@example.com', 'Hi Cy')
      Actual: send('cy@example.com', 'Welcome, Cy!')
/usr/lib/python3.13/unittest/mock.py:979: AssertionError: expected call not found.
=========================== short test summary info ============================
FAILED test_notify.py::test_wrong_expectation - AssertionError: expected call...
1 failed, 2 passed in 0.03s

Passing the mailer in as a parameter made it trivial to substitute. The failing test shows what a broken expectation looks like: even with the one-line traceback style (--tb=line), the mock’s error message lists the expected and the actual call side by side.

patch: replacing what a module imported

When code reaches out to a dependency itself, patch swaps it for the duration of a test — patch the name where it’s used.

# weather.py
import urllib.request
import json

def temperature(city: str) -> float:
    with urllib.request.urlopen(f"https://weather.example/api?city={city}") as resp:
        return json.load(resp)["temp_c"]

# test_weather.py
import io
import json
from unittest.mock import patch
from weather import temperature

@patch("weather.urllib.request.urlopen")
def test_temperature(mock_urlopen):
    mock_urlopen.return_value.__enter__.return_value = io.StringIO(json.dumps({"temp_c": 21.5}))
    assert temperature("Pune") == 21.5
    mock_urlopen.assert_called_once_with("https://weather.example/api?city=Pune")

$ pytest -q test_weather.py
.                                                                        [100%]
1 passed in 0.02s

No request was sent — urlopen was replaced inside weather for this one test and restored afterwards. Because urlopen is used as a context manager, the fake response is configured on __enter__’s return value.

Coverage

pytest-cov reports which lines the tests never executed.

# grades.py
def grade(score: int) -> str:
    if score < 0 or score > 100:
        raise ValueError("score out of range")
    if score >= 90:
        return "A"
    if score >= 50:
        return "pass"
    return "fail"

# test_grades.py
from grades import grade

def test_a():
    assert grade(95) == "A"

def test_pass():
    assert grade(60) == "pass"

$ pytest -q --cov=grades --cov-report=term-missing test_grades.py
..                                                                       [100%]
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.16-final-0 _______________

Name        Stmts   Miss  Cover   Missing
-----------------------------------------
grades.py       8      2    75%   4, 9
-----------------------------------------
TOTAL           8      2    75%
2 passed in 0.03s

Lines 4 and 9 — the out-of-range error and the "fail" branch — never ran, so nothing checks them. Coverage can’t tell you whether covered lines are tested well, only which ones aren’t tested at all. --cov-fail-under=80 turns a minimum into a CI failure.

Async tests

With pytest-asyncio, mark a coroutine test and it runs inside an event loop.

# test_async.py
import asyncio
import pytest

async def fetch_status(delay: float) -> str:
    await asyncio.sleep(delay)
    return "ok"

@pytest.mark.asyncio
async def test_fetch_status():
    assert await fetch_status(0.01) == "ok"

@pytest.mark.asyncio
async def test_timeout():
    with pytest.raises(TimeoutError):
        async with asyncio.timeout(0.01):
            await fetch_status(1)

$ pytest -q test_async.py
..                                                                       [100%]
2 passed in 0.03s

Forget the marker and the test doesn’t silently pass — pytest refuses to run it:

# test_async_unmarked.py
import asyncio

async def test_no_marker():
    await asyncio.sleep(0)

$ pytest -q -p no:cacheprovider test_async_unmarked.py
F                                                                        [100%]
=================================== FAILURES ===================================
________________________________ test_no_marker ________________________________
async def functions are not natively supported.
You need to install a suitable plugin for your async framework, for example:
  - anyio
  - pytest-asyncio
  - pytest-tornasync
  - pytest-trio
  - pytest-twisted
=========================== short test summary info ============================
FAILED test_async_unmarked.py::test_no_marker - Failed: async def functions a...
1 failed in 0.01s

In strict mode (pytest-asyncio’s default) each coroutine test needs @pytest.mark.asyncio; setting asyncio_mode = "auto" in your pytest config applies it to every async def test.

unittest

The standard library’s unittest uses classes and assert* methods; pytest runs these tests unchanged.

# test_unit.py
import unittest
from calc import add, average

class TestCalc(unittest.TestCase):
    def setUp(self):
        self.values = [1, 2, 3, 4]

    def test_add(self):
        self.assertEqual(add(2, 3), 5)

    def test_average(self):
        self.assertAlmostEqual(average(self.values), 2.5)

    def test_empty(self):
        with self.assertRaises(ZeroDivisionError):
            average([])

if __name__ == "__main__":
    unittest.main()

$ python test_unit.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK
$ pytest -q test_unit.py
...                                                                      [100%]
3 passed in 0.01s

setUp runs before each test method, like a function-scoped fixture.

Practice

1. Add a parametrized test for grade that covers 0, 49, 50, 89, 90 and 100.

# test_grade_params.py
import pytest
from grades import grade

@pytest.mark.parametrize("score, expected", [
    (0, "fail"), (49, "fail"), (50, "pass"), (89, "pass"), (90, "A"), (100, "A"),
])
def test_grade_boundaries(score, expected):
    assert grade(score) == expected

$ pytest -q test_grade_params.py
......                                                                   [100%]
6 passed in 0.01s

2. Bring grades.py to 100% coverage by adding the missing tests.

# test_grades_full.py
import pytest
from grades import grade

def test_a():
    assert grade(95) == "A"

def test_pass():
    assert grade(60) == "pass"

def test_fail():
    assert grade(10) == "fail"

@pytest.mark.parametrize("bad", [-1, 101])
def test_out_of_range(bad):
    with pytest.raises(ValueError, match="out of range"):
        grade(bad)

$ pytest -q --cov=grades --cov-report=term-missing test_grades_full.py
.....                                                                    [100%]
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.16-final-0 _______________

Name        Stmts   Miss  Cover   Missing
-----------------------------------------
grades.py       8      0   100%
-----------------------------------------
TOTAL           8      0   100%
5 passed in 0.04s

3. Write a fixture tmp_config that uses pytest’s built-in tmp_path fixture to create a config.json, and a test that reads it.

# test_config.py
import json
import pytest

@pytest.fixture
def tmp_config(tmp_path):
    path = tmp_path / "config.json"
    path.write_text(json.dumps({"debug": True}))
    return path

def test_reads_debug(tmp_config):
    assert json.loads(tmp_config.read_text())["debug"] is True
    assert tmp_config.parent.exists()

$ pytest -q test_config.py
.                                                                        [100%]
1 passed in 0.01s

4. Use Mock(side_effect=...) to make the mailer raise ConnectionError, and assert that send_welcome lets the error propagate.

# test_mailer_error.py
import pytest
from unittest.mock import Mock
from notify import send_welcome

def test_mailer_failure_propagates():
    mailer = Mock()
    mailer.send.side_effect = ConnectionError("smtp down")
    with pytest.raises(ConnectionError, match="smtp down"):
        send_welcome({"name": "Ana", "email": "ana@example.com"}, mailer)
    assert mailer.send.call_count == 1

$ pytest -q test_mailer_error.py
.                                                                        [100%]
1 passed in 0.01s