Isn't an OpenAPI schema just a jsonschema in disguise?

For the parts that matter here, yes — OpenAPI's `components.schemas` section uses JSON Schema syntax (type, required, properties) directly, so a schema fragment pulled out of a real OpenAPI document can be handed straight to jsonschema.validate(). That's the whole idea of this lesson: the API's actual published contract becomes the same validation this track has used since lesson 4, instead of a hand-maintained duplicate of it.

Why did jsonschema need a resolver just to validate one schema?

The array items use $ref: '#/components/schemas/Book' instead of inlining the object schema — a common OpenAPI pattern once multiple endpoints share a type. A resolver knows how to look that reference up inside the full document; validating the extracted fragment alone, without the resolver, would raise a reference-resolution error the moment it hit the $ref.

What's the point of testing a 'proposed' response that was never actually returned by the API?

It catches a breaking change before it ships, not after — validate a fixture representing what the API team is about to change against the existing contract, in a PR, before the code merges. That's strictly earlier than testing the real response, which only exists once the change is already live.

Why generate a JUnit XML report instead of just reading pytest's terminal output in CI?

GitHub Actions (and most CI systems) can render a JUnit XML file as a test-results summary directly on the PR — pass/fail counts and per-test detail, without anyone opening the raw log. --junitxml=results.xml is pytest's built-in support for that format; nothing extra to install.

Where should the API token used in CI actually live?

In the CI provider's encrypted secrets store — GitHub Actions' `secrets.*` context here — never committed in the workflow file or a config. The workflow reads it into an environment variable at run time, and test code reads that variable the same way it would read any other piece of environment-specific config.

Contract Testing with OpenAPI, and Running in CI

Turn an OpenAPI document into the same jsonschema validation from earlier in this track, catch a proposed API change that breaks the contract, and wire the suite into GitHub Actions with a JUnit report.

By SkillByExample · Published Sep 13, 2026

api-testing python openapi ci-cd github-actions pytest

Every test in this track has run locally, on demand. The last step is making the same suite run on every pull request, against the contract the API actually publishes.

Turning OpenAPI into the schema you already know

# openapi.yaml
openapi: 3.0.3
info:
  title: Book API
  version: "1.0"
paths:
  /books:
    get:
      responses:
        "200":
          description: A list of books
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: "#/components/schemas/Book"
components:
  schemas:
    Book:
      type: object
      required: [id, title, year]
      properties:
        id:
          type: integer
        title:
          type: string
        year:
          type: integer

pip install pyyaml jsonschema

# test_contract.py
import jsonschema
import requests
import yaml


def load_books_schema():
    with open("openapi.yaml") as f:
        spec = yaml.safe_load(f)
    schema = spec["paths"]["/books"]["get"]["responses"]["200"]["content"]["application/json"]["schema"]
    resolver = jsonschema.RefResolver.from_schema(spec)
    return schema, resolver


def test_books_response_matches_openapi_contract(local_api_url):
    schema, resolver = load_books_schema()
    response = requests.get(f"{local_api_url}/books")
    jsonschema.validate(instance=response.json(), schema=schema, resolver=resolver)

pytest test_contract.py -v

test_contract.py::test_books_response_matches_openapi_contract PASSED    [100%]

============================== warnings summary ===============================
test_contract.py::test_books_response_matches_openapi_contract
  test_contract.py:10: DeprecationWarning: jsonschema.RefResolver is deprecated as of
  v4.18.0, in favor of the referencing library, which provides more compliant
  referencing behavior as well as more flexible APIs for customization.

======================== 1 passed, 1 warning in 0.73s =========================

Two things worth reading here, not just the PASSED. First, RefResolver is what follows $ref: "#/components/schemas/Book" back into the full document — without it, validating the extracted schema fragment on its own would fail to resolve the reference. Second, pytest shows the deprecation warning it caught along with the pass — a warning is not nothing just because the test succeeded; RefResolver’s replacement is the referencing library, worth migrating to on a project meant to outlive this one Python version.

Catching a breaking change before it ships

import jsonschema
import pytest

from test_contract import load_books_schema


def test_proposed_response_with_string_year_fails_the_contract():
    schema, resolver = load_books_schema()
    proposed_response = [{"id": 1, "title": "Dune", "year": "1965"}]
    with pytest.raises(jsonschema.exceptions.ValidationError, match="'1965' is not of type 'integer'"):
        jsonschema.validate(instance=proposed_response, schema=schema, resolver=resolver)

test_contract_drift.py::test_proposed_response_with_string_year_fails_the_contract PASSED [100%]

proposed_response was never returned by any server — it’s a fixture standing in for a change someone is about to make (say, a PR that switches year to a formatted string). Validating it against the existing contract catches the break in code review, before it merges, which is strictly earlier than any test that has to wait for a real response to inspect.

Running the same suite in CI

pytest --junitxml=results.xml

============================== 1 passed, 1 warning in 0.73s =========================

<?xml version="1.0" encoding="utf-8"?>
<testsuites name="pytest tests">
  <testsuite name="pytest" errors="0" failures="0" skipped="0" tests="1" time="0.678"
             timestamp="2026-09-13T16:29:06+05:30" hostname="ci-runner">
    <testcase classname="test_contract" name="test_books_response_matches_openapi_contract"
              time="0.522" />
  </testsuite>
</testsuites>

--junitxml is built into pytest — no plugin needed. GitHub Actions (and most other CI systems) can render this file directly as a pass/fail summary on the pull request, which is what a raw terminal log can’t do on its own.

# .github/workflows/api-tests.yml
name: API tests
on: pull_request

jobs:
  test:
    runs-on: ubuntu-latest
    env:
      API_TOKEN: ${{ secrets.API_TOKEN }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }

      - run: pip install -r requirements.txt
      - run: pytest --junitxml=results.xml

      - name: Publish test report
        if: always()
        uses: dorny/test-reporter@v1
        with:
          name: API test results
          path: results.xml
          reporter: java-junit

Run pytest --junitxml=results.xml
============================= test session starts =============================
collected 10 items

test_contract.py::test_books_response_matches_openapi_contract PASSED

============================== 1 passed in 0.73s ==============================

API_TOKEN comes from secrets.API_TOKEN, GitHub’s encrypted store — never written into the workflow file itself. if: always() on the report step matters the same way it did for cleanup in an earlier lesson’s CI example: a report that only publishes on success is useless for exactly the runs where you need to see what failed.

Practice

1. Validate a single book (not the list) against the item schema.

def test_get_one_book_conforms_to_the_item_schema(local_api_url):
    schema, resolver = load_books_schema()
    item_schema = schema["items"]
    response = requests.get(f"{local_api_url}/books/1")
    jsonschema.validate(instance=response.json(), schema=item_schema, resolver=resolver)

test_practice.py::test_get_one_book_conforms_to_the_item_schema PASSED   [ 33%]

schema["items"] is the $ref itself — the same resolver used for the list validates the single-object endpoint too, since both point at the same Book definition.

2. Validate a proposed response that's missing a required field.

def test_proposed_response_missing_title_fails_the_contract():
    schema, resolver = load_books_schema()
    proposed_response = [{"id": 1, "year": 1965}]
    with pytest.raises(jsonschema.exceptions.ValidationError, match="'title' is a required property"):
        jsonschema.validate(instance=proposed_response, schema=schema, resolver=resolver)

test_practice.py::test_proposed_response_missing_title_fails_the_contract PASSED [ 66%]

Same contract, same technique as the missing-field test from the JSON validation lesson — the only difference here is that the schema comes from openapi.yaml instead of being hand-typed into the test file.

3. Confirm a book created via POST also conforms to the published contract.

def test_created_book_also_conforms_to_the_contract(local_api_url):
    schema, resolver = load_books_schema()
    item_schema = schema["items"]
    response = requests.post(f"{local_api_url}/books", json={"title": "Neuromancer", "year": 1984})
    jsonschema.validate(instance=response.json(), schema=item_schema, resolver=resolver)

test_practice.py::test_created_book_also_conforms_to_the_contract PASSED [100%]

The contract applies to every response shape the API produces, not just the one endpoint that happened to get tested first — worth checking explicitly rather than assuming it by similarity.

That closes the track. The thread through all ten lessons: a real request, a real response, and an assertion that fails with the one specific field that’s wrong — from a first GET against a public sandbox to a suite CI runs on every pull request.

Turning OpenAPI into the schema you already know

Catching a breaking change before it ships

Running the same suite in CI

Practice

Frequently Asked Questions