Testing a GraphQL API: 200 OK Is Not a Pass

A GraphQL API answers almost everything with HTTP 200, including queries that failed. The errors are in the body, so the REST habit of asserting on the status code tests nothing. This lesson runs a real GraphQL server and builds up a pytest suite that checks what GraphQL actually reports: data, errors, and the schema itself.

The lab: a GraphQL server and a test container

The server is Strawberry 0.336.0, served by uvicorn 0.54.0. It has books, their authors and a price field whose backing service is down for one book, so one field can fail while the rest succeed.

# server.py - a small GraphQL API for books and authors, served by uvicorn on port 8000
import strawberry
from strawberry.asgi import GraphQL
from strawberry.extensions import QueryDepthLimiter

AUTHORS = {"1": "Ursula K. Le Guin", "2": "Terry Pratchett"}
BOOKS = [
    {"id": "1", "title": "A Wizard of Earthsea", "author_id": "1", "price": 9.99},
    {"id": "2", "title": "The Dispossessed", "author_id": "1", "price": 12.50},
    {"id": "3", "title": "Guards! Guards!", "author_id": "2", "price": None},  # the price service is down
]


@strawberry.type
class Author:
    id: strawberry.ID
    name: str

    @strawberry.field
    def books(self) -> list["Book"]:
        return [to_book(b) for b in BOOKS if b["author_id"] == self.id]


@strawberry.type
class Book:
    id: strawberry.ID
    title: str
    author_id: strawberry.Private[str]

    @strawberry.field
    def author(self) -> Author:
        return Author(id=self.author_id, name=AUTHORS[self.author_id])

    @strawberry.field
    def price(self) -> float | None:
        row = next(b for b in BOOKS if b["id"] == self.id)
        if row["price"] is None:
            raise ValueError(f"price service unavailable for book {self.id}")
        return row["price"]


def to_book(row):
    return Book(id=row["id"], title=row["title"], author_id=row["author_id"])


@strawberry.type
class Query:
    @strawberry.field
    def book(self, id: strawberry.ID) -> Book:
        for row in BOOKS:
            if row["id"] == id:
                return to_book(row)
        raise ValueError(f"no book with id {id}")

    @strawberry.field
    def books(self, limit: int = 10) -> list[Book]:
        return [to_book(b) for b in BOOKS[:limit]]


@strawberry.type
class Mutation:
    @strawberry.mutation
    def add_book(self, title: str, author_id: strawberry.ID) -> Book:
        if author_id not in AUTHORS:
            raise ValueError(f"no author with id {author_id}")
        row = {"id": str(len(BOOKS) + 1), "title": title, "author_id": author_id, "price": 0.0}
        BOOKS.append(row)
        return to_book(row)


schema = strawberry.Schema(query=Query, mutation=Mutation, extensions=[lambda: QueryDepthLimiter(max_depth=4)])
app = GraphQL(schema)

The depth limiter is passed as a factory (lambda: ...) so each request gets a fresh one. Passing an instance still works in Strawberry 0.336.0, but it prints a DeprecationWarning when the server starts.

One image holds the server and the test tools, pinned. The server runs in one container and pytest in a second, on a private Docker network where the server’s name is books.

FROM python:3.13-slim
RUN pip install --no-cache-dir "strawberry-graphql[asgi]==0.336.0" uvicorn==0.54.0 requests==2.34.2 pytest==9.1.1
WORKDIR /lab

# start.sh - build the image and (re)start the GraphQL API as "books" on a Docker network called gqllab
export MSYS_NO_PATHCONV=1  # Git Bash on Windows: don't rewrite the /lab paths
docker build -q -t gqllab . >/dev/null
docker rm -f books >/dev/null 2>&1
docker network create gqllab >/dev/null 2>&1
docker run -d --rm --name books --network gqllab -v "$PWD:/lab" gqllab \
  uvicorn server:app --host 0.0.0.0 --port 8000 >/dev/null
sleep 3
docker logs books 2>&1 | grep -i running

# test.sh [PYTEST ARGS...] - run pytest in a second container on the same network as the API
export MSYS_NO_PATHCONV=1
docker run --rm --network gqllab -v "$PWD:/lab" -e PYTHONDONTWRITEBYTECODE=1 gqllab \
  pytest -q -p no:cacheprovider "$@"

Save server.py, the Dockerfile and both scripts in one folder, and every test file below next to them. The data lives in memory, so sh start.sh also resets it.

$ sh start.sh
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

1. A GraphQL request is a POST with a JSON body

There’s one URL, /graphql. The query goes in the query field of a JSON body, and you name exactly the fields you want back.

# test_first_query.py - a GraphQL request is an HTTP POST with the query in a JSON body
import requests

URL = "http://books:8000/graphql"


def test_lists_book_titles():
    response = requests.post(URL, json={"query": "{ books { title } }"}, timeout=5)
    print(response.status_code, response.json())
    assert response.status_code == 200
    titles = [book["title"] for book in response.json()["data"]["books"]]
    assert titles == ["A Wizard of Earthsea", "The Dispossessed", "Guards! Guards!"]

$ sh test.sh -s test_first_query.py
200 {'data': {'books': [{'title': 'A Wizard of Earthsea'}, {'title': 'The Dispossessed'}, {'title': 'Guards! Guards!'}]}}
.
1 passed in 0.22s

The body has a data key whose shape mirrors the query: books, a list, and only title in each item, because that’s all the query asked for. (-s lets the test’s print through.) Section 4 adds books to the in-memory data, so run sh start.sh before re-running this test on its own later.

2. The trap: 200 OK on a failed query

Ask for a book that doesn’t exist, and check the response the way a REST test would.

# test_status_trap.py - the REST habit: check the status code. This test passes, and the query failed.
import requests

URL = "http://books:8000/graphql"


def test_missing_book_looks_fine_by_status_code():
    response = requests.post(URL, json={"query": '{ book(id: "99") { title } }'}, timeout=5)
    print(response.status_code, response.json())
    assert response.status_code == 200

$ sh test.sh -s test_status_trap.py
200 {'data': None, 'errors': [{'message': 'no book with id 99', 'locations': [{'line': 1, 'column': 3}], 'path': ['book']}]}
.
1 passed in 0.21s

The test passed. The query failed: data is None, and an errors entry says no book with id 99, points at line 1, column 3 of the query, and names the field that failed (path: ['book']). A resolver that raises is a field error, and GraphQL reports field errors in the body with status 200. Don’t rely on the status code.

The fix is a helper that every test uses, which looks inside the body.

# gql_client.py - send a GraphQL operation; fail on transport errors AND on GraphQL errors in the body
import requests

URL = "http://books:8000/graphql"


def post(query, variables=None):
    """Return the whole response body: {"data": ..., "errors": [...]}."""
    response = requests.post(URL, json={"query": query, "variables": variables or {}}, timeout=5)
    response.raise_for_status()  # catches 4xx/5xx, which GraphQL rarely sends
    return response.json()


def gql(query, variables=None):
    """Return only "data", and fail loudly if the server reported any errors."""
    body = post(query, variables)
    assert "errors" not in body, f"GraphQL errors: {body['errors']}"
    return body["data"]

# test_with_helper.py - the same missing book, checked through gql(), which looks inside the body
from gql_client import gql


def test_missing_book_is_caught():
    data = gql('{ book(id: "99") { title } }')
    assert data["book"]["title"]

$ sh test.sh test_with_helper.py
F                                                                        [100%]
=================================== FAILURES ===================================
_________________________ test_missing_book_is_caught __________________________

    def test_missing_book_is_caught():
>       data = gql('{ book(id: "99") { title } }')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

test_with_helper.py:6: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

query = '{ book(id: "99") { title } }', variables = None

    def gql(query, variables=None):
        """Return only "data", and fail loudly if the server reported any errors."""
        body = post(query, variables)
>       assert "errors" not in body, f"GraphQL errors: {body['errors']}"
               ^^^^^^^^^^^^^^^^^^^^
E       AssertionError: GraphQL errors: [{'message': 'no book with id 99', 'locations': [{'line': 1, 'column': 3}], 'path': ['book']}]

gql_client.py:17: AssertionError
=========================== short test summary info ============================
FAILED test_with_helper.py::test_missing_book_is_caught - AssertionError: Gra...
1 failed in 0.27s

Now it fails, and the failure shows the server’s own message. Without the helper, data["book"] would be None, so the next line would fail with a TypeError about subscripting None, which says nothing about why. Every test from here on uses gql() when it expects success, and post() when it wants to look at the errors itself.

3. Variables, not string formatting

Putting values into the query with an f-string breaks on quotes and invites injection. GraphQL has variables: the query declares $id with a type, and the values travel separately in variables.

# test_queries.py - variables instead of string formatting, and nested fields in one round trip
import pytest

from gql_client import gql

BOOK = """
query Book($id: ID!) {
  book(id: $id) { title author { name } }
}
"""


@pytest.mark.parametrize(
    "book_id, title, author",
    [("1", "A Wizard of Earthsea", "Ursula K. Le Guin"), ("3", "Guards! Guards!", "Terry Pratchett")],
)
def test_book_by_id(book_id, title, author):
    data = gql(BOOK, {"id": book_id})
    assert data["book"] == {"title": title, "author": {"name": author}}

$ sh test.sh test_queries.py
..                                                                       [100%]
2 passed in 0.22s

One query text, two cases. Each request also fetched the author through the book, which in REST would usually be a second request. Comparing the whole data["book"] dict, not one field, also proves the server returned nothing you didn’t ask for.

4. Mutations: prove the write by reading it back

A mutation’s response shows what the server says it did. A separate query shows what it actually stored.

# test_mutation.py - a write is only proven by reading it back through a separate query
from gql_client import gql, post

ADD = """
mutation Add($title: String!, $authorId: ID!) {
  addBook(title: $title, authorId: $authorId) { id title }
}
"""
READ = "query Read($id: ID!) { book(id: $id) { title author { name } } }"


def test_added_book_can_be_read_back():
    added = gql(ADD, {"title": "Mort", "authorId": "2"})["addBook"]
    read = gql(READ, {"id": added["id"]})["book"]
    assert read == {"title": "Mort", "author": {"name": "Terry Pratchett"}}


def test_unknown_author_is_rejected():
    body = post(ADD, {"title": "Ghost", "authorId": "42"})
    assert body["data"] is None
    assert body["errors"][0]["message"] == "no author with id 42"
    assert body["errors"][0]["path"] == ["addBook"]

$ sh test.sh test_mutation.py
..                                                                       [100%]
2 passed in 0.28s

The Python method add_book with argument author_id became addBook(authorId:) in the schema: Strawberry turns snake_case names into camelCase. The second test is a negative test, so it uses post() and asserts on the exact error and its path. Each run adds another “Mort” to the in-memory data, which is why the tests look books up by the returned id instead of counting them.

5. Partial data: data and errors together

One book’s price can’t be fetched. Ask for every price and see what comes back.

# test_partial.py - one field fails, the rest of the answer still arrives: data AND errors in one body
from gql_client import gql, post

PRICES = "{ books(limit: 3) { title price } }"


def test_prices_through_the_strict_helper():
    gql(PRICES)


def test_partial_data_names_the_failed_field():
    body = post(PRICES)
    books = body["data"]["books"]
    assert [b["price"] for b in books] == [9.99, 12.5, None]
    [error] = body["errors"]
    assert error["path"] == ["books", 2, "price"]
    assert error["message"] == "price service unavailable for book 3"

$ sh test.sh test_partial.py
F.                                                                       [100%]
=================================== FAILURES ===================================
____________________ test_prices_through_the_strict_helper _____________________

    def test_prices_through_the_strict_helper():
>       gql(PRICES)

test_partial.py:8: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

query = '{ books(limit: 3) { title price } }', variables = None

    def gql(query, variables=None):
        """Return only "data", and fail loudly if the server reported any errors."""
        body = post(query, variables)
>       assert "errors" not in body, f"GraphQL errors: {body['errors']}"
               ^^^^^^^^^^^^^^^^^^^^
E       AssertionError: GraphQL errors: [{'message': 'price service unavailable for book 3', 'locations': [{'line': 1, 'column': 27}], 'path': ['books', 2, 'price']}]

gql_client.py:17: AssertionError
=========================== short test summary info ============================
FAILED test_partial.py::test_prices_through_the_strict_helper - AssertionErro...
1 failed, 1 passed in 0.25s

The response had both keys. data held all three books with the third price set to null, and errors named the exact spot: ['books', 2, 'price'] is the price of item 2 (the third book) of books. The strict helper fails, which is what you want in most tests: a client that ignores errors would show this book as free, or crash. The second test is how you test a partial failure on purpose: assert on the null, the path and the message.

The field can only be null because the schema declares it nullable (float | None, Float in the schema). If a non-nullable field fails, the null moves up to the nearest nullable parent. Declared as -> float (Float!), the failed price would make the whole book null; a book can’t be null inside [Book!]!, and books can’t be null either, so the null would reach the top and data itself would be null, with the same single error.

6. Requests refused before anything runs

GraphQL checks every request against the schema before it runs a single resolver: unknown fields and variables of the wrong type are rejected.

# test_rejected.py - requests the server refuses before running any resolver
import requests

from gql_client import URL

BOOKS = "query Books($limit: Int!) { books(limit: $limit) { title } }"


def send(query, variables=None):
    response = requests.post(URL, json={"query": query, "variables": variables or {}}, timeout=5)
    return response.status_code, response.json()


def test_wrong_variable_type():
    status, body = send(BOOKS, {"limit": "ten"})
    assert status == 200  # still 200: refused requests are reported in the body too
    assert body["data"] is None
    assert body["errors"][0]["message"] == (
        "Variable '$limit' has invalid value: Int cannot represent non-integer value: 'ten'"
    )


def test_unknown_field():
    status, body = send("{ books { title isbn } }")
    assert status == 200
    assert body["data"] is None
    assert body["errors"][0]["message"] == "Cannot query field 'isbn' on type 'Book'."
    assert body["errors"][0]["locations"] == [{"line": 1, "column": 17}]

$ sh test.sh test_rejected.py
..                                                                       [100%]
2 passed in 0.21s

Both refusals came back as 200 too, with data set to None and no path, since no field ever ran. The locations entry points at column 17, where isbn starts in the query text. These tests pin the API’s input rules: if someone loosened limit to accept strings, test_wrong_variable_type would fail.

These are request errors, not field errors, and the GraphQL-over-HTTP spec treats them differently when the client asks for its newer media type, application/graphql-response+json: the server should then answer a request that fails to parse or validate with a 4xx status and no data, while field errors (sections 2 and 5) stay 200. Here’s what this server does with that Accept header.

# test_accept_header.py - ask for the newer GraphQL-over-HTTP media type and see what a refused query gets
import requests

from gql_client import URL


def test_invalid_query_with_graphql_response_media_type():
    response = requests.post(
        URL,
        json={"query": "{ books { title isbn } }"},
        headers={"Accept": "application/graphql-response+json"},
        timeout=5,
    )
    print(response.status_code, response.headers["content-type"], response.json()["data"])
    assert response.status_code == 200

$ sh test.sh -s test_accept_header.py
200 application/json None
.
1 passed in 0.36s

Strawberry 0.336.0 answered 200, in application/json, with data set to None: it didn’t switch to the newer media type or to a 4xx status. Another server, or a later Strawberry, may behave differently, which is one more reason the tests check errors in the body and not the status code.

7. The schema is the contract: snapshot it

A GraphQL server describes itself. An introspection query returns the whole schema, and graphql-core (installed with Strawberry) can print it as SDL, the readable schema language. Save that once, then fail when it changes.

# test_schema.py - the schema is the API's contract: compare the live one with a saved copy
import difflib
from pathlib import Path

import pytest
from graphql import build_client_schema, get_introspection_query, print_schema

from gql_client import gql

SNAPSHOT = Path(__file__).with_name("schema.graphql")


def live_schema():
    return print_schema(build_client_schema(gql(get_introspection_query()))) + "\n"


def test_schema_matches_snapshot():
    if not SNAPSHOT.exists():
        SNAPSHOT.write_text(live_schema())
        pytest.skip("no snapshot yet: saved schema.graphql, review it and commit it")
    diff = difflib.unified_diff(
        SNAPSHOT.read_text().splitlines(keepends=True),
        live_schema().splitlines(keepends=True),
        "schema.graphql (saved)",
        "live server",
    )
    changes = "".join(diff)
    assert not changes, "the schema changed:\n" + changes

$ sh test.sh test_schema.py -rs
s                                                                        [100%]
=========================== short test summary info ============================
SKIPPED [1] test_schema.py:20: no snapshot yet: saved schema.graphql, review it and commit it
1 skipped in 0.31s
$ cat schema.graphql
type Query {
  book(id: ID!): Book!
  books(limit: Int! = 10): [Book!]!
}

type Book {
  id: ID!
  title: String!
  author: Author!
  price: Float
}

type Author {
  id: ID!
  name: String!
  books: [Book!]!
}

type Mutation {
  addBook(title: String!, authorId: ID!): Book!
}
$ sh test.sh test_schema.py
.                                                                        [100%]
1 passed in 0.36s

The first run saved the snapshot and skipped, so a missing file can’t pass by accident (-rs prints the skip reason). The second run compared and passed. price: Float has no !, so it can be null, and author_id isn’t there at all, because strawberry.Private keeps it out of the schema.

Now make a change that looks harmless on the server: rename the author’s name field.

 class Author:
     id: strawberry.ID
-    name: str
+    full_name: str
...
     def author(self) -> Author:
-        return Author(id=self.author_id, name=AUTHORS[self.author_id])
+        return Author(id=self.author_id, full_name=AUTHORS[self.author_id])

$ sh start.sh
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
$ sh test.sh test_schema.py
F                                                                        [100%]
=================================== FAILURES ===================================
_________________________ test_schema_matches_snapshot _________________________

    def test_schema_matches_snapshot():
        if not SNAPSHOT.exists():
            SNAPSHOT.write_text(live_schema())
            pytest.skip("no snapshot yet: saved schema.graphql, review it and commit it")
        diff = difflib.unified_diff(
            SNAPSHOT.read_text().splitlines(keepends=True),
            live_schema().splitlines(keepends=True),
            "schema.graphql (saved)",
            "live server",
        )
        changes = "".join(diff)
>       assert not changes, "the schema changed:\n" + changes
E       AssertionError: the schema changed:
E         --- schema.graphql (saved)
E         +++ live server
E         @@ -12,7 +12,7 @@
E          
E          type Author {
E            id: ID!
E         -  name: String!
E         +  fullName: String!
E            books: [Book!]!
E          }
E          
E         
E       assert not '--- schema.graphql (saved)\n+++ live server\n@@ -12,7 +12,7 @@\n \n type Author {\n   id: ID!\n-  name: String!\n+  fullName: String!\n   books: [Book!]!\n }\n \n'

test_schema.py:28: AssertionError
=========================== short test summary info ============================
FAILED test_schema.py::test_schema_matches_snapshot - AssertionError: the sch...
1 failed in 0.38s

The diff shows the break exactly: every client that asks for author { name } would now get a validation error. In a pull request, this test makes the change visible: either rename it back, or keep name alongside fullName and mark it deprecated, then update the snapshot on purpose. It’s the GraphQL version of the OpenAPI contract check earlier in this track. Undo the rename and run sh start.sh before going on.

8. A depth limit refuses a query that nests too far

Authors have books and books have authors, so a client can nest them as deep as it likes, and each level multiplies the work. The server was built with QueryDepthLimiter(max_depth=4). Test that it’s really on.

# test_depth.py - a query nested deeper than the server allows is refused before it runs
from gql_client import gql, post

ALLOWED = "{ books { author { books { author { name } } } } }"  # 5 field levels: the most max_depth=4 allows
TOO_DEEP = "{ books { author { books { author { books { title } } } } } }"  # 6 field levels


def test_five_levels_are_allowed():
    books = gql(ALLOWED)["books"]
    assert books[0]["author"]["books"][0]["author"]["name"] == "Ursula K. Le Guin"


def test_six_levels_are_refused():
    body = post(TOO_DEEP)
    print(body)
    assert body["data"] is None
    assert body["errors"][0]["message"] == "'anonymous' exceeds maximum operation depth of 4"

$ sh test.sh -s test_depth.py
.{'data': None, 'errors': [{'message': "'anonymous' exceeds maximum operation depth of 4", 'locations': [{'line': 1, 'column': 45}]}]}
.
2 passed in 0.28s

max_depth=4 allowed five levels of fields, not four. Strawberry’s limiter counts the levels below the top-level field: books is level 0, so { books { author { books { author { name } } } } } is depth 4 and passes, and the six-level query is depth 5 and is refused before any resolver runs. That’s easy to get wrong, which is why the allowed query sits exactly on the limit. With max_depth=3, the same five-level query was refused (exceeds maximum operation depth of 3), so the first test would catch a limit set one lower, and the second catches one that’s missing or set higher. The message says 'anonymous' because the query has no operation name; the same query written as query Deep { ... } was refused as 'Deep' exceeds maximum operation depth of 4.

Across three full runs in this order, every output above was the same apart from the timings (0.19 to 0.72 s).

Common gotchas

• Asserting on the status code. Failed, partial and refused queries all returned 200 here. Check errors.
• Reading data without checking errors. A None or a null field turns into a confusing TypeError later. Fail with the server’s message instead (the gql() helper).
• Building queries with f-strings. Use variables: they’re typed, validated, and can’t break the query’s quoting.
• Forgetting the camelCase. Strawberry and most servers expose add_book as addBook. Test against the schema, not the Python names.
• Non-null fields that fail. The null moves up to the nearest nullable parent, so one failing non-null field can null a whole list, or all of data (section 5). Test which fields are nullable (the snapshot shows every !).
• Shared state between tests. Mutations change the server’s data. Restart it or look things up by returned IDs.

Practice

Start the server with sh start.sh and add each test file next to the others.

1. One GraphQL refusal in this lab does come back as a real HTTP error: a mutation sent with GET instead of POST. Write a test that sends addBook as a GET request and asserts the status code and body.

# test_get_mutation.py - the one refusal in this lab that does arrive as an HTTP status code
import requests

from gql_client import URL


def test_mutation_over_get_is_a_400():
    query = 'mutation { addBook(title: "Sneaky", authorId: "1") { id } }'
    response = requests.get(URL, params={"query": query}, timeout=5)
    assert response.status_code == 400
    assert response.text == "mutations are not allowed when using GET"

$ sh test.sh test_get_mutation.py
.                                                                        [100%]
1 passed in 0.24s

2. Fetch books 1 and 3 in a single request. Both use the book field, so you’ll need aliases to tell the answers apart.

# test_aliases.py - two lookups of the same field in one request, told apart by aliases
from gql_client import gql


def test_two_books_in_one_request():
    data = gql('{ first: book(id: "1") { title } third: book(id: "3") { title } }')
    assert data == {"first": {"title": "A Wizard of Earthsea"}, "third": {"title": "Guards! Guards!"}}

$ sh test.sh test_aliases.py
.                                                                        [100%]
1 passed in 0.29s

3. Add a new nullable field to Book: year: int | None = None, on the line after author_id. Restart the server and run test_schema.py. Adding a field doesn’t break any existing client, so write a second schema test that fails only on changes that can break one. Then check it really catches a removal: hide Author.id from the schema by changing its type to strawberry.Private[str], and restart.

# test_schema_breaking.py - fail only on changes that can break an existing client; additions pass
from pathlib import Path

from graphql import build_client_schema, build_schema, find_breaking_changes, get_introspection_query

from gql_client import gql

SNAPSHOT = Path(__file__).with_name("schema.graphql")


def test_no_breaking_changes():
    saved = build_schema(SNAPSHOT.read_text())
    live = build_client_schema(gql(get_introspection_query()))
    breaking = [f"{change.type.name}: {change.description}" for change in find_breaking_changes(saved, live)]
    assert not breaking, breaking

$ sh start.sh
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
$ sh test.sh test_schema.py test_schema_breaking.py
F.                                                                       [100%]
=================================== FAILURES ===================================
_________________________ test_schema_matches_snapshot _________________________

    def test_schema_matches_snapshot():
        if not SNAPSHOT.exists():
            SNAPSHOT.write_text(live_schema())
            pytest.skip("no snapshot yet: saved schema.graphql, review it and commit it")
        diff = difflib.unified_diff(
            SNAPSHOT.read_text().splitlines(keepends=True),
            live_schema().splitlines(keepends=True),
            "schema.graphql (saved)",
            "live server",
        )
        changes = "".join(diff)
>       assert not changes, "the schema changed:\n" + changes
E       AssertionError: the schema changed:
E         --- schema.graphql (saved)
E         +++ live server
E         @@ -6,6 +6,7 @@
E          type Book {
E            id: ID!
E            title: String!
E         +  year: Int
E            author: Author!
E            price: Float
E          }
E         
E       assert not '--- schema.graphql (saved)\n+++ live server\n@@ -6,6 +6,7 @@\n type Book {\n   id: ID!\n   title: String!\n+  year: Int\n   author: Author!\n   price: Float\n }\n'

test_schema.py:28: AssertionError
=========================== short test summary info ============================
FAILED test_schema.py::test_schema_matches_snapshot - AssertionError: the sch...
1 failed, 1 passed in 0.71s

$ sh start.sh
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
$ sh test.sh test_schema_breaking.py
F                                                                        [100%]
=================================== FAILURES ===================================
___________________________ test_no_breaking_changes ___________________________

    def test_no_breaking_changes():
        saved = build_schema(SNAPSHOT.read_text())
        live = build_client_schema(gql(get_introspection_query()))
        breaking = [f"{change.type.name}: {change.description}" for change in find_breaking_changes(saved, live)]
>       assert not breaking, breaking
E       AssertionError: ['FIELD_REMOVED: Field Author.id was removed.']
E       assert not ['FIELD_REMOVED: Field Author.id was removed.']

test_schema_breaking.py:15: AssertionError
=========================== short test summary info ============================
FAILED test_schema_breaking.py::test_no_breaking_changes - AssertionError: ['...
1 failed in 0.72s