Skip to main content
FastAPI · lesson 19 of 19

Beginner

6

Intermediate

10

Advanced

3

Also part of FastAPI

6
Open the FastAPI page →
FastAPI advanced Lesson 19 of 19

Also part of: Microservices Docker Kubernetes

Deploying FastAPI with Docker and Kubernetes: Images, Signals, and Probes

Ship FastAPI on Docker and Kubernetes: a 171 MB multi-stage image, clean SIGTERM shutdown, and liveness, readiness and startup probes on a live minikube.

By SkillByExample · Updated

fastapi docker kubernetes deployment uvicorn

What This Covers

FastAPI’s documentation explains deployment concepts but doesn’t take an app through a real cluster. This lesson does. It builds a production-shaped image and measures it against the naive one, shows what process 1 inside the container means for shutdown, then deploys to a local Kubernetes cluster and breaks it twice on purpose: once with a blocked event loop (the bug from the sync vs async lesson) and once with a slow startup. Each time, you read what the probes did from the cluster’s own events.

The tools are Docker 29.8.2, minikube v1.38.1 running Kubernetes v1.35.1 with the Docker driver, and kubectl v1.35.2. The app pins fastapi 0.143.0 and uvicorn 0.54.0. The Docker and Kubernetes tracks cover the platforms themselves.

The App

main.py has a root route that reports which pod answered, a health route for the probes, an optional startup delay standing in for real warmup work, and a route that deliberately blocks the event loop:

import asyncio
import os
import socket
import time
from contextlib import asynccontextmanager
from fastapi import FastAPI

@asynccontextmanager
async def lifespan(app: FastAPI):
    warmup = float(os.environ.get("WARMUP_SECONDS", "0"))
    await asyncio.sleep(warmup)            # stands in for loading a model or opening a DB pool
    print(f"warm after {warmup:.0f}s", flush=True)
    yield

app = FastAPI(lifespan=lifespan)

@app.get("/")
async def root():
    return {"pod": socket.gethostname(), "version": os.environ.get("APP_VERSION", "dev")}

@app.get("/healthz")                       # liveness: can this process still answer?
async def healthz():
    return {"status": "ok"}

@app.post("/debug/block")                  # the bug from the sync-vs-async lesson, on purpose
async def block(seconds: float = 30):
    time.sleep(seconds)                    # blocks the event loop: nothing else is answered
    return {"blocked_for": seconds}

requirements.txt pins exact versions, fastapi==0.143.0 and uvicorn[standard]==0.54.0, one per line, so every image build installs the same code. A .dockerignore keeps the Dockerfiles, YAML and __pycache__ out of the build context.

The Naive Image

This is the Dockerfile most tutorials start with, saved as Dockerfile.naive:

FROM python:3.12
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD uvicorn main:app --host 0.0.0.0 --port 8000

Build it with docker build -f Dockerfile.naive -t fastapi-demo:naive ., run it on host port 8080, and ask the container who it runs as and what its process 1 is:

$ docker run -d --name demo -p 8080:8000 fastapi-demo:naive
b331ae6f8a97ba29a9184c644426b110896a95c321ea9f8bb4a8924ef301fec7
$ curl -s 127.0.0.1:8080/
{"pod":"b331ae6f8a97","version":"dev"}
$ docker exec demo sh -c 'id -un; tr "\0" " " < /proc/1/cmdline; echo'
root
/bin/sh -c uvicorn main:app --host 0.0.0.0 --port 8000

It works, which is why it gets shipped. Inside a plain container the hostname, and so the pod field, is the first 12 characters of the container ID. Two things are wrong already. The server runs as root, so a remote-code-execution bug in any dependency gets root inside the container. And process 1 is /bin/sh -c ..., not uvicorn, because shell-form CMD wraps the command in a shell.

The Failure: A Container That Can’t Stop

docker stop sends SIGTERM to process 1, waits for the timeout given with -t, then sends SIGKILL. Stop the naive container with a 10-second timeout and remove it:

$ time docker stop -t 10 demo
demo

real	0m10.834s
user	0m0.061s
sys	0m0.045s
$ docker inspect demo --format "{{.State.ExitCode}}"
137
$ docker rm demo
demo

The container ignored SIGTERM for the whole 10 seconds and was killed: exit code 137 is 128 + 9, SIGKILL. /bin/sh was process 1, and it neither handled SIGTERM nor passed it on to uvicorn. Process 1 in a Linux namespace also ignores any signal it has no handler for, so the shell didn’t even exit itself. Uvicorn never ran its shutdown, so in-flight requests were cut off and the lifespan cleanup after yield (closing a DB pool, flushing a queue) never ran. On Kubernetes this happens on every rollout, scale-down and node drain, with a default grace period of 30 seconds.

A Multi-Stage, Non-Root Image

The production Dockerfile installs dependencies into a virtual environment in one stage and copies only that venv into a clean slim image:

# --- build stage: has pip and build tools, never shipped
FROM python:3.12-slim AS build
RUN python -m venv /venv
COPY requirements.txt .
RUN /venv/bin/pip install --no-cache-dir -r requirements.txt

# --- runtime stage: the venv and the code, nothing else
FROM python:3.12-slim
COPY --from=build /venv /venv
WORKDIR /app
COPY main.py .
RUN useradd --uid 10001 --no-create-home app
USER 10001
ENV PATH=/venv/bin:$PATH PYTHONUNBUFFERED=1
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Build it with docker build -t fastapi-demo:1.0 ., then compare the images and run the same checks, including the stop:

$ docker images fastapi-demo
IMAGE                ID             DISK USAGE   CONTENT SIZE   EXTRA
fastapi-demo:1.0     8ddd4fed894a        171MB             0B        
fastapi-demo:naive   6b31a3e5b26c       1.17GB             0B        
$ docker run -d --name demo -p 8080:8000 fastapi-demo:1.0
6372a4a08559af2e520eed4da49ef3941b6d099e159f3a2a90211d290ba0864a
$ curl -s 127.0.0.1:8080/
{"pod":"6372a4a08559","version":"dev"}
$ docker exec demo sh -c 'id -un; tr "\0" " " < /proc/1/cmdline; echo'
app
/venv/bin/python /venv/bin/uvicorn main:app --host 0.0.0.0 --port 8000 
$ time docker stop -t 10 demo
demo

real	0m0.965s
user	0m0.015s
sys	0m0.107s
$ docker inspect demo --format "{{.State.ExitCode}}"
0
$ docker logs demo 2>&1 | tail -4
INFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [1]
$ docker rm demo
demo

The image is 171 MB instead of 1.17 GB, nearly 7× smaller, so it pulls faster on every new node and has fewer packages to patch. The full python:3.12 base brings compilers and development headers this app never uses. Only the second FROM ships, so pip’s cache and anything built in the first stage stay behind. The process runs as app (uid 10001), and process 1 is uvicorn itself, because exec-form CMD ["uvicorn", ...] runs it directly with no shell in between. So uvicorn received SIGTERM, logged a clean shutdown and exited 0 in under a second, instead of being killed after 10. Copying requirements.txt before the code also lets Docker cache the dependency layer, so editing main.py doesn’t reinstall FastAPI.

Deploy to Kubernetes

Start a local cluster and copy the image into it, so nothing has to be pushed to a registry:

minikube start -p fapi --driver=docker --cpus=4 --memory=4096
minikube -p fapi image load fastapi-demo:1.0
$ minikube -p fapi image ls | grep fastapi
docker.io/library/fastapi-demo:1.0

deployment.yaml runs two replicas with both probes pointed at /healthz:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fastapi-demo
spec:
  replicas: 2
  selector:
    matchLabels: {app: fastapi-demo}
  template:
    metadata:
      labels: {app: fastapi-demo}
    spec:
      containers:
        - name: api
          image: fastapi-demo:1.0
          imagePullPolicy: Never          # use the image loaded into minikube
          ports: [{containerPort: 8000}]
          env: [{name: APP_VERSION, value: "1.0"}]
          resources:
            requests: {cpu: 100m, memory: 64Mi}
            limits: {memory: 256Mi}
          readinessProbe:                 # failing: removed from the Service, not restarted
            httpGet: {path: /healthz, port: 8000}
            periodSeconds: 2
          livenessProbe:                  # failing 3 times in a row: container restarted
            httpGet: {path: /healthz, port: 8000}
            periodSeconds: 5
            timeoutSeconds: 1
            failureThreshold: 3

service.yaml gives the pods one stable name inside the cluster:

apiVersion: v1
kind: Service
metadata:
  name: fastapi-demo
spec:
  selector: {app: fastapi-demo}
  ports:
    - port: 80
      targetPort: 8000

Apply both, wait for the rollout, and call the Service six times from a throwaway curl pod inside the cluster:

$ kubectl apply -f deployment.yaml -f service.yaml
deployment.apps/fastapi-demo created
service/fastapi-demo created
$ kubectl rollout status deployment/fastapi-demo --timeout=120s
Waiting for deployment "fastapi-demo" rollout to finish: 0 of 2 updated replicas are available...
Waiting for deployment "fastapi-demo" rollout to finish: 1 of 2 updated replicas are available...
deployment "fastapi-demo" successfully rolled out
$ kubectl get pods -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP            NODE   NOMINATED NODE   READINESS GATES
fastapi-demo-6d4cf99857-7pb42   1/1     Running   0          2s    10.244.0.13   fapi   <none>           <none>
fastapi-demo-6d4cf99857-dwktq   1/1     Running   0          2s    10.244.0.12   fapi   <none>           <none>
$ kubectl run client -q --rm -i --restart=Never --image=curlimages/curl -- sh -c 'for i in 1 2 3 4 5 6; do curl -s fastapi-demo/; echo; done'
{"pod":"fastapi-demo-6d4cf99857-dwktq","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-7pb42","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-7pb42","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-7pb42","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-7pb42","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-7pb42","version":"1.0"}

Both pods are 1/1 ready, and curl fastapi-demo/ resolved through cluster DNS to the Service, which spread the requests across both. The split is uneven (one to dwktq, five to 7pb42) because kube-proxy picks a backend at random for each new connection, so six requests are too few to look balanced. The pod name in each response is the hostname Kubernetes gave the container, which shows that the code is reporting from inside the pod.

The Failure: A Blocked Event Loop

/debug/block makes one pod do what the sync vs async lesson warned about: time.sleep(60) inside async def. Call it on one pod’s IP with a 3-second client timeout, then look at the Service’s endpoints:

$ kubectl run blocker -q --rm -i --restart=Never --image=curlimages/curl -- curl -s -m 3 -X POST '10.244.0.13:8000/debug/block?seconds=60'
pod default/blocker terminated (Error)
$ kubectl get endpointslices -l kubernetes.io/service-name=fastapi-demo -o jsonpath='{range .items[*].endpoints[*]}{.addresses[0]}  ready={.conditions.ready}  {.targetRef.name}{"\n"}{end}'
10.244.0.13  ready=false  fastapi-demo-6d4cf99857-7pb42
10.244.0.12  ready=true  fastapi-demo-6d4cf99857-dwktq
$ kubectl get pods
NAME                            READY   STATUS    RESTARTS   AGE
fastapi-demo-6d4cf99857-7pb42   0/1     Running   0          22s
fastapi-demo-6d4cf99857-dwktq   1/1     Running   0          22s

The blocker pod reports Error because its curl gave up after 3 seconds, as intended. The server is still sleeping. Within a few seconds the readiness probe failed, and the EndpointSlice marked 7pb42 as ready=false, so the Service stopped sending it traffic while dwktq carried everything. The plain kubectl get endpointslices table still lists both IPs, which is why the jsonpath above prints each endpoint’s ready condition. Wait until the pod is ready again, then read its events:

$ kubectl get pods
NAME                            READY   STATUS    RESTARTS     AGE
fastapi-demo-6d4cf99857-7pb42   1/1     Running   1 (2s ago)   58s
fastapi-demo-6d4cf99857-dwktq   1/1     Running   0            58s
$ kubectl describe pod fastapi-demo-6d4cf99857-7pb42 | sed -n '/^Events:/,$p'
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  58s                default-scheduler  Successfully assigned default/fastapi-demo-6d4cf99857-7pb42 to fapi
  Warning  Unhealthy  58s                kubelet            spec.containers{api}: Readiness probe failed: Get "http://10.244.0.13:8000/healthz": dial tcp 10.244.0.13:8000: connect: connection refused
  Warning  Unhealthy  32s (x3 over 42s)  kubelet            spec.containers{api}: Liveness probe failed: Get "http://10.244.0.13:8000/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  Normal   Killing    32s                kubelet            spec.containers{api}: Container api failed liveness probe, will be restarted
  Warning  Unhealthy  7s (x21 over 46s)  kubelet            spec.containers{api}: Readiness probe failed: Get "http://10.244.0.13:8000/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  Normal   Pulled     2s (x2 over 58s)   kubelet            spec.containers{api}: Container image "fastapi-demo:1.0" already present on machine and can be accessed by the pod
  Normal   Created    2s (x2 over 58s)   kubelet            spec.containers{api}: Container created
  Normal   Started    2s (x2 over 58s)   kubelet            spec.containers{api}: Container started
$ kubectl logs fastapi-demo-6d4cf99857-7pb42 --previous | tail -3
INFO:     10.244.0.1:38504 - "GET /healthz HTTP/1.1" 200 OK
INFO:     10.244.0.1:38514 - "GET /healthz HTTP/1.1" 200 OK
INFO:     10.244.0.1:38524 - "GET /healthz HTTP/1.1" 200 OK

Read the events from the top. The first readiness failure, connection refused, is from the pod’s first second, before uvicorn was listening. Then come three liveness failures with context deadline exceeded: the port was open, but the blocked loop couldn’t answer within timeoutSeconds: 1. After the third, the kubelet logged Killing, which means it sent SIGTERM. Note the timing: Killing was 32 seconds ago but Started only 2 seconds ago. Uvicorn received the SIGTERM, and its handler ran and set its exit flag, but the shutdown itself runs on the event loop, and time.sleep was holding the loop. So the container received SIGTERM but couldn’t act on it for the full 30-second terminationGracePeriodSeconds, and was SIGKILLed. The previous container’s log ends on ordinary health checks, with no Shutting down line. One blocking call took that pod out of service for about 45 seconds. Without the readiness probe, the Service would have kept sending it requests the whole time.

The Failure: Killed While Starting

Real apps do work before serving: loading a model, warming a cache, running migrations. Release version 1.1 with a 20-second warmup and watch the rollout:

$ kubectl set env deployment/fastapi-demo WARMUP_SECONDS=20 APP_VERSION=1.1
deployment.apps/fastapi-demo env updated
$ kubectl rollout status deployment/fastapi-demo --timeout=90s
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 2 new replicas have been updated...
error: timed out waiting for the condition
$ kubectl get pods
NAME                            READY   STATUS    RESTARTS      AGE
fastapi-demo-6d4cf99857-7pb42   1/1     Running   1 (93s ago)   2m29s
fastapi-demo-6d4cf99857-dwktq   1/1     Running   0             2m29s
fastapi-demo-85689f9f45-xknhj   0/1     Running   4 (5s ago)    90s
$ kubectl describe pod fastapi-demo-85689f9f45-xknhj | sed -n '/^Events:/,$p'
Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Scheduled  91s                 default-scheduler  Successfully assigned default/fastapi-demo-85689f9f45-xknhj to fapi
  Warning  Unhealthy  58s (x19 over 90s)  kubelet            spec.containers{api}: Readiness probe failed: Get "http://10.244.0.16:8000/healthz": dial tcp 10.244.0.16:8000: connect: connection refused
  Warning  Unhealthy  56s (x6 over 86s)   kubelet            spec.containers{api}: Liveness probe failed: Get "http://10.244.0.16:8000/healthz": dial tcp 10.244.0.16:8000: connect: connection refused
  Normal   Killing    16s (x4 over 76s)   kubelet            spec.containers{api}: Container api failed liveness probe, will be restarted
  Normal   Pulled     6s (x5 over 91s)    kubelet            spec.containers{api}: Container image "fastapi-demo:1.0" already present on machine and can be accessed by the pod
  Normal   Created    6s (x5 over 91s)    kubelet            spec.containers{api}: Container created
  Normal   Started    6s (x5 over 91s)    kubelet            spec.containers{api}: Container started
$ kubectl logs fastapi-demo-85689f9f45-xknhj --previous
INFO:     Started server process [1]
INFO:     Waiting for application startup.
warm after 20s
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [1]

The new pod restarted 4 times in 90 seconds and never became ready. Uvicorn doesn’t open its port until the lifespan startup finishes, so every probe during the 20-second warmup got connection refused. The liveness window is three failures 5 seconds apart, about 15 seconds, which is shorter than the warmup. The log shows the timing: the kubelet sent SIGTERM during the warmup, uvicorn finished starting at 20 seconds, printed Uvicorn running, and shut down straight away without serving a request. kubectl set env changed only the environment, which is why the events still name the fastapi-demo:1.0 image. The newest probe event is 56 seconds old even though the kills continued. That isn’t a gap in the failures: the kubelet’s event recorder rate-limits events per pod and event type, with a burst of 25. The 19 + 6 = 25 Warning probe events used it up, so later probe failures weren’t recorded, while the Normal events (Killing, Started) have their own budget and kept appearing.

The rollout itself protected users. A Deployment’s default rolling update keeps old pods until new ones are ready, so a client running during the failure still got answers from the two 1.0 pods:

$ kubectl run client -q --rm -i --restart=Never --image=curlimages/curl -- sh -c 'for i in 1 2 3 4; do curl -s fastapi-demo/; echo; done'
{"pod":"fastapi-demo-6d4cf99857-dwktq","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-dwktq","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-7pb42","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-7pb42","version":"1.0"}

Raising failureThreshold on the liveness probe would also let the pod start, but it would make a hung pod wait just as long before being restarted later on. A startupProbe gives startup its own budget: liveness and readiness don’t run until it has passed once. Put the new environment and the startup probe in deployment.yaml, so the file matches what is running again:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fastapi-demo
spec:
  replicas: 2
  selector:
    matchLabels: {app: fastapi-demo}
  template:
    metadata:
      labels: {app: fastapi-demo}
    spec:
      containers:
        - name: api
          image: fastapi-demo:1.0
          imagePullPolicy: Never
          ports: [{containerPort: 8000}]
          env:
            - {name: APP_VERSION, value: "1.1"}
            - {name: WARMUP_SECONDS, value: "20"}
          resources:
            requests: {cpu: 100m, memory: 64Mi}
            limits: {memory: 256Mi}
          startupProbe:                   # liveness and readiness wait until this passes once
            httpGet: {path: /healthz, port: 8000}
            periodSeconds: 2
            failureThreshold: 30          # allows up to 60 s to start
          readinessProbe:
            httpGet: {path: /healthz, port: 8000}
            periodSeconds: 2
          livenessProbe:
            httpGet: {path: /healthz, port: 8000}
            periodSeconds: 5
            timeoutSeconds: 1
            failureThreshold: 3
$ kubectl apply -f deployment.yaml
deployment.apps/fastapi-demo configured
$ time kubectl rollout status deployment/fastapi-demo --timeout=180s
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 2 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 2 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 2 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 2 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "fastapi-demo" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "fastapi-demo" rollout to finish: 1 old replicas are pending termination...
deployment "fastapi-demo" successfully rolled out

real	0m44.839s
user	0m0.015s
sys	0m0.031s
$ kubectl get pods
NAME                            READY   STATUS        RESTARTS   AGE
fastapi-demo-6d4cf99857-dwktq   1/1     Terminating   0          3m21s
fastapi-demo-7fc94b787b-85mbc   1/1     Running       0          23s
fastapi-demo-7fc94b787b-np59r   1/1     Running       0          45s
$ kubectl logs fastapi-demo-7fc94b787b-85mbc | head -5
INFO:     Started server process [1]
INFO:     Waiting for application startup.
warm after 20s
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
$ kubectl run client -q --rm -i --restart=Never --image=curlimages/curl -- sh -c 'for i in 1 2 3 4; do curl -s fastapi-demo/; echo; done'
{"pod":"fastapi-demo-7fc94b787b-np59r","version":"1.1"}
{"pod":"fastapi-demo-7fc94b787b-np59r","version":"1.1"}
{"pod":"fastapi-demo-7fc94b787b-np59r","version":"1.1"}
{"pod":"fastapi-demo-7fc94b787b-np59r","version":"1.1"}

Both 1.1 pods started with 0 restarts. The rollout took 44.8 seconds, about two warmups, because the rolling update replaced the pods one at a time (np59r is 45s old, 85mbc 23s). The last old pod is still Terminating, finishing its requests within the grace period. The liveness probe is unchanged, so a pod that hangs after startup is still restarted after about 15 seconds.

On worker count: the image runs one uvicorn process, and the Deployment scales by adding replicas. The sync vs async lesson measured --workers multiplying throughput on a single machine, but inside Kubernetes several workers in one container share a single probe, a single restart and a single resource request. One process per container keeps each of those per process.

CI: Test Before the Image Is Built

Every deploy above trusted that main.py worked. A CI job runs the tests on each push, before anyone builds an image. test_main.py covers the health route, the version field, and the lifespan startup that the probes depend on:

from fastapi.testclient import TestClient
from main import app

client = TestClient(app)

def test_healthz():
    r = client.get("/healthz")
    assert r.status_code == 200
    assert r.json() == {"status": "ok"}

def test_root_reports_version(monkeypatch):
    monkeypatch.setenv("APP_VERSION", "1.2")
    assert client.get("/").json()["version"] == "1.2"

def test_startup_runs(capsys):
    client.get("/healthz")
    assert "warm after 0s" in capsys.readouterr().out

The workflow lives at .github/workflows/test.yml. It’s shown as configuration only: it wasn’t run on GitHub for this lesson. The steps it runs were run locally below.

name: test
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: {python-version: "3.12", cache: pip}
      - run: pip install -r requirements-dev.txt
      - run: pytest -q

requirements-dev.txt holds -r requirements.txt followed by starlette==1.7.0, pytest==9.1.1 and httpx2==2.13.1, one per line. httpx2 is what TestClient uses to send requests. Starlette is pinned because fastapi==0.143.0 accepts a range of Starlette versions, and a fresh install picked 1.8.0. In a real project, put that pin in requirements.txt too, or use a lock file, so the image and the tests get the same versions. Run the same two steps in a fresh virtual environment:

$ pip install -q --disable-pip-version-check -r requirements-dev.txt
$ pytest -q
..F                                                                      [100%]
================================== FAILURES ===================================
______________________________ test_startup_runs ______________________________

capsys = <_pytest.capture.CaptureFixture object at 0x000001D8261A0860>

    def test_startup_runs(capsys):
        client.get("/healthz")
>       assert "warm after 0s" in capsys.readouterr().out
E       AssertionError: assert 'warm after 0s' in ''
E        +  where '' = CaptureResult(out='', err='').out
E        +    where CaptureResult(out='', err='') = readouterr()
E        +      where readouterr = <_pytest.capture.CaptureFixture object at 0x000001D8261A0860>.readouterr

test_main.py:17: AssertionError
=========================== short test summary info ===========================
FAILED test_main.py::test_startup_runs - AssertionError: assert 'warm after 0...
1 failed, 2 passed in 0.86s

The routes work, but the lifespan never ran. A TestClient created at module level sends requests without running startup or shutdown. The lifespan only runs when the client is used as a context manager. So these tests would pass even if the warmup code crashed, and the crash would only show up later as the startup restart loop from the previous section. The fix is a fixture that opens the client with with:

import pytest
from fastapi.testclient import TestClient
from main import app

@pytest.fixture
def client():
    with TestClient(app) as c:             # entering the block runs the lifespan
        yield c

def test_healthz(client):
    r = client.get("/healthz")
    assert r.status_code == 200
    assert r.json() == {"status": "ok"}

def test_root_reports_version(client, monkeypatch):
    monkeypatch.setenv("APP_VERSION", "1.2")
    assert client.get("/").json()["version"] == "1.2"

def test_startup_runs(capsys, client):
    assert "warm after 0s" in capsys.readouterr().out
$ pytest -q
...                                                                      [100%]
3 passed in 0.33s

Each test now gets a client whose startup ran, and the lifespan’s shutdown runs when the with block exits. In test_startup_runs, capsys comes before client on purpose: pytest sets fixtures up in that order, so output capture is already on when the lifespan prints. With client, capsys the test fails again with the same empty string. A failing pytest step fails the job, and GitHub marks the commit red, so a broken startup is caught before an image is built. The GitHub Actions track covers caching, matrices and deployment jobs.

Liveness vs Readiness: Checking a Dependency

Every probe above pointed at /healthz, which only proves the process can answer. A real app also needs its database. health.py adds a readiness route that opens a TCP connection to it:

import asyncio
import os
from fastapi import FastAPI
from fastapi.responses import JSONResponse

DB_HOST = os.environ.get("DB_HOST", "127.0.0.1")
DB_PORT = int(os.environ.get("DB_PORT", "5432"))
app = FastAPI()

@app.get("/healthz")                       # liveness: is this process answering?
async def healthz():
    return {"status": "ok"}

@app.get("/readyz")                        # readiness: can it reach what it needs?
async def readyz():
    try:
        conn = asyncio.open_connection(DB_HOST, DB_PORT)
        _, writer = await asyncio.wait_for(conn, timeout=1)
        writer.close()
        await writer.wait_closed()
    except (OSError, TimeoutError) as exc:
        return JSONResponse({"status": "unavailable", "db": type(exc).__name__},
                            status_code=503)
    return {"status": "ok", "db": f"{DB_HOST}:{DB_PORT}"}

Run it in a container on port 8011 with no database yet:

docker run -d --name fapi-api -p 8011:8011 -v "$PWD":/app -w /app python:3.12-slim \
  sh -c "pip install -q fastapi==0.143.0 starlette==1.7.0 uvicorn==0.54.0 && uvicorn health:app --host 0.0.0.0 --port 8011"
$ curl -s -w ' %{http_code}\n' localhost:8011/healthz
{"status":"ok"} 200
$ curl -s -w ' %{http_code}\n' localhost:8011/readyz
{"status":"unavailable","db":"ConnectionRefusedError"} 503

The process is alive but can’t do its job. Next, start Postgres inside the app container’s network namespace, so it listens on the app’s 127.0.0.1:5432. Wait until docker exec fapi-db pg_isready -h 127.0.0.1 succeeds (about 2 seconds here), then check readiness, stop the database, and check both routes again:

docker run -d --name fapi-db --network container:fapi-api -e POSTGRES_PASSWORD=dev postgres:17
$ curl -s -w ' %{http_code}\n' localhost:8011/readyz
{"status":"ok","db":"127.0.0.1:5432"} 200
$ docker stop fapi-db
fapi-db
$ curl -s -w ' %{http_code}\n' localhost:8011/readyz
{"status":"unavailable","db":"ConnectionRefusedError"} 503
$ curl -s -w ' %{http_code}\n' localhost:8011/healthz
{"status":"ok"} 200

Readiness follows the database, and liveness doesn’t, which is the point. In deployment.yaml, point readinessProbe at /readyz and leave livenessProbe on /healthz. While the database is down, every pod drops out of the Service’s endpoints, as 7pb42 did in the blocked-loop failure, and rejoins when the database is back. Nothing is restarted. If liveness checked the database instead, one database outage would make the kubelet restart every pod at once. The restarts can’t fix the database, and the pods all reconnect together when it returns. Keep the check cheap and bounded. It runs every periodSeconds on every pod, and if it takes longer than the probe’s timeoutSeconds (1 second by default), the kubelet counts it as a failure anyway. Clean up with docker rm -f fapi-api fapi-db. Probes only answer up or down. For request rates and latency, expose a /metrics endpoint that Prometheus scrapes, covered in the monitoring track’s first lesson.

Practice

  1. Copy Dockerfile.naive to Dockerfile.exec and change only the CMD line to exec form, keeping the full python:3.12 base. Build it as fastapi-demo:exec, run it as demo, and check process 1 and docker stop -t 10 again. Which of the naive image’s problems does this fix, and which remain?
Solution 
FROM python:3.12
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
$ docker build -q -f Dockerfile.exec -t fastapi-demo:exec .
sha256:78ee73b0a571def912d66e1c6a56ce31f6ad624f2cb9104582f947e04c966c98
$ docker run -d --name demo -p 8080:8000 fastapi-demo:exec
65bb0e1c8283730dfa984269f66d38ff604ccfca6e181e8f8dc810dec6e1f851
$ docker exec demo sh -c 'id -un; tr "\0" " " < /proc/1/cmdline; echo'
root
/usr/local/bin/python3.12 /usr/local/bin/uvicorn main:app --host 0.0.0.0 --port 8000 
$ time docker stop -t 10 demo
demo

real	0m0.995s
user	0m0.046s
sys	0m0.060s
$ docker inspect demo --format "{{.State.ExitCode}}"
0
$ docker rm demo
demo

The shutdown problem is fixed: uvicorn is now process 1, receives SIGTERM, and exits 0 in under a second. The other problems remain: id -un still prints root, and the image is still based on the 1.17 GB python:3.12. Signal handling, size and user are three separate fixes. -q makes docker build print only the image ID; .dockerignore keeps the Dockerfiles themselves out of the image.

  1. With version 1.1 running, scale to three replicas and time the rollout. Predict how long it takes, then confirm that all three endpoints are ready.
Solution 
$ kubectl scale deployment/fastapi-demo --replicas=3
deployment.apps/fastapi-demo scaled
$ time kubectl rollout status deployment/fastapi-demo
Waiting for deployment "fastapi-demo" rollout to finish: 2 of 3 updated replicas are available...
deployment "fastapi-demo" successfully rolled out

real	0m22.384s
user	0m0.000s
sys	0m0.046s
$ kubectl get endpointslices -l kubernetes.io/service-name=fastapi-demo -o jsonpath='{range .items[*].endpoints[*]}{.addresses[0]}  ready={.conditions.ready}  {.targetRef.name}{"\n"}{end}'
10.244.0.18  ready=true  fastapi-demo-7fc94b787b-np59r
10.244.0.19  ready=true  fastapi-demo-7fc94b787b-85mbc
10.244.0.21  ready=true  fastapi-demo-7fc94b787b-j8tgh

About 22 seconds: one new pod, one 20-second warmup, plus a probe interval or two. Scaling out is only as fast as your slowest startup, so warmup time is a direct cost when an autoscaler needs capacity quickly.

  1. Roll back to the original 1.0 release. Run kubectl rollout history first and decide which revision to target. Why is a plain kubectl rollout undo the wrong choice here?
Solution 
$ kubectl rollout history deployment/fastapi-demo
deployment.apps/fastapi-demo 
REVISION  CHANGE-CAUSE
1         <none>
2         <none>
3         <none>

$ kubectl rollout undo deployment/fastapi-demo --to-revision=1
deployment.apps/fastapi-demo rolled back
$ time kubectl rollout status deployment/fastapi-demo
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "fastapi-demo" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "fastapi-demo" rollout to finish: 1 old replicas are pending termination...
deployment "fastapi-demo" successfully rolled out

real	0m6.447s
user	0m0.000s
sys	0m0.031s
$ kubectl run client -q --rm -i --restart=Never --image=curlimages/curl -- sh -c 'for i in 1 2 3; do curl -s fastapi-demo/; echo; done'
{"pod":"fastapi-demo-6d4cf99857-bnq4q","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-q8frp","version":"1.0"}
{"pod":"fastapi-demo-6d4cf99857-q8frp","version":"1.0"}
$ kubectl rollout history deployment/fastapi-demo
deployment.apps/fastapi-demo 
REVISION  CHANGE-CAUSE
2         <none>
3         <none>
4         <none>

Revision 2 is the broken kubectl set env release with a 20-second warmup and no startup probe. A plain undo goes back exactly one revision, which would have been revision 2. Revision 1 has no warmup, so the rollback finished in 6.4 seconds and all responses say 1.0. Afterwards revision 1 is gone and revision 4 has its contents: a rollback is recorded as a new revision. CHANGE-CAUSE is empty because nothing set the kubernetes.io/change-cause annotation, which is what makes this history readable in practice.

  1. On the rolled-back 1.0 Deployment, set terminationGracePeriodSeconds to 5. Wait for that rollout, pick one new pod and its IP from kubectl get pods -o wide, and block it for 60 seconds again. Then wait about 20 seconds for the liveness failures and the restart, and compare the gap between Killing and Started with the 30 seconds you saw before.
Solution 
$ kubectl patch deployment fastapi-demo -p '{"spec":{"template":{"spec":{"terminationGracePeriodSeconds":5}}}}'
deployment.apps/fastapi-demo patched
$ kubectl rollout status deployment/fastapi-demo
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 2 out of 3 new replicas have been updated...
Waiting for deployment "fastapi-demo" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "fastapi-demo" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "fastapi-demo" rollout to finish: 1 old replicas are pending termination...
deployment "fastapi-demo" successfully rolled out
$ kubectl get pods -o wide
NAME                            READY   STATUS    RESTARTS   AGE   IP           NODE   NOMINATED NODE   READINESS GATES
fastapi-demo-6f6f786847-fw4vj   1/1     Running   0          7s    10.244.0.6   fapi   <none>           <none>
fastapi-demo-6f6f786847-gsttg   1/1     Running   0          5s    10.244.0.7   fapi   <none>           <none>
fastapi-demo-6f6f786847-rp8qc   1/1     Running   0          3s    10.244.0.8   fapi   <none>           <none>
$ kubectl run blocker -q --rm -i --restart=Never --image=curlimages/curl -- curl -s -m 3 -X POST '10.244.0.6:8000/debug/block?seconds=60'
pod default/blocker terminated (Error)
$ kubectl describe pod fastapi-demo-6f6f786847-fw4vj | sed -n '/^Events:/,$p' | grep -E 'Killing|Started|Liveness'
  Warning  Unhealthy  8s (x3 over 18s)   kubelet            spec.containers{api}: Liveness probe failed: Get "http://10.244.0.6:8000/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  Normal   Killing    8s                 kubelet            spec.containers{api}: Container api failed liveness probe, will be restarted
  Normal   Started    3s (x2 over 39s)   kubelet            spec.containers{api}: Container started

The gap fell from 30 seconds to 5, exactly the new grace period. The patch itself started a new rollout, which is why every pod has a new name (6f6f786847), and fw4vj at 10.244.0.6 is the one picked for blocking. A shorter grace period gets a hung pod back sooner, but it also cuts off healthy pods that are still finishing requests during a normal rollout. Set it a little above your slowest real request, not as low as possible. Uvicorn’s own --timeout-graceful-shutdown can bound shutdown from inside the process.

Frequently Asked Questions

How do I make a small Docker image for a FastAPI app?
Use a multi-stage build: install the requirements into a virtual environment in a python:3.12-slim build stage, then copy only that venv and your code into a fresh python:3.12-slim runtime stage. In a measured build, a FastAPI and uvicorn app came to 171 MB this way against 1.17 GB on the full python:3.12 image with a plain pip install. Add a non-root USER in the runtime stage as well.
Why does docker stop take 10 seconds for my FastAPI container?
Usually because the Dockerfile uses shell-form CMD, such as CMD uvicorn main:app, which makes /bin/sh process 1 in the container. The shell doesn't pass SIGTERM on to uvicorn, so Docker waits out the stop timeout and sends SIGKILL, and the container exits with code 137. Exec-form CMD ["uvicorn", "main:app", ...] makes uvicorn process 1, so it receives SIGTERM, shuts down gracefully and exits 0 in about a second.
What is the difference between liveness and readiness probes for FastAPI on Kubernetes?
A failing readiness probe removes the pod from the Service's endpoints so it stops receiving traffic, but leaves the container running. A failing liveness probe makes the kubelet restart the container. Point both at a cheap route such as /healthz. A FastAPI event loop blocked by a synchronous call fails both, because it can't answer any request until the call returns.
Why does Kubernetes keep restarting my FastAPI pod during startup?
If startup work in the lifespan hook, such as loading a model, takes longer than the liveness probe's failure window, the kubelet kills the container before it ever starts listening, and the cycle repeats. In a measured run, a 20-second warmup against a liveness window of three 5-second checks restarted the pod 4 times in 90 seconds. Add a startupProbe with a larger failure budget; liveness and readiness checks don't start until it passes.
Should I use uvicorn --workers inside a Kubernetes pod?
Usually not. Run one uvicorn process per container and scale with replicas, so Kubernetes can restart, schedule and load-balance each process on its own and the pod's CPU request matches one process. Multiple workers in one container make a single liveness probe answer for several processes and hide which one is unhealthy. Use --workers when there's no orchestrator, such as a single VM.