MCP Observability with OpenTelemetry Traces

What You’ll Trace

When a host calls your server and something is slow or wrong, logs tell you what one process saw. A trace tells you what happened to one request end to end. This lesson has two parts:

• Required MCP concepts: the spans the SDK emits on its own, what a failing tool looks like, how one trace crosses from client to server, and adding your own spans. All of this runs in one Python process with no extra services.
• Optional production infrastructure: exporting the same spans to a real backend, Jaeger, and reading them back. You need this once more than one person looks at the traces.

Everything here ran in Docker python:3.12-slim with pip install mcp==2.3.0 opentelemetry-sdk==1.45.1 opentelemetry-exporter-otlp-proto-http==1.45.1. No API key and no model are involved.

Part 1: Required MCP Concepts

Collect spans in memory

Each example imports this helper, which installs a TracerProvider that keeps finished spans in a list.

# otel_setup.py - send every span to a list we can print
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter

spans = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(spans))
trace.set_tracer_provider(provider)

There is nothing MCP-specific in it: it is the standard OpenTelemetry SDK. The SDK’s spans stay silent until a provider like this is installed.

The spans the SDK gives you

# spans.py - the spans an MCPServer emits on its own
import asyncio
from otel_setup import spans
from mcp import Client
from mcp.server import MCPServer

mcp = MCPServer("calc")


@mcp.tool()
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b


async def main():
    async with Client(mcp) as client:
        await client.call_tool("add", {"a": 2, "b": 3})
    for s in spans.get_finished_spans():
        print(s.kind.name, "|", s.name, "|", s.attributes.get("gen_ai.tool.name", "-"))


asyncio.run(main())

$ python spans.py
SERVER | server/discover | -
SERVER | tools/call add | add
SERVER | tools/list | -

One tool call produced three SERVER spans, and you wrote no tracing code at all. server/discover is the client’s first request. tools/call add carries the attribute gen_ai.tool.name, so a backend can group calls by tool. tools/list finished last because the client fetched the tool list after the call, to check the result against the tool’s output schema. Every span is named after the JSON-RPC method, plus the tool name for tools/call.

The in-memory Client(mcp) used here creates no CLIENT spans, only the server’s. A real transport does, as the stdio example below shows.

What a failing tool looks like

# failing.py - what a crashing tool and bad arguments look like as spans
import asyncio
from otel_setup import spans
from mcp import Client
from mcp.server import MCPServer

mcp = MCPServer("calc")


@mcp.tool()
def divide(a: int, b: int) -> float:
    """Divide a by b."""
    return a / b


async def main():
    async with Client(mcp) as client:
        for args in ({"a": 6, "b": 3}, {"a": 1, "b": 0}, {"a": "one", "b": 2}):
            result = await client.call_tool("divide", args)
            print(args, "isError:", result.is_error)
    for s in spans.get_finished_spans():
        if s.name.startswith("tools/call"):
            print(s.name, "|", s.status.status_code.name, "|", s.attributes.get("error.type"))


asyncio.run(main())

$ python failing.py 2>/dev/null
{'a': 6, 'b': 3} isError: False
{'a': 1, 'b': 0} isError: True
{'a': 'one', 'b': 2} isError: True
tools/call divide | UNSET | None
tools/call divide | ERROR | tool_error
tools/call divide | ERROR | tool_error

The server logs a traceback to stderr for the division by zero, hidden here with 2>/dev/null. The spans tell the same story: the good call has status UNSET, which OpenTelemetry uses for “nothing went wrong”, and both failures are ERROR with error.type set to tool_error. A crash inside the tool and a string where an integer belongs both come back to the client as isError: True, so the span can’t tell them apart. The traceback in the log can. Alert on error.type, then read the log for the cause.

One trace across client and server

This stdio server writes its own spans to a file, so the client can compare them with its spans after the call.

# traced_server.py - a stdio server that writes its spans to a file
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
from mcp.server import MCPServer

log = open("server_spans.txt", "w")
fmt = lambda s: f"{s.name},{s.context.trace_id:032x},{s.parent.span_id if s.parent else 0:016x}\n"
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter(out=log, formatter=fmt)))
trace.set_tracer_provider(provider)
mcp = MCPServer("calc")


@mcp.tool()
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b


if __name__ == "__main__":
    mcp.run()

# traced_client.py - does the server's span join the client's trace?
import asyncio
from otel_setup import spans
from mcp import Client, StdioServerParameters

server = StdioServerParameters(command="python", args=["traced_server.py"])


async def main():
    async with Client(server) as client:
        await client.call_tool("add", {"a": 2, "b": 3})
    sent = {s.name: s for s in spans.get_finished_spans()}["MCP send tools/call add"]
    print("client:", sent.kind.name, "|", sent.name)
    for line in open("server_spans.txt"):
        name, trace_id, parent = line.strip().split(",")
        if name == "tools/call add":
            print("server: SERVER |", name)
            print("same trace:", trace_id == f"{sent.context.trace_id:032x}")
            print("parent is the client span:", parent == f"{sent.context.span_id:016x}")


asyncio.run(main())

$ python traced_client.py
client: CLIENT | MCP send tools/call add
server: SERVER | tools/call add
same trace: True
parent is the client span: True

The client and the server are two processes joined by a pipe, yet the server’s span is a child of the client’s. Before sending each request, the client opens a CLIENT span named MCP send tools/call add and writes its W3C traceparent into the request’s _meta. The server reads _meta and starts its SERVER span under that parent. In a backend that shows up as one trace, so you can see how much of a slow call was spent on the wire and how much inside the tool.

Your own spans inside a tool

# custom_span.py - your own span inside a tool nests under the SDK's span
import asyncio
from opentelemetry import trace
from otel_setup import spans
from mcp import Client
from mcp.server import MCPServer

tracer = trace.get_tracer("orders-app")
mcp = MCPServer("orders")


@mcp.tool()
def order_total(order_id: int) -> float:
    """Total price of an order."""
    with tracer.start_as_current_span("db.query orders") as span:
        span.set_attribute("db.rows", 3)
        return 19.5 + 5.0 + 0.5


async def main():
    async with Client(mcp) as client:
        await client.call_tool("order_total", {"order_id": 7})
    done = spans.get_finished_spans()
    names = {s.context.span_id: s.name for s in done}
    for s in done:
        print(s.name, "<- parent:", names.get(s.parent.span_id) if s.parent else None)


asyncio.run(main())

$ python custom_span.py
server/discover <- parent: None
db.query orders <- parent: tools/call order_total
tools/call order_total <- parent: None
tools/list <- parent: None

db.query orders lands under tools/call order_total without any wiring. The SDK’s span is the current span while the tool runs, and OpenTelemetry carries it into the worker thread that runs a sync tool. Wrap the slow parts of a tool (a database query, an HTTP call) in your own spans and they show up exactly where they happened.

Traces only, no metrics

# no_metrics.py - which OpenTelemetry signals does the SDK itself use?
import pathlib
import mcp

files = list(pathlib.Path(mcp.__file__).parent.rglob("*.py"))
for signal in ("opentelemetry.trace", "opentelemetry.metrics", "opentelemetry._logs"):
    users = [f.name for f in files if signal in f.read_text()]
    print(f"{signal}: {len(users)} files {sorted(users)}")

$ python no_metrics.py
opentelemetry.trace: 3 files ['_otel.py', '_otel.py', 'jsonrpc_dispatcher.py']
opentelemetry.metrics: 0 files []
opentelemetry._logs: 0 files []

mcp 2.3.0 uses only the tracing API (the two _otel.py files are the server and shared helpers). There are no MCP metrics to turn on. You can derive request counts, error rates and latency from the spans in your backend, or record your own metrics in application code.

Part 2: Optional Production Infrastructure

Export to Jaeger

Spans in memory vanish when the process exits. To keep them, send them over OTLP to a backend. Jaeger is a small open-source one that runs in a single container. Start it on a Docker network, and run the next two scripts in a container on the same network:

docker network create mcp-lab
docker run -d --name jaeger --network mcp-lab jaegertracing/jaeger:2.22.0

# export_jaeger.py - ship the same spans to Jaeger over OTLP/HTTP
import asyncio
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from mcp import Client
from mcp.server import MCPServer

provider = TracerProvider(resource=Resource.create({"service.name": "calc-mcp"}))
exporter = OTLPSpanExporter(endpoint="http://jaeger:4318/v1/traces")
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)
mcp = MCPServer("calc")


@mcp.tool()
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b


async def main():
    async with Client(mcp) as client:
        await client.call_tool("add", {"a": 2, "b": 3})
    provider.shutdown()  # flush the batch before the process exits
    print("exported")


asyncio.run(main())

$ python export_jaeger.py
exported

Two lines differ from the in-memory setup. Resource.create({"service.name": "calc-mcp"}) names the service, which is how you find it in the backend. BatchSpanProcessor queues spans and sends them in the background, the right choice for a server, and provider.shutdown() flushes the queue before the process exits. Leave that out of a short script and the last spans are lost.

Read the spans back

Jaeger 2.x serves its query API under /api/v3. The older /api/traces path returns 404.

# query_jaeger.py - read the spans back from Jaeger's HTTP API (v3)
from datetime import datetime, timedelta, timezone
import httpx

now = datetime.now(timezone.utc)
params = {"query.service_name": "calc-mcp",
          "query.start_time_min": (now - timedelta(hours=1)).isoformat(),
          "query.start_time_max": now.isoformat()}
r = httpx.get("http://jaeger:16686/api/v3/traces", params=params)
resource_spans = r.json()["result"]["resourceSpans"]
spans = [s for rs in resource_spans for ss in rs["scopeSpans"] for s in ss["spans"]]
KINDS = {1: "INTERNAL", 2: "SERVER", 3: "CLIENT"}
print("spans stored:", len(spans))
for s in sorted(spans, key=lambda s: s["name"]):
    attrs = {a["key"]: a["value"].get("stringValue") for a in s.get("attributes", [])}
    print(s["name"], "|", KINDS[s["kind"]], "|", attrs.get("gen_ai.tool.name", "-"))

$ python query_jaeger.py
spans stored: 3
server/discover | SERVER | -
tools/call add | SERVER | add
tools/list | SERVER | -

The same three SERVER spans, with the same tool attribute, are now stored outside the process. The Jaeger UI at port 16686 shows them as a timeline. In production you’d usually put an OpenTelemetry Collector between your servers and the backend, so you can change backends, sample or redact without redeploying the server. The exporter code stays the same, only the endpoint changes.

Practice

1. Tag your span. Change custom_span.py so the db.query orders span carries an order.id attribute, call the tool for orders 7 and 9, and print the attribute from each span.

# p12_1.py - tag the custom span with the order it served
import asyncio
from opentelemetry import trace
from otel_setup import spans
from mcp import Client
from mcp.server import MCPServer

tracer = trace.get_tracer("orders-app")
mcp = MCPServer("orders")


@mcp.tool()
def order_total(order_id: int) -> float:
    """Total price of an order."""
    with tracer.start_as_current_span("db.query orders") as span:
        span.set_attribute("order.id", order_id)
        return 25.0


async def main():
    async with Client(mcp) as client:
        await client.call_tool("order_total", {"order_id": 7})
        await client.call_tool("order_total", {"order_id": 9})
    for s in spans.get_finished_spans():
        if s.name == "db.query orders":
            print(s.name, "order.id =", s.attributes["order.id"])


asyncio.run(main())

$ python p12_1.py
db.query orders order.id = 7
db.query orders order.id = 9

2. Opt out. The SDK’s tracing comes from a middleware on the server. Remove it and count the spans a tool call produces.

# p12_2.py - opt out of the SDK's spans by dropping its middleware
import asyncio
from otel_setup import spans
from mcp import Client
from mcp.server import MCPServer

mcp = MCPServer("calc")
mcp._lowlevel_server.middleware = []  # provisional API in mcp 2.x


@mcp.tool()
def add(a: int, b: int) -> int:
    """Add two integers."""
    return a + b


async def main():
    async with Client(mcp) as client:
        print((await client.call_tool("add", {"a": 2, "b": 3})).structured_content)
    print("spans:", len(spans.get_finished_spans()))


asyncio.run(main())

$ python p12_2.py
{'result': 5}
spans: 0

3. An error rate from spans. Call divide with b set to 1, 0, 2, 0 and 5, then compute how many tools/call spans failed and why, using only the spans.

# p12_3.py - an error rate per tool, computed from the spans
import asyncio
from collections import Counter
from otel_setup import spans
from mcp import Client
from mcp.server import MCPServer

mcp = MCPServer("calc")


@mcp.tool()
def divide(a: int, b: int) -> float:
    """Divide a by b."""
    return a / b


async def main():
    async with Client(mcp) as client:
        for b in (1, 0, 2, 0, 5):
            await client.call_tool("divide", {"a": 10, "b": b})
    calls = [s for s in spans.get_finished_spans() if s.name.startswith("tools/call")]
    errors = Counter(s.attributes.get("error.type") for s in calls if s.status.status_code.name == "ERROR")
    print(f"{len(calls)} calls, {sum(errors.values())} errors:", dict(errors))


asyncio.run(main())

$ python p12_3.py 2>/dev/null
5 calls, 2 errors: {'tool_error': 2}