Why not just migrate all the old events in the log to the new shape once, instead of upcasting on every replay?

An append-only event log is usually treated as immutable — rewriting historical events undermines the audit trail and durability guarantees event sourcing exists for in the first place. Upcasting on read keeps old events exactly as they were originally recorded while still letting new code understand them.

Doesn't upcasting forever mean old code paths accumulate indefinitely?

Yes, and that's a real, acknowledged cost of this approach — a long-lived event-sourced system does carry its schema history along with it. Some teams cap this by periodically taking a snapshot (see the earlier snapshots lesson) and treating events older than the oldest surviving snapshot as no longer needing direct replay, though the raw events are typically still archived.

What's the actual bug when an upcast function mutates its input instead of returning a new dict?

Mutating the dict in place changes the very object stored in the event log, so a second call sees the new field already present and treats the event as if it always had the new shape. It happens to look harmless in a single run, but any other code still holding a reference to that original event object now sees a value it never actually recorded.

Is a schema_version field always necessary?

Not always — inferring version from which keys are present works fine for a small number of changes, which is what this lesson's earlier upcast functions do. That inference stops being reliable once schema changes accumulate or a key's mere presence no longer implies a single specific version; an explicit schema_version field scales past that in a way guessing from field presence doesn't.

Event Schema Evolution: Replaying Events That Predate a Field

An event log that’s been running for a while accumulates events written under more than one schema — replay has to understand all of them, not just the newest one.

A field that didn’t exist yet

old_events = [
    {"type": "Deposited", "amount": 100},
    {"type": "Withdrawn", "amount": 30},
]

new_events = [
    {"type": "Deposited", "amount_cents": 5000},
]

full_log = old_events + new_events

def apply_naive(events):
    balance = 0
    for event in events:
        if event["type"] == "Deposited":
            balance += event["amount_cents"]
        elif event["type"] == "Withdrawn":
            balance -= event["amount_cents"]
    return balance

try:
    apply_naive(full_log)
except KeyError as e:
    print(f"Replay crashed on an old event: missing key {e}")

def upcast(event):
    if "amount" in event and "amount_cents" not in event:
        event = {**event, "amount_cents": event["amount"] * 100}
    return event

def apply_with_upcasting(events):
    balance = 0
    for raw_event in events:
        event = upcast(raw_event)
        if event["type"] == "Deposited":
            balance += event["amount_cents"]
        elif event["type"] == "Withdrawn":
            balance -= event["amount_cents"]
    return balance

print(f"Balance replaying old and new events together: {apply_with_upcasting(full_log)}")

$ python event_schema_evolution.py
Replay crashed on an old event: missing key 'amount_cents'
Balance replaying old and new events together: 12000

amount became amount_cents (dollars to integer cents, a real reason to change a stored event’s shape) at some point after the first two events were already written. apply_naive only knows the new field, so it crashes the instant it meets an event from before the change. upcast fills in the missing field on the way in, computed from the old one, so both event generations produce a correct combined balance — 100*100 - 30*100 + 5000 = 12000.

Practice

1. Add a second schema change — renaming "Deposited" to "MoneyDeposited" — and upcast events from all three generations.

v1_events = [{"type": "Deposited", "amount": 100}]
v2_events = [{"type": "Deposited", "amount_cents": 5000}]
v3_events = [{"type": "MoneyDeposited", "amount_cents": 2000}]

def upcast(event):
    if event["type"] == "Deposited" and "amount_cents" not in event:
        event = {**event, "amount_cents": event["amount"] * 100}
    if event["type"] == "Deposited":
        event = {**event, "type": "MoneyDeposited"}
    return event

def apply_all(events):
    balance = 0
    for raw_event in events:
        event = upcast(raw_event)
        if event["type"] == "MoneyDeposited":
            balance += event["amount_cents"]
    return balance

full_log = v1_events + v2_events + v3_events
print(f"Balance across three event shapes: {apply_all(full_log)}")

$ python two_schema_changes.py
Balance across three event shapes: 17000

Both changes live in the same upcast function, applied in sequence — the field rename runs first (conditioned on still being "Deposited"), then the type rename runs unconditionally for anything that started as "Deposited", whichever generation it came from.

2. Split the two changes into separate upcast functions, chained together.

def upcast_v1_to_v2(event):
    if "amount" in event and "amount_cents" not in event:
        return {**event, "amount_cents": event["amount"] * 100}
    return event

def upcast_v2_to_v3(event):
    if event.get("type") == "Deposited":
        return {**event, "type": "MoneyDeposited"}
    return event

def upcast_chain(event):
    event = upcast_v1_to_v2(event)
    event = upcast_v2_to_v3(event)
    return event

v1_event = {"type": "Deposited", "amount": 100}
print(f"v1 event: {v1_event}")
print(f"after chain: {upcast_chain(v1_event)}")

$ python chained_upcasts.py
v1 event: {'type': 'Deposited', 'amount': 100}
after chain: {'type': 'MoneyDeposited', 'amount': 100, 'amount_cents': 10000}

Each function only knows about one schema transition, and upcast_chain runs them in the order the schema actually changed — a new transition means adding one function and one line to the chain, not editing the logic for every version at once.

3. Compare an upcast that mutates its input against one that returns a new dict, and see the mutation corrupt the stored event.

events = [{"type": "Deposited", "amount": 100}]

def upcast_mutates(event):
    if "amount_cents" not in event:
        event["amount_cents"] = event["amount"] * 100
    return event

first_pass = [upcast_mutates(e) for e in events]
print(f"Stored log after first replay: {events}")

def upcast_safe(event):
    if "amount_cents" not in event:
        return {**event, "amount_cents": event["amount"] * 100}
    return event

events2 = [{"type": "Deposited", "amount": 100}]
second_pass = [upcast_safe(e) for e in events2]
print(f"Stored log after safe replay:  {events2}")

$ python mutation_bug.py
Stored log after first replay: [{'type': 'Deposited', 'amount': 100, 'amount_cents': 10000}]
Stored log after safe replay:  [{'type': 'Deposited', 'amount': 100}]

upcast_mutates writes amount_cents directly onto the dict that’s still sitting in events — the “original” log is no longer what was actually recorded, just from calling a function that’s supposed to be read-only. upcast_safe’s {**event, ...} builds a new dict every time, so the source list is untouched no matter how many times replay runs.

4. Add an explicit schema_version field and dispatch on it instead of inferring version from which keys exist.

unversioned_event = {"type": "Deposited", "amount": 100}
versioned_event = {"type": "Deposited", "schema_version": 2, "amount_cents": 5000}

def upcast(event):
    version = event.get("schema_version", 1)
    if version == 1:
        return {**event, "schema_version": 2, "amount_cents": event["amount"] * 100}
    return event

print(f"Unversioned (implicit v1): {upcast(unversioned_event)}")
print(f"Explicitly versioned v2:   {upcast(versioned_event)}")

$ python explicit_versioning.py
Unversioned (implicit v1): {'type': 'Deposited', 'amount': 100, 'schema_version': 2, 'amount_cents': 10000}
Explicitly versioned v2:   {'type': 'Deposited', 'schema_version': 2, 'amount_cents': 5000}

event.get("schema_version", 1) treats any event with no version field as version 1, so existing events never need to be touched to adopt this — only new events start carrying the field, and old ones are still correctly recognized by its absence.

A field that didn’t exist yet

Practice

Frequently Asked Questions