Skip to main content
Event-Driven Architecture · lesson 4 of 10
Event-Driven Architecture intermediate Lesson 4 of 10

Event Sourcing: Snapshots So Replay Doesn't Get Slower Forever

Replaying 200,000 events gets measurably slower as a log grows; a snapshot resumes from a known balance and replays only what changed since.

By SkillByExample · Published

event-driven event-sourcing performance python

Full replay of an event log is always correct, but a log with 200,000 events takes measurably longer to replay than one with a handful — a snapshot is a cached answer you’re always allowed to throw away.

Measuring the actual speedup, not assuming one

import time

def apply(events):
    balance = 0
    for event in events:
        if event["type"] == "Deposited":
            balance += event["amount"]
        else:
            balance -= event["amount"]
    return balance

events = []
for i in range(200_000):
    if i % 3 == 0:
        events.append({"type": "Withdrawn", "amount": 1})
    else:
        events.append({"type": "Deposited", "amount": 1})

start = time.perf_counter()
full_replay_balance = apply(events)
full_replay_time = time.perf_counter() - start
print(f"Full replay ({len(events)} events): balance={full_replay_balance}, took {full_replay_time * 1000:.1f}ms")

snapshot_at = 150_000
snapshot_balance = apply(events[:snapshot_at])
snapshot = {"balance": snapshot_balance, "as_of_event": snapshot_at}
print(f"Snapshot taken at event {snapshot['as_of_event']}: balance={snapshot['balance']}")

start = time.perf_counter()
remaining = events[snapshot["as_of_event"]:]
resumed_balance = snapshot["balance"] + apply(remaining)
snapshot_replay_time = time.perf_counter() - start
print(f"Snapshot + replay ({len(remaining)} events): balance={resumed_balance}, took {snapshot_replay_time * 1000:.1f}ms")

print(f"Speedup: {full_replay_time / snapshot_replay_time:.1f}x")
$ python event_sourcing_snapshots.py
Full replay (200000 events): balance=66666, took 9.6ms
Snapshot taken at event 150000: balance=50000
Snapshot + replay (50000 events): balance=66666, took 2.3ms
Speedup: 4.2x

Both paths agree on the final balance — 66666 either way — which is the actual point of a snapshot: it changes how much work replay does, never what answer it produces. The exact speedup will vary run to run (measured here, not assumed), but it tracks the ratio of events skipped to events replayed, which is 150,000 : 50,000 in this run.

Practice

1. Take a snapshot at several different points and confirm each one matches a direct balance-history calculation.
def apply(events):
    balance = 0
    for event in events:
        if event["type"] == "Deposited":
            balance += event["amount"]
        else:
            balance -= event["amount"]
    return balance

events = [
    {"type": "Deposited", "amount": 100},
    {"type": "Withdrawn", "amount": 30},
    {"type": "Deposited", "amount": 50},
    {"type": "Withdrawn", "amount": 20},
]

snapshot_points = [1, 2, 3, 4]
for point in snapshot_points:
    snapshot_balance = apply(events[:point])
    print(f"Snapshot after {point} event(s): balance={snapshot_balance}")

full_history = [apply(events[:i + 1]) for i in range(len(events))]
print(f"Direct balance_history: {full_history}")
$ python multiple_snapshots.py
Snapshot after 1 event(s): balance=100
Snapshot after 2 event(s): balance=70
Snapshot after 3 event(s): balance=120
Snapshot after 4 event(s): balance=100
Direct balance_history: [100, 70, 120, 100]

A “snapshot” is nothing more than apply called on a prefix of the log — there’s no special snapshot machinery here, just a convention of remembering which prefix you already computed.

2. Take a snapshot, then let new events arrive, and confirm resuming from the (now not-newest) snapshot still matches a full replay.
def apply(events):
    balance = 0
    for event in events:
        if event["type"] == "Deposited":
            balance += event["amount"]
        else:
            balance -= event["amount"]
    return balance

events = [
    {"type": "Deposited", "amount": 100},
    {"type": "Withdrawn", "amount": 30},
]

snapshot = {"balance": apply(events), "as_of_event": len(events)}
print(f"Snapshot: {snapshot}")

events.append({"type": "Deposited", "amount": 40})
events.append({"type": "Withdrawn", "amount": 10})
print(f"New events arrived, log now has {len(events)} total events")

new_events = events[snapshot["as_of_event"]:]
resumed_balance = snapshot["balance"] + apply(new_events)
print(f"Replaying only the {len(new_events)} new event(s): balance={resumed_balance}")
print(f"Full replay for comparison: {apply(events)}")
$ python resume_after_new_events.py
Snapshot: {'balance': 70, 'as_of_event': 2}
New events arrived, log now has 4 total events
Replaying only the 2 new event(s): balance=100
Full replay for comparison: 100

The snapshot doesn’t need to be the newest thing available to be useful — as_of_event says precisely where to pick replay back up, so any older snapshot plus everything after it is just as correct as a full replay, only cheaper.

3. Persist both the events and the snapshot itself in SQLite, so the snapshot survives a restart.
import sqlite3

conn = sqlite3.connect(":memory:")
conn.execute("CREATE TABLE events (type TEXT, amount INTEGER)")
conn.execute("CREATE TABLE snapshots (balance INTEGER, as_of_event INTEGER)")

for amount, kind in [(100, "Deposited"), (30, "Withdrawn"), (50, "Deposited")]:
    conn.execute("INSERT INTO events VALUES (?, ?)", (kind, amount))
conn.commit()

def apply_range(conn, start_rowid=0):
    balance = 0
    for event_type, amount in conn.execute(
        "SELECT type, amount FROM events WHERE rowid > ? ORDER BY rowid", (start_rowid,)
    ):
        balance += amount if event_type == "Deposited" else -amount
    return balance

current_count = conn.execute("SELECT COUNT(*) FROM events").fetchone()[0]
snapshot_balance = apply_range(conn, 0)
conn.execute("INSERT INTO snapshots VALUES (?, ?)", (snapshot_balance, current_count))
conn.commit()
print(f"Snapshot saved: balance={snapshot_balance}, as_of_event={current_count}")

conn.execute("INSERT INTO events VALUES ('Withdrawn', 20)")
conn.commit()

saved_balance, saved_as_of = conn.execute("SELECT balance, as_of_event FROM snapshots").fetchone()
resumed_balance = saved_balance + apply_range(conn, saved_as_of)
print(f"Resumed from saved snapshot: balance={resumed_balance}")
$ python persisted_snapshot.py
Snapshot saved: balance=120, as_of_event=3
Resumed from saved snapshot: balance=100

as_of_event here is a row count, which only lines up with SQLite’s rowid because rows are never deleted and rowid is assigned sequentially starting at 1 — a real system storing this long-term would use an explicit sequence column rather than lean on that assumption.

4. Deliberately introduce an off-by-one in as_of_event and see how it silently corrupts the resumed balance.
def apply(events):
    balance = 0
    for event in events:
        if event["type"] == "Deposited":
            balance += event["amount"]
        else:
            balance -= event["amount"]
    return balance

events = [
    {"type": "Deposited", "amount": 100},
    {"type": "Withdrawn", "amount": 30},
    {"type": "Deposited", "amount": 50},
]

correct_snapshot = {"balance": apply(events), "as_of_event": len(events)}
print(f"Correct snapshot: {correct_snapshot}")

off_by_one_snapshot = {"balance": apply(events), "as_of_event": len(events) - 1}
print(f"Off-by-one snapshot (as_of_event wrong by 1): {off_by_one_snapshot}")

events.append({"type": "Withdrawn", "amount": 10})

correct_result = correct_snapshot["balance"] + apply(events[correct_snapshot["as_of_event"]:])
buggy_result = off_by_one_snapshot["balance"] + apply(events[off_by_one_snapshot["as_of_event"]:])

print(f"Resumed from correct snapshot:   {correct_result}")
print(f"Resumed from off-by-one snapshot: {buggy_result}")
print(f"Full replay (ground truth):      {apply(events)}")
$ python off_by_one_snapshot.py
Correct snapshot: {'balance': 120, 'as_of_event': 3}
Off-by-one snapshot (as_of_event wrong by 1): {'balance': 120, 'as_of_event': 2}
Resumed from correct snapshot:   110
Resumed from off-by-one snapshot: 160
Full replay (ground truth):      110

The off-by-one snapshot replays the third event (Deposited 50) a second time on top of a balance that already includes it — no exception is raised anywhere, the result is just wrong. This is exactly why snapshot logic needs tests that compare against a full replay on the same data, not tests that only check the snapshot path in isolation.

Frequently Asked Questions

Doesn't a snapshot bring back the 'can I trust this stored number' problem event sourcing was supposed to avoid?
Not quite — the snapshot is a disposable cache of a value the event log can always recompute independently, not the only copy of the truth. If a snapshot is lost or found to be wrong, the fix is to delete it and replay from event zero; a corrupted mutable field has no such fallback.
How often should a real system take a snapshot?
There's no universal number — it depends on how many events accumulate per aggregate and how expensive replay actually is for your event shapes. A common approach is 'every N events' or 'every N minutes for hot aggregates,' tuned against measured replay time, not a fixed rule copied from another system.
What if the snapshot's as_of_event bookkeeping is wrong?
An off-by-one as_of_event silently reprocesses or skips an event, producing a balance that's wrong but not obviously so — no exception, no crash, just a number that doesn't match a full replay. That silent-wrongness is why snapshot logic deserves its own tests that compare against full replay, not just spot-checking a few values.
Can a snapshot go stale?
Only in the sense that it stops being the *latest* snapshot once new events arrive after it — it never becomes incorrect for the point in time it was taken at. Resuming from an older, no-longer-newest snapshot and replaying everything since still produces the same answer as a full replay, because its recorded position says exactly which events still need replaying.