Microservices · lesson 6 of 8
Beginner
2Intermediate
5Also part of Microservices
9- + Idempotent Consumers: Surviving a Duplicate Delivery from Event-Driven Architecture
- + Transactional Outbox: Committing the Event With the Data from Event-Driven Architecture
- + Scaling & Reliability: Retries With Backoff and Jitter from System Design
- + Rate Limiting: Token Bucket and the Fixed-Window Boundary Bug from System Design
- + Docker Compose from Docker
- + Deploying FastAPI with Docker and Kubernetes: Images, Signals, and Probes from FastAPI
- + OpenTelemetry Buffers Spans — a SIGKILL Can Lose Them from Monitoring & Observability
- + Events vs Requests: Why the Order Service Doesn't Wait for Email from Event-Driven Architecture
- + Contract Testing with OpenAPI, and Running in CI from API Testing
Also part of: Event-Driven Architecture
Saga Pattern: Orchestrating a Multi-Step Transaction
No single database transaction spans order, payment, and inventory services — a saga orchestrator runs each step in order and tracks what's completed.
By SkillByExample · Published
Three services, three separate databases, one logical transaction — a saga orchestrator is what makes “all three succeed or none of them do” possible without a shared database transaction.
Every step succeeds
class SagaStep:
def __init__(self, name, action, compensation):
self.name = name
self.action = action
self.compensation = compensation
class SagaOrchestrator:
def __init__(self, steps):
self.steps = steps
def execute(self):
completed = []
for step in self.steps:
print(f"Executing: {step.name}")
try:
step.action()
except RuntimeError as e:
print(f"Step '{step.name}' failed: {e}")
self._compensate(completed)
return False
completed.append(step)
print("Saga completed successfully")
return True
def _compensate(self, completed):
print("Rolling back completed steps...")
for step in reversed(completed):
print(f"Compensating: {step.name}")
step.compensation()
def create_order():
print(" order #1001 created")
def cancel_order():
print(" order #1001 cancelled")
def charge_payment():
print(" charged $49.99")
def refund_payment():
print(" refunded $49.99")
def reserve_inventory():
print(" reserved 1x widget")
def release_inventory():
print(" released 1x widget")
steps = [
SagaStep("create-order", create_order, cancel_order),
SagaStep("charge-payment", charge_payment, refund_payment),
SagaStep("reserve-inventory", reserve_inventory, release_inventory),
]
SagaOrchestrator(steps).execute()
$ python saga_happy.py
Executing: create-order
order #1001 created
Executing: charge-payment
charged $49.99
Executing: reserve-inventory
reserved 1x widget
Saga completed successfully
Each SagaStep pairs a forward action with its own compensation, but on the happy path the
compensations are never even called — completed grows with every successful step, and the
orchestrator only reaches into it if something fails. That asymmetry (compensation code that
exists but doesn’t run 99% of the time) is exactly why sagas are easy to under-test in
practice: the failure path is the one that matters most and runs least often.
Practice
1. Add a fourth step, sending a confirmation email, with a no-op compensation.
def send_email():
print(" confirmation email sent")
def no_compensation():
print(" (no compensation needed for email)")
steps = [
SagaStep("create-order", create_order, cancel_order),
SagaStep("charge-payment", charge_payment, refund_payment),
SagaStep("reserve-inventory", reserve_inventory, release_inventory),
SagaStep("send-confirmation-email", send_email, no_compensation),
]
SagaOrchestrator(steps).execute()
$ python fourth_step.py
Executing: create-order
order #1001 created
Executing: charge-payment
charged $49.99
Executing: reserve-inventory
reserved 1x widget
Executing: send-confirmation-email
confirmation email sent
Saga completed successfully
The orchestrator’s code didn’t change at all — SagaStep and SagaOrchestrator don’t know or
care how many steps there are, only that each one has an action and a compensation.
2. Measure how long the whole saga takes to run.
import time
def create_order():
time.sleep(0.1)
print(" order #1005 created")
def charge_payment():
time.sleep(0.2)
print(" charged $60.00")
steps = [
SagaStep("create-order", create_order, cancel_order),
SagaStep("charge-payment", charge_payment, refund_payment),
]
start = time.perf_counter()
SagaOrchestrator(steps).execute()
print(f"Total saga time: {(time.perf_counter() - start) * 1000:.0f}ms")
$ python timed_saga.py
Executing: create-order
order #1005 created
Executing: charge-payment
charged $60.00
Saga completed successfully
Total saga time: 301ms
Steps run strictly in sequence here — the total is the sum of each step’s time (100ms + 200ms), not the maximum. A saga with independent steps could run them concurrently instead, but that trades away the simple “step 2 only starts once step 1 is confirmed done” guarantee.
3. Build the step list conditionally — only include a discount step if a coupon code was provided.
def build_steps(coupon_code):
steps = [SagaStep("create-order", create_order, cancel_order)]
if coupon_code:
def apply_discount():
print(f" applied coupon {coupon_code}")
def remove_discount():
print(f" removed coupon {coupon_code}")
steps.append(SagaStep("apply-discount", apply_discount, remove_discount))
else:
print("No coupon provided, skipping discount step")
return steps
SagaOrchestrator(build_steps(None)).execute()
print()
SagaOrchestrator(build_steps("SAVE10")).execute()
$ python conditional_steps.py
No coupon provided, skipping discount step
Executing: create-order
order #1001 created
Saga completed successfully
Executing: create-order
order #1001 created
Executing: apply-discount
applied coupon SAVE10
Saga completed successfully
The saga’s shape isn’t fixed at definition time — build_steps decides which steps even exist
before the orchestrator ever sees the list, which is how real sagas handle optional stages
like gift-wrapping or loyalty-point redemption.
4. Retry a flaky step up to 3 times before letting it fail the saga.
attempts = {"count": 0}
def flaky_charge():
attempts["count"] += 1
if attempts["count"] < 3:
raise RuntimeError(f"payment gateway busy (attempt {attempts['count']})")
print(" charged $40.00 on attempt", attempts["count"])
def with_retry(action, max_attempts=3):
def wrapped():
last_error = None
for attempt in range(1, max_attempts + 1):
try:
return action()
except RuntimeError as e:
last_error = e
print(f" retrying after: {e}")
raise last_error
return wrapped
steps = [
SagaStep("create-order", create_order, cancel_order),
SagaStep("charge-payment", with_retry(flaky_charge), refund_payment),
]
SagaOrchestrator(steps).execute()
$ python retry_step.py
Executing: create-order
order #1001 created
Executing: charge-payment
retrying after: payment gateway busy (attempt 1)
retrying after: payment gateway busy (attempt 2)
charged $40.00 on attempt 3
Saga completed successfully
with_retry wraps an action in a new function with the same signature the orchestrator
already expects, so SagaStep never needs to know retries are happening underneath it — the
retry logic and the compensation logic stay completely separate concerns.
Next: the failure path — a step failing partway through, and the orchestrator actually rolling back what already succeeded.