Distributed Locks: A Real Double-Booking Bug, Fixed With a Redis Lock

Two processes can each believe they’re allowed to act, because between checking and acting, nothing stopped a second process from checking too. A distributed lock is how machines that don’t share memory agree on who gets to go first.

The bug: double-booking a single slot

docker run -d --name redis-lesson -p 6379:6379 redis:8-alpine
docker exec redis-lesson redis-cli DEL room:2pm:bookings

Two processes race to book the same 1-seat slot — each checks the current booking count, and only proceeds if there’s room:

# book_no_lock.sh <bidder-name>
count=$(docker exec redis-lesson redis-cli LLEN room:2pm:bookings)
if [ "$count" -lt 1 ]; then
  sleep 0.3   # simulate real processing time — a payment call, a confirmation email
  docker exec redis-lesson redis-cli RPUSH room:2pm:bookings "$1" > /dev/null
  echo "$1: booked (saw count=$count)"
else
  echo "$1: rejected (saw count=$count)"
fi

$ ./book_no_lock.sh alice & ./book_no_lock.sh bob & wait
bob: booked (saw count=0)
alice: booked (saw count=0)
$ docker exec redis-lesson redis-cli LRANGE room:2pm:bookings 0 -1
bob
alice

Both processes call LLEN before either has written anything back, so both see count=0 and both conclude the slot is free. The sleep 0.3 — standing in for real work like charging a card — is what widens the race window enough to hit reliably; a 1-seat slot ends up with 2 bookings.

The fix: a real Redis-based lock

# book_with_lock.sh <bidder-name> <token>
acquired=$(docker exec redis-lesson redis-cli SET lock:room:2pm "$2" NX PX 2000)
if [ "$acquired" != "OK" ]; then
  echo "$1: could not acquire lock, rejected immediately"
  exit 0
fi
count=$(docker exec redis-lesson redis-cli LLEN room:2pm:bookings)
if [ "$count" -lt 1 ]; then
  sleep 0.3
  docker exec redis-lesson redis-cli RPUSH room:2pm:bookings "$1" > /dev/null
  echo "$1: booked (saw count=$count)"
else
  echo "$1: rejected (saw count=$count)"
fi
owner=$(docker exec redis-lesson redis-cli GET lock:room:2pm)
[ "$owner" == "$2" ] && docker exec redis-lesson redis-cli DEL lock:room:2pm > /dev/null

$ docker exec redis-lesson redis-cli DEL room:2pm:bookings lock:room:2pm
$ ./book_with_lock.sh alice token-alice & ./book_with_lock.sh bob token-bob & wait
bob: could not acquire lock, rejected immediately
alice: booked (saw count=0)
$ docker exec redis-lesson redis-cli LRANGE room:2pm:bookings 0 -1
alice

SET key value NX PX 2000 is Redis’s atomic “set only if this key doesn’t already exist, expiring in 2 seconds” — exactly one of alice/bob’s SET calls can win it, so whichever loses (bob, here) is rejected before it ever touches the booking count. The PX expiry matters independently of who wins: if the winning process crashed instead of finishing normally, the lock would still self-expire in 2 seconds rather than jamming the slot forever.

The real danger: unlocking without checking who you are

The unlock above checks owner == my token before deleting — that check isn’t decoration. Here’s what a blind DEL (no check) actually does:

echo "resetting demo key:"
docker exec redis-lesson redis-cli DEL lock:demo

# slow_client: acquires with a short 500ms TTL, then takes 1000ms of real work
echo "slow_client acquiring:"
docker exec redis-lesson redis-cli SET lock:demo slow-client-token PX 500 NX
sleep 1

# meanwhile, 700ms in, slow_client's lock has already expired —
# fast_client legitimately acquires it fresh:
echo "fast_client acquiring after expiry:"
docker exec redis-lesson redis-cli SET lock:demo fast-client-token PX 5000 NX

# slow_client now finishes and blindly deletes, with no idea it's no longer the owner:
echo "slow_client blindly unlocking (no token check):"
docker exec redis-lesson redis-cli DEL lock:demo

echo "lock owner after slow_client's blind DEL:"
docker exec redis-lesson redis-cli GET lock:demo

$ ./blind_unlock_demo.sh
resetting demo key:
0
slow_client acquiring:
OK
fast_client acquiring after expiry:
OK
slow_client blindly unlocking (no token check):
1
lock owner after slow_client's blind DEL:

fast_client’s brand-new lock — with 5 real seconds left on its own TTL — is gone. slow_client never checked whether it still owned the lock before deleting; it just deleted the key, which by that point belonged to someone else entirely. The token check in the earlier fix exists specifically to prevent this: running the same scenario with GET-before-DELETE correctly finds fast-client-token where it expects slow-client-token, and skips the delete.

Making the check-then-delete fully atomic

Checking with GET and then deleting is still two separate round-trips, with a real (if narrow) gap between them. A Lua script run via EVAL closes that gap by making the whole check-and-delete one atomic server-side operation:

docker exec redis-lesson redis-cli SET lock:demo my-token PX 5000 NX
docker exec redis-lesson redis-cli EVAL \
  "if redis.call('GET', KEYS[1]) == ARGV[1] then return redis.call('DEL', KEYS[1]) else return 0 end" \
  1 lock:demo my-token

$ (correct token)
1
$ (wrong token, run against a freshly re-set lock:demo)
0

1 means the script’s DEL actually ran (the token matched); 0 means it didn’t touch the key at all. Real production Redis lock libraries (Redlock’s own reference implementation included) use exactly this pattern rather than a plain DEL.

Practice