Redis · lesson 11 of 14
Beginner
4Intermediate
7- 5 MULTI/EXEC Transactions and the WATCH-Based Optimistic Lock
- 6 Pub/Sub: Channels, Patterns, and the Message That Never Arrives
- 7 RDB vs AOF: What Survives a Hard Kill and What Doesn't
- 8 Replication: A Read-Only Replica, and What WAIT Actually Guarantees
- 9 ACL: a User Restricted to One Key Pattern and One Command
- 12 maxmemory and Eviction: What Redis Throws Away When It's Full
- 13 Cache-Aside with Redis: Stale Reads, Races and Stampedes
Advanced
3Also part of Redis
3Clustering: Hash Slots, MOVED, and the CROSSSLOT Error
A real 3-node cluster, a real MOVED redirect from writing to the wrong node, and a real CROSSSLOT error the first time a multi-key command touches two different slots — fixed with a hash tag.
By SkillByExample · Published
Three real nodes, one keyspace split between them — and two errors that only show up once data actually spans more than one node.
Bootstrapping a real cluster
docker network create redis-net
for i in 1 2 3; do
docker run -d --name redis-node$i --network redis-net -p 700$i:6379 \
redis:8-alpine redis-server --cluster-enabled yes --cluster-config-file nodes.conf
done
# get each node's address on the docker network
for i in 1 2 3; do
docker inspect -f '{{(index .NetworkSettings.Networks "redis-net").IPAddress}}' redis-node$i
done
172.19.0.3
172.19.0.4
172.19.0.5
docker exec redis-node1 redis-cli --cluster create \
172.19.0.3:6379 172.19.0.4:6379 172.19.0.5:6379 \
--cluster-replicas 0 --cluster-yes
>>> Performing hash slots allocation on 3 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
...
[OK] All nodes agree about slots configuration.
>>> Check slots coverage...
[OK] All 16384 slots covered.
Three masters, no replicas (--cluster-replicas 0, fine for this lesson — production
clusters want replicas per master for failover), 16384 slots split roughly evenly across
them.
Every key hashes to a slot, deterministically
docker exec redis-node1 redis-cli CLUSTER KEYSLOT book:1
docker exec redis-node1 redis-cli CLUSTER KEYSLOT book:2
759
12948
book:1 lands in slot 759 — owned by node 1 (range 0-5460). book:2 lands in slot
12948 — owned by node 3 (range 10923-16383). Same hashing formula, same result on any
node; nothing needs to ask anyone else where a key belongs.
The bug: writing to the wrong node
docker exec redis-node1 redis-cli SET book:1 "Dune"
docker exec redis-node1 redis-cli SET book:2 "Foundation"
OK
MOVED 12948 172.19.0.5:6379
book:1 succeeded — node 1 actually owns slot 759. book:2 was refused with MOVED — node 1
doesn’t own slot 12948, and it tells you exactly which node does (172.19.0.5:6379, node 3).
The fix: a cluster-aware client follows the redirect
docker exec redis-node1 redis-cli -c SET book:2 "Foundation"
docker exec redis-node1 redis-cli -c GET book:2
OK
Foundation
-c puts redis-cli itself into cluster mode — it catches the MOVED reply, reconnects to
the node it points to, and retries automatically. A production client library does the same
thing under the hood, which is what “cluster-aware” actually means in practice.
The second bug: a multi-key command spanning two slots
docker exec redis-node1 redis-cli -c MSET book:1 "Dune" book:2 "Foundation"
CROSSSLOT Keys in request don't hash to the same slot
Even in cluster mode, this fails — -c can follow a redirect to one node, but it can’t make
one MSET atomically span two different nodes. book:1 and book:2 hash to different slots,
so the command is refused outright.
The fix: hash tags force keys to the same slot
docker exec redis-node1 redis-cli CLUSTER KEYSLOT "{books}:1"
docker exec redis-node1 redis-cli CLUSTER KEYSLOT "{books}:2"
docker exec redis-node1 redis-cli -c MSET "{books}:1" "Dune" "{books}:2" "Foundation"
15901
15901
OK
Both keys hash to slot 15901 this time — a {books} hash tag makes Redis hash only the text
inside the braces, ignoring the rest of the key name, so both keys deliberately land on the
same slot and the same node. Any keys meant to be touched together in one multi-key command
need a shared hash tag like this.
Practice
1. Check overall cluster health without inspecting individual nodes.
docker exec redis-node1 redis-cli CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:3
cluster_state:ok plus all 16384 slots assigned and ok is the quick health check — any
other state, or slots not fully covered, means the cluster isn’t safely serving all keys.
2. List every node and the exact slot ranges each one owns.
docker exec redis-node1 redis-cli CLUSTER NODES
ffb0cbc3e3944a1dce983b724be3e3c97e961d49 172.19.0.5:6379@16379 master - 0 1789314581000 3 connected 10923-16383
590646e7a035f1746ec0cdf8975f856ac73ace01 172.19.0.3:6379@16379 myself,master - 0 0 1 connected 0-5460
fd62f0a8d49d90f954a29f5896d3bdeb684dfe25 172.19.0.4:6379@16379 master - 0 1789314582339 2 connected 5461-10922
Node ID, address, role, and slot range per line — myself marks which node you’re actually
connected to, useful when comparing what each node in the cluster believes about the others.
3. Count how many keys currently live in a specific slot.
docker exec redis-node1 redis-cli CLUSTER COUNTKEYSINSLOT 759
1
Just book:1 — the only key in this lesson that hashed to slot 759. This is one of the
primitives resharding tools use to move keys slot-by-slot between nodes without guessing at
what’s actually stored where.
That closes the track. The thread through all eleven lessons: Redis trusts you to use the
right command for a key’s actual type and the right client behavior for its actual topology —
WRONGTYPE, the -1/-2 TTL split, a WATCH-aborted transaction, a lost PUBLISH, a hard
kill losing unset-AOF data, a NOPERM, a NOSCRIPT, and finally a MOVED/CROSSSLOT pair are
all the same lesson in different clothes: Redis tells you exactly what’s wrong, in detail,
every time — the skill is reading the message instead of guessing.