🧰 Redis

Prerequisites: Caching, Data Models | You'll be able to: explain why Redis commands are atomic without locks; pick the right data structure for locks, leaderboards, rate limits, geo search and queues; size a Redis deployment and name what breaks first.

🧨 The problem (why this exists)

"We'll put Redis in front of it" is the most-said sentence in system design, and the least examined.

Naming a technology only earns marks if you can say what it buys and what it costs. Redis is the extreme case: it is so easy to add that people add it without noticing they have introduced a second source of truth with weaker durability than the first.

This lesson answers four questions about it, and every later technology lesson answers the same four:

• What it actually is, underneath the word "cache."
• The mechanisms worth knowing.
• What it costs you.
• When to name it, and when naming it is wrong.

💡 Intuition first

Think of a bank with exactly one teller and no queue-jumping.

Every customer is served start to finish before the next begins. Nobody needs a rulebook for two people touching the same account at once, because that situation cannot arise. The teller is the bottleneck and the safety mechanism, and those are the same fact.

That is Redis. Commands from thousands of connections arrive at a single-threaded execution loop and run one at a time, to completion. This is why INCR is atomic, why SET NX is a usable lock primitive, and why Redis needs no row locks, no transactions in the database sense, and no deadlock detector.

The second surprise is where the speed comes from.

The obvious answer — "it avoids disk" — is wrong, and DDIA says so plainly: a disk-based engine with enough RAM may never read from disk either, because the operating system caches recently used blocks anyway [i]. The real advantage is that Redis never pays to encode its in-memory structures into a form that can be written to disk [i]. A hash map stays a hash map, rather than being serialised into pages.

⚙️ How it works

🔁 One loop, many connections

Redis multiplexes every client socket onto one event loop. A command is read, executed to completion, and its reply queued — then the next.

Two consequences follow directly, and both show up in design discussions:

• Any slow command stalls everything. KEYS * on a large keyspace, a big SORT, or deleting a huge collection blocks every other client for the duration. There is no other thread to make progress.
• Round trips are cheap enough to change your instincts. Firing 100 queries at a relational database to build one list is a mistake; you write one query instead. Against Redis the per-command overhead is low enough that the same pattern is merely inelegant rather than fatal — and pipelining collapses those 100 round trips into one anyway.

🧱 The data structures are the product

Treating Redis as a string-to-string map wastes most of it. DDIA makes the same point from the other side: in-memory storage makes data models practical that are awkward on disk, naming Redis's priority queues and sets as the example [i].

The command names are worth a glance because they are unusually honest: the wire protocol is plain text, and the set commands are close analogues of any language's set type.

🧩 Seven things you can actually build

This is the section that turns "Redis is a cache" into a usable toolbox. Each row is a real design answer, not a feature.

Cache. The default. Read-through or cache-aside in front of a database, with a TTL per key.

Distributed lock. SET key value NX PX 30000 sets the key only if absent and expires it after 30 seconds. The expiry is the important half — it releases a lock whose holder crashed. See the warning about failover below before relying on it for correctness.

Leaderboard. A sorted set keeps members ordered by score and answers rank and range queries in logarithmic time. "Top 100 of five million players" becomes one command. The same shape solves "most-liked posts containing this keyword" — keep a sorted set per keyword and trim the tail periodically to bound memory.

Rate limiter. A fixed-window limiter is two commands: INCR the counter for this caller, and EXPIRE it after the window so it resets. If the returned value exceeds the limit, reject. Because the loop is single-threaded, the increment is atomic without a transaction — this is the whole design in the rate limiter case study.

Proximity search. GEOADD stores coordinates and GEOSEARCH returns everything inside a radius or box. This is how Uber finds nearby drivers without a specialised spatial database.

Work queue and event log. Streams are append-only logs with consumer groups, deliberately close in shape to Kafka topics. Producers XADD; a consumer group tracks which entries have been handed out; and when a worker dies mid-message, another worker can XCLAIM it and finish the job. That claim mechanism is what makes streams a real queue rather than a list you pop from.

Pub/Sub. Publishers send to a channel and every connected subscriber receives it. It is ideal for chat fan-out and live notifications — and it is at-most-once: messages are never stored, so a subscriber that is offline misses them permanently.

One practical note on Pub/Sub connections: a client uses one connection per node, not per channel. A million channels does not mean a million connections.

💾 Persistence: RDB and AOF

Redis writes to disk two ways, and the acronyms hide what they are.

• RDB takes point-in-time snapshots of the entire dataset. The file is compact and loads fast on restart, but everything written since the last snapshot is lost in a crash.
• AOF appends every write command to a log and replays it at startup. The loss window is far smaller, and the fsync policy sets it: every write is safest and slowest, once per second is the usual default.

Many deployments run both — RDB for fast restarts, AOF for a tighter loss window. DDIA classifies Redis exactly here: among in-memory stores it provides weak durability by writing to disk asynchronously [i]. That is a description, not a criticism — it is the trade that buys the speed.

🖧 Deployment: single node, replica, or cluster

Three topologies, and the jump between them is bigger than it looks.

A cluster splits the keyspace into 16,384 hash slots, and each node owns a range of them. Clients cache the slot map so they connect straight to the node holding the key; nodes gossip enough to redirect a misdirected request, but hitting the right node first is the design goal.

🔗 Replication is asynchronous, and that is the whole risk

A primary does not wait for replicas to acknowledge a write before replying to the client. So a write can be acknowledged, the primary can die, and a replica with no knowledge of that write can be promoted.

That is fine for a cache, where the fallback is a slower read from the database. It is not fine for anything you cannot reconstruct — which is why "Redis as the only record of a payment, a booking, or a lock you rely on for correctness" is a design smell. If two clients can believe they hold the same lock after a failover, you need consensus, not a TTL.

⚖️ Trade-offs

🔢 Numbers that matter

The capacity rungs are this book's standard cache figures and are industry practice rather than sourced — see Numbers Quick Reference. The point of memorising them is the comparison: at 20k operations/second you are nowhere near one node's limit, and proposing a cluster is over-engineering.

🏭 In production

🔥 The hot key problem

Imagine caching product details across a 100-node cluster, evenly spread. One item goes viral, and traffic for that single item matches the traffic for everything else combined.

Sharding does not help. That key lives in one hash slot on one node, and that node now carries half the fleet's load while ninety-nine others idle. Unless every node was heavily over-provisioned, it starts failing.

Three remediations, each with a real cost:

• Cache the hot key in the client, so most requests never reach Redis. Cheapest, at the price of a second staleness window you now have to reason about.
• Write the same value under several key names and have clients pick one at random, spreading it across slots. Effective, and every write must now update every copy.
• Add read replicas and scale them with load. Straightforward, and it only helps reads.

What matters in a design discussion is recognising the risk before being asked, and naming a remediation with its trade-off.

💣 Eviction is the failure people actually hit

When Redis reaches maxmemory it applies an eviction policy, and the default in many deployments evicts nothing and starts refusing writes. The two worth knowing are allkeys-<abbr title="Least Recently Used — evicts whatever has gone longest without access">lru</abbr> (evict anything — treat it as a pure cache) and volatile-lru (evict only keys carrying a TTL). Choosing a volatile-* policy while writing keys without expiry produces a cache that fills up and then rejects writes — a slow-motion outage that looks like an application bug.

🛠️ Two habits that separate operators from readers

• Never run KEYS in production. It scans the whole keyspace on the single thread. SCAN exists precisely because it returns a cursor and yields between batches.
• Watch the slow log. Because one command can block every client, the tail latency of the slowest command is the latency of the whole server.

Where it sits in this book. Redis is the concrete instance behind Caching, the seat-hold store in Ticketmaster, the counter in the rate limiter, and the geospatial index in Uber. Four lessons, four different data structures — which is the argument against thinking of it as one thing.

🪤 Pitfalls & interview traps

• Calling it "just a cache." It is a data-structure server whose most common use is caching. The sorted set and the geospatial index are why it wins problems a memcached-shaped store cannot touch.
• Claiming Redis is atomic because it is fast. The causation runs the other way: it is atomic because it is single-threaded, and that same property makes one slow command catastrophic.
• Relying on a Redis lock for correctness. Asynchronous replication means two clients can hold the same lock after a failover.
• Using Pub/Sub where you needed a stream. At-most-once delivery loses messages for anyone offline. If that matters, use streams or a real broker.
• Forgetting that a cluster cannot span slots in one operation. Multi-key commands need their keys co-located, which is a key-design problem, not a configuration flag.
• Reaching for a cluster too early. Below roughly 1 TB or 100k operations/second, one node with a replica is simpler and faster to reason about.

✅ Check yourself

🔬 PoC — Proof of concepts

• redis/redis — the implementation. t_zset.c and t_string.c are short enough to read, and the single-threaded loop is easier to follow in source than in prose.
• Redis persistence documentation — the authoritative statement of what RDB and AOF each guarantee and what the fsync settings cost.
• Redis command reference — grouped by data structure; skimming the sorted-set and stream groups is the fastest way to see what the toolbox actually contains.
• codecrafters-io/build-your-own-x — collects "write your own Redis" walkthroughs; implementing SET, GET, EXPIRE and the wire protocol makes the event loop concrete.

📚 Sources

• DDIA2 ch. 4 p. 134 — in-memory databases: the performance advantage comes from avoiding the encoding of in-memory structures for disk, not from avoiding disk reads; Redis and Couchbase provide weak durability via asynchronous disk writes; in-memory storage enables data models such as Redis's priority queues and sets.
• DDIA2 ch. 6 pp. 197–213 — asynchronous replication and what a leader failover can lose.
• [web: Redis cluster specification] — the 16,384 hash-slot scheme and client-side slot caching.
• [web: Redis persistence documentation] — RDB versus AOF and the fsync policies.
• Throughput, latency and capacity rungs are industry practice figures carried from Numbers Quick Reference, not sourced from DDIA.