Making it scale, and keeping it maintainable

You'll be able to: turn a throughput measurement into a defensible user-capacity number; tell which component a load test actually saturated; and stage a scaling plan by trigger rather than by ambition.

What it can serve today, measured

Assertions about headroom are cheap, so here is the platform under real load. Measured 2026-07-18 against the production deployment, one replica, driven from another node inside the cluster so the CDN and the home uplink are out of the picture — this is the origin's own ceiling.

Memory under sustained load: ~36 MiB, against a 256 MiB limit. The application never came close to its memory cap, and never used one of the node's eight cores.

The saturation curve

Throughput is only meaningful next to the latency it costs. Driving /api/synapse/index harder:

The knee is around 200 connections. Past it, throughput falls while latency climbs — the classic signature of a saturated system spending its time on queueing rather than work. 1,690 req/s is the number; anything beyond it is worse in both dimensions.

From requests per second to readers

A req/s figure is not a capacity answer until it is joined to a model of what a reader does. Mine, stated so you can substitute your own:

• An actively reading person issues about 1 content request every 30 seconds — navigating to a lesson, opening a section. Assets are immutable-cached, so they are free after first load.
• A reader has about 4 sessions a month, roughly 20 content requests each.
• Traffic is not uniform, so peak ≈ 10× average.

Concurrent readers

At 1,690 req/s, with every single request missing the cache:

1,690 req/s ÷ (1 request / 30 s per reader) ≈ 50,000 concurrent active readers

Monthly readers

1,000,000 monthly readers × 4 sessions × 20 requests = 80,000,000 requests/month
80,000,000 ÷ 2,592,000 seconds                       ≈ 31 req/s average
× 10 peak factor                                     ≈ 310 req/s peak

So one million monthly readers is about 310 req/s at peak — against a measured origin ceiling of 1,690 req/s with the cache switched off entirely. That is roughly 5.5 million monthly readers on a single replica with no CDN at all.

Turn the cache back on and the read path stops being a question. At the ~99% cache-eligibility this content has, the origin sees about 3 req/s per million monthly readers, and the arithmetic runs off into numbers not worth printing.

Where the cache hit rate starts to matter

The origin's egress rides a domestic uplink, so a falling hit rate is felt as bandwidth long before it is felt as CPU. At one million monthly readers and 33 KB responses:

Which is why "cache hit rate falling" is a trigger below, and why it is a more urgent signal than CPU on this deployment specifically.

The judge is the real ceiling

Reads scale by not being answered. Runs cannot be cached — they execute untrusted code — so this is where CPU genuinely goes.

Measured through the production sandbox, a small Python program reports:

{"status":"Accepted","stdout":"4950\n","timeSeconds":0.012044,"memoryKb":3584}

12 ms of CPU and 3.5 MB per trivial run. The adapter caps in-flight runs at a semaphore of 8, deliberately, so the sandbox cannot be swamped:

At a realistic 200 ms mean, ~40 runs/s. If runs are ~1% of traffic, that supports roughly 4,000 req/s overall — about 13 million monthly readers on the same model as above.

The semaphore is a chosen limit, not a hardware one: the sandbox node has 18 cores, so raising it is a config change once there is evidence to justify it. The right time to raise it is when queueing delay appears, not before — the cap exists so that a burst degrades into waiting rather than into thrashing.

What breaks first, in order

Given the above, an honest ranking of what actually limits this platform today:

Origin CPU — the thing most capacity planning starts with — is last, by a wide margin. Everything above it is a structural or operational limit, not a throughput one.

The staged plan, gated by triggers

Each stage names the observation that justifies the work. Until the trigger fires, the work is speculative — and given the numbers above, most of it is a long way off.

Today → tens of thousands of monthly readers

No capacity work. The measured headroom covers this by two orders of magnitude. The work that is justified is availability, which the numbers say is the binding constraint:

Thousands to tens of thousands — trigger: cache hit rate falling, or origin CPU sustained above 50%

The ordering matters. Replicas are not first: two replicas would silently double the effective rate limit and leave abandoned submissions unswept until some instance happened to boot. The prerequisites are the work; the replica count is a one-line change afterwards.

Hundreds of thousands — trigger: judge queueing delay appears, or write latency rises

Extracting execution is cheap by construction: its port is already a trait, its adapter already speaks HTTP to a separate process, and it already runs on its own node. It is a wiring change, not a rewrite — which is the whole return on having drawn that boundary early.

Millions and beyond — trigger: a single database no longer absorbs the write rate

Sharding is last and hedged on purpose. At a million monthly readers, submissions arrive at well under one per second; time-partitioning solves the growth problem long before key-sharding is needed, and sharding buys a distributed-systems problem that is very hard to give back.

What stays the same at every stage

The read path. It is edge-cached, derived from a content commit, and reconstructable from a git repository — so it scales by not being answered. Ten times the readers is ten times the cache hits and roughly the same origin load.

That is the highest-leverage property in the system, and it came from one decision made at the start: content is derived data, not database rows. Every capacity number in this chapter is comfortable because of it.

Maintainability is automated or it does not exist

A one-person codebase decays quietly, because there is no reviewer to notice. Six gates run in CI, each catching a specific kind of drift:

They share the property worth copying: each fails the build on a measurable threshold, not on judgement. "Keep files focused" is advice nobody can enforce. "This file is 512 lines and the limit is 500" is a build failure with an obvious next action.

Two of them changed shape when the web tier did, which is the maintenance cost of gates that nobody warns you about. The purity grep was pointed at a Rust client that no longer exists and had to be re-aimed at the visualisation engine — a gate whose subject disappears reports green forever. And the bundle budget was replaced rather than adjusted: with per-page assets there is no bundle to cap, so the threshold moved from 700 KiB of critical path to 250 KiB per page kind. A gate is a measurement, and a measurement outlives its instrument only if someone checks.

The file-size cap has earned its keep — a client file reached 889 lines and the cap forced the split into logic, state and view it should have had from the start. The gate did not just detect a problem; it named the fix.

A gate that does not work

Honesty requires naming one that fails. A test diffs the generated API description against a committed snapshot, so contract drift should be a red test. In practice that snapshot documents one endpoint out of roughly twenty, so it has never caught anything.

That is worse than no gate, because it appears in the list above and produces confidence without coverage. A gate that cannot fail is decoration. It should be regenerated in full or deleted — leaving it as ornament is the one option that should be off the table.

The documentation that stays true

Two mechanisms keep the docs from rotting, and both work by removing the option to drift.

The architecture model lives in the content repository and is compiled by the same push that publishes this prose. The chapter and its diagrams cannot disagree, because they are the same commit.

The build book is immutable. Each chapter documents one step and is not retrofitted; later changes are documented forward. That sounds like a recipe for stale docs and is the opposite: a chapter that claims to describe a moment in time is accurate about that moment, whereas a continuously-edited document claims to be current and quietly stops being so.

The failure mode this avoids is the one every project has: a README that was true two years ago, with no way to tell which sentences still hold.