The Master Template: Write to a Queue, Read from a Cache

Most large systems share the same basic design: writes go to a message queue, and reads come from a cache. This template combines the fixes from the previous lesson: stateless servers that replicate, a queue that keeps responses fast, and data copies that serve reads. It serves as a starting point for most design problems rather than a final answer.

Why Reads and Writes Take Different Paths

A read does not change data, so any current copy can answer it. A write changes data, so it must reach the source of truth in a defined order. Because of this difference, the template separates reads and writes so each path scales in the way that suits it.

The two paths also carry different loads because most products read much more than they write. A post is written one time and read many times. Consequently, the read path must be fast and cheap, while the write path must be reliable.

The Write Path

The write service receives a write, checks it, puts it on the message queue, and answers the client without updating the database itself. Workers that read from the queue do that work:

1. The database updater writes the change to the database, which is the source of truth.
2. The cache updater updates the data in the cache so later reads find the new value.

The queue solves two problems, starting with a difference in speed. Clients can send writes faster than the database can apply them, especially in a burst. The queue holds the extra writes while workers process them at the rate the database can handle.

Press Burst below to send a few seconds of extra traffic, and compare the two modes. Without a queue, the server cannot keep up and drops requests. With a queue, the queue grows during the burst and drains afterward.

The second problem is failure: if the write service calls the workers directly and a worker crashes, the write is lost. A message queue stores each message durably until a worker confirms that it has processed it. If a worker fails, another worker processes the message later, as the Message Queue lesson explains in detail.

The Read Path

The read service answers reads from the cache, which keeps data in memory. Reading from memory is much faster than querying a database, and the cache absorbs most of the read load. The database remains the source of truth, even though most reads never reach it.

The cache updater fills the cache as writes happen, so the data is often ready before anyone asks for it. If the data is not in the cache, the read is a miss. The read service then reads from the database and stores the result in the cache for the next reader.

Stateless Services

The write service and the read service keep no data between requests, so any instance can handle any request. Adding more instances behind the load balancer handles more load. The state lives in the queue, the cache, and the database, which scale in their own ways.

Each database belongs to one service, and other services and clients call the service's API instead of querying the database directly. This boundary lets the owning service change its storage without breaking its callers.

The three instances are identical and keep nothing between requests, while the state stays in the cache and the database.

The Cost: Reads Can Lag Behind Writes

Because the client receives an answer before the workers update the database and the cache, a read can return the old value for a short time. Most products accept this stale-read trade-off from the previous lesson, since a new post appearing a second later is not a problem.

Change the price below, then step time forward. The buyer reads the price from the cache once per second. Count the reads that return the old price after the seller saw "saved". Then raise the delay, and switch to the synchronous path to compare.

With the template, the seller gets an answer at once, and the buyer sees the old price until a worker applies the change. With the synchronous path, no read returns an old price after "saved", but the seller waits for the full delay on every write.

Some operations cannot accept this delay because a payment, a seat reservation, or a stock order must confirm the result before answering. For these operations, the write path writes to the database in a transaction and waits for the result. A common design uses the template for most features and a synchronous path for the few that need it.

Using the Template

The template fits many problems with small changes:

• Twitter: when a user posts, workers copy the post into the cached timeline of each follower, and opening the app reads that timeline from the cache.
• Chat app: a message goes on a queue. If the receiver is online, a worker sends it through their open connection, and if not, it waits in the receiver's inbox.
• YouTube: an upload puts a job on a queue for workers to convert the video into several formats. Viewers then read video data from a CDN, a cache near the users.

In each case, the first step of the design applies the template. The rest of the interview is about what the template does not solve. Common examples include a celebrity with millions of followers or a video file that is too large for a single request.

Key idea. Write to a queue and read from a cache so the queue keeps writes fast and safe during bursts while the cache keeps reads fast and cheap. The cost is that reads can lag behind writes, so operations that need the current value use a synchronous path.