System Design Glossary

Language syntax for code that waits for I/O without blocking the thread. At each await, the runtime runs other tasks until the result is ready.

A fast storage layer that keeps copies of frequently read data. Reads that hit the cache skip the slower source, such as a database.

Removing entries when the cache is full to make room for new ones. A policy such as LRU (least recently used) decides which entry goes.

Also called: LRU

Removing or updating a cached value after its source data changes. Without it, readers see stale data.

Many requests miss the same expired key at once and all hit the database together. The sudden load can overload the database.

Also called: thundering herd, dog-piling

A read pattern where the application checks the cache first. On a miss, it reads the database and writes the result into the cache.

Also called: lazy loading

During a network partition, a distributed system must choose between consistency and availability. It cannot give both.

Also called: PACELC

A network of servers placed near users that cache static content such as images and video. Users download from a nearby server instead of the origin.

Reading a database's change log and publishing each change as an event. Other systems, such as caches and search indexes, stay in sync from that stream.

A guard that stops calls to a failing dependency after too many errors. It fails fast for a period, then tests the dependency again.

An atomic operation that writes a new value only if the current value still equals an expected value. If the value changed, the write fails and the caller retries.

Also called: CAS

Running several tasks in overlapping time periods. The tasks can take turns on one CPU core; they do not need to run at the same instant.

A protocol that lets a group of nodes agree on one value even when some nodes fail. Raft and Paxos are common consensus algorithms.

Also called: Raft, Paxos

The rules a distributed system follows about which value a read can return after a write. Models range from strong to eventual.

A way to map keys to nodes on a hash ring. Adding or removing a node moves only the keys next to it, not all keys.

A block of code that reads and changes shared data. Only one thread can safely run it at a time.

A separate queue for messages that fail processing too many times. It keeps one bad message from blocking the main queue.

Also called: DLQ

Two or more threads or transactions each hold a lock and wait for a lock the other holds. None of them can continue.

A lock that processes on different machines share through a common service. Only one process holds it at a time, and it expires if the holder dies.

A single thread that runs a loop, picks the next ready event, and runs its handler. It serves many I/O tasks without one thread per task.

Replicas can disagree for a short time, but they converge to the same value when writes stop. Reads in that window can return old data.

Sending one event to many receivers, such as delivering a post to every follower's feed. The cost grows with the number of receivers.

A number that increases each time a lock is granted. The storage layer rejects writes that carry an older token, so an expired lock holder cannot corrupt data.

A single key that receives far more traffic than others. All of that traffic lands on the one node that owns the key.

An operation is idempotent when running it twice has the same effect as running it once. It makes retries safe, for example no double charge.

Also called: idempotent

A unique ID the client sends with a request. The server stores the result under that key and returns it again for a retry.

A setting that controls what one transaction can see of other transactions that run at the same time. Stronger levels block more anomalies but allow less concurrency.

One node (the leader) accepts writes and sends changes to the other nodes (followers). Followers serve reads and can replace a failed leader.

Also called: primary-replica, master-slave

A queue that appends messages to an ordered, durable log. Consumers track their own position in the log and can replay old messages.

Also called: event log, Kafka

Two writers read the same value, change it, and write it back. The second write replaces the first, so the first change disappears.

A buffer that holds messages from producers until consumers process them. It decouples services and absorbs traffic spikes.

A lock that lets only one thread enter a critical section at a time. Other threads wait until the owner releases it.

Also called: mutual exclusion, lock

Reading without a lock and checking a version number at write time. If another writer changed the row, the write fails and the caller retries.

Running several tasks at the same instant on different CPU cores or machines. Parallelism is one way to get concurrency.

Locking a row before reading it so no other transaction can change it until the lock is released. It prevents conflicts but makes other writers wait.

Also called: SELECT FOR UPDATE

A running program with its own memory space. Processes do not share memory unless they set it up on purpose.

A pattern where producers put work into a shared queue and consumers take work from it. The queue lets both sides run at different speeds.

A bug where the result depends on the timing of operations that run at the same time. For example, two buyers both read stock=1 and both buy the last item.

Limiting how many requests a client can make in a time window. Requests over the limit get rejected, usually with HTTP 429.

Also called: throttling

Sending writes to the primary database and reads to replicas. It scales read traffic without changing the write path.

Keeping copies of the same data on several machines. Copies give fault tolerance and extra read capacity.

The delay between a write on the leader and the same write on a follower. A read from a lagging follower returns old data.

Retrying a failed call with a wait that doubles after each attempt. Random jitter spreads retries so clients do not retry at the same moment.

Also called: backoff, jitter

A counter that lets up to N threads use a resource at the same time. A thread takes a permit to enter and returns it when done.

The field used to decide which shard stores a row, such as user_id. A poor choice puts too much data or traffic on one shard.

Also called: partition key

Splitting one dataset across several databases so each holds a part of the data. Each part is a shard, and a shard key decides where a row goes.

Also called: partitioning, horizontal partitioning

Every read returns the most recent completed write, as if there were one copy of the data. Linearizability is the strictest form.

Also called: linearizability

A unit of execution inside a process. Threads in the same process share memory, which makes data exchange fast but needs coordination.

A fixed set of threads that take tasks from a queue. It reuses threads and caps how many tasks run at once.

A limit on how long a caller waits for a response. After the limit, the caller stops waiting and handles the call as failed.

A rate limiting algorithm where a bucket refills with tokens at a fixed rate. Each request spends one token, so saved tokens allow short bursts.

A group of database operations that succeed or fail as one unit. If any step fails, the database rolls back all of them.

Also called: ACID

An expiry time attached to a cached value. After the TTL passes, the cache deletes the value and the next read fetches fresh data.

A write pattern that writes to the cache first and flushes to the database later. Writes are fast, but a cache crash can lose unflushed data.

Also called: write-behind

A write pattern that updates the cache and the database in the same write path. The cache stays current, but each write pays for both stores.