Availability
Availability is the fraction of time that a system works and answers requests. A requirement like "the service must be highly available" is not useful until it has a number. This lesson shows how to measure availability, what common targets allow, and how a design reaches them.
Measuring Availability
Availability is operational time divided by total time:
Availability (%) = (Total time − Downtime) / Total time × 100
A year has 365 × 24 × 60 = 525,600 minutes. If a system is down for 500 minutes in a year, its availability is (525,600 − 500) / 525,600 ≈ 99.905%.
The Nines
Targets are usually stated as a number of nines, where "three nines" is 99.9% and "five nines" is 99.999%. Each extra nine divides the allowed downtime by ten. The table below shows the downtime each target allows, calculated as (1 − availability) × the length of the period.
| Availability | Downtime per year | Downtime per month | Downtime per day |
|---|---|---|---|
| 90% (one nine) | 36.5 days | 3.0 days | 2.4 hours |
| 99% (two nines) | 3.7 days | 7.3 hours | 14.4 minutes |
| 99.9% (three nines) | 8.8 hours | 43.8 minutes | 1.4 minutes |
| 99.99% (four nines) | 52.6 minutes | 4.4 minutes | 9 seconds |
| 99.999% (five nines) | 5.3 minutes | 26 seconds | 0.9 seconds |
At four or five nines, a person cannot react in time. One failure that needs an engineer to wake up and log in can use the entire yearly budget. High targets therefore require automatic recovery, which the rest of this lesson builds.
Service Level Agreements
A Service Level Agreement (SLA) is a contract between a service provider and its customers. It states a target, such as 99.9% uptime per month, and what the provider gives back if it misses the target, usually service credits. Cloud providers publish an SLA for each service:
At the end of each period, the provider compares the measured uptime against the target:
Providers also publish targets outside the SLA. Object storage services, for example, state a durability target, which is the chance that a stored object is not lost. Amazon S3 is designed for 11 nines of annual durability (99.999999999%). Availability measures whether you can read the object now, while durability measures whether the object still exists.
Why Systems Fail
A design must survive two kinds of failure:
- Failures of the system's own parts. A disk breaks, a server crashes, a deploy has a bug, a disk fills up, or a whole data center loses power.
- Failures that come from outside. A client sends a flood of requests, or a dependency such as a payment provider stops answering.
Outside failures require protection at the system's edges: a rate limiter against request floods, plus timeouts, retries, and circuit breakers around each dependency. Internal failures require redundancy, which the next sections explain.
Redundancy
Redundancy means running more than one copy of each part. A tier with several copies fails only when all its copies fail at the same time. However, a request passes through several tiers in series—such as a load balancer, an app server, and a database—and needs every tier up. Because all tiers must work, their availabilities multiply.
Change the number of copies per tier below and watch the total availability. One tier with a single copy limits the whole path.
Two rules come from the widget. First, tiers in series lower total availability, so a path with more tiers needs more reliable tiers. Second, copies in parallel raise a tier's availability fast, but only if they fail independently. Two servers on the same power supply, or two copies running the same bad deploy, fail together. The rest of this lesson shows how each tier gets independent copies.
Stateless Servers Behind a Load Balancer
A load balancer sends each request to one of several identical app servers. If the servers keep no session data in memory, any server can handle any request. When a server fails, it does not take the service down. Instead, the load balancer stops sending it requests, and the remaining servers handle the traffic.
Health Checks
The load balancer finds a failed server using health checks by calling a health endpoint on each server every few seconds. After a set number of failed checks in a row, the load balancer marks the server unhealthy and stops sending it traffic. When the checks pass again, the load balancer adds the server back into rotation.
The check interval and the failure count determine how fast the system detects a failure. Shorter values detect failures sooner, but a brief network delay can then cause the load balancer to remove a healthy server.
Database Replication and Failover
Because a database holds state, a replacement copy must have the same data. With replication, the primary database copies each change to a standby. If the primary fails, the system promotes the standby to primary, and the app servers connect to it. This process is called failover. Writes that the primary accepted but did not yet copy can be lost, unless replication is synchronous, where each write waits for the standby.
Availability Zones and Regions
Copies in the same building share power, cooling, and networking. Cloud providers split each region into availability zones, which are one or more separate data centers with independent power and networking. Zones sit close enough together for fast replication, and deploying copies across two or more zones protects against a single data center failure.
If zone A fails, the load balancer sends all traffic to zone B, and the standby database becomes the primary. A region is a group of zones in one geographic area. Deploying across several regions also protects against the loss of a whole region. Because regions are far apart, however, replication between them is slower, and the design must handle data that lags behind.
Making the Load Balancer Redundant
The load balancer itself is a tier, so a single load balancer is a single point of failure. A common open-source setup runs two HAProxy machines with Keepalived, where clients connect to one virtual IP address. Keepalived assigns the address to the active machine and moves it to the backup when the heartbeat between them stops. Clients continue using the same address.
Cloud providers sell these same building blocks as managed services. A managed load balancer spans zones, a managed database runs a standby in a second zone with automatic failover (such as Amazon RDS Multi-AZ), and an instance group replaces failed servers. In an interview, service names matter less than the mechanism: copies in separate zones, health checks, and automatic failover.
Availability and Fault Tolerance
Fault tolerance is a system's ability to keep working correctly when a part fails, while availability is the result you measure. Redundancy, health checks, and failover make a system fault tolerant, which gives it higher availability. In most interviews, a question about fault tolerance asks what happens when each part fails and what takes over.
The widget below puts the mechanisms together. Kill a server and press Play. Watch the health checks mark it unhealthy, the traffic move to the other servers, and the failed requests stop. Then kill a whole zone. Last, set a short interval with one failed check, and slow the network to B1.
The failed requests come only from the time needed to detect the failure, and a shorter interval or smaller failure count ends them sooner. However, the slow-network case shows the trade-off: the load balancer removes a healthy server, forcing the remaining servers to take its load.
Key idea. Each extra nine divides allowed downtime by ten, so high availability requires automatic recovery. Give every tier independent copies, detect failures with health checks, and fail over without a person in the loop.