QPS and System Design

Request rate is often the first bottleneck in a system, and two metrics commonly measure it in web services and databases.

RPS vs QPS

Requests Per Second (RPS) and Queries Per Second (QPS) measure how many incoming requests a system handles each second at different layers.

RPS: RPS counts the total requests a server receives in one second, including static resources like images and dynamic content like API calls.
QPS: QPS counts the database queries executed in one second, measuring load on the database layer alone.

Because the two metrics count at different layers, RPS counts every request reaching the web servers, while QPS counts only requests reaching the database. A request for an image adds to RPS but not to QPS.

Stateless services that handle web requests are easier to scale than databases, which are often the most constrained part of the system. Engineers therefore monitor QPS closely so they do not overload the system. People often say QPS when they mean RPS, and this lesson uses QPS for both.

To turn daily users into a QPS figure for your own design, use the QPS Calculator.

QPS and System Design

Each QPS range requires a different architecture. An undersized design overloads a component, while an oversized design wastes machines and adds operational work.

Use the widget below to match a design to a load. Raise the load with Load ×2 and watch which part runs out first. Then pick each design and find the range where it fits without waste. The machine capacities are illustrative.

LOW QPS

50 requests/s

101001K10K100K1M

Over-built: 6 machines where "One server" needs 1.

Load balancer ×1 · spreads requests–

App ×2 · stateless, adds servers2%

Cache ×1 · answers 80% of reads0%

Primary DB ×1 · every write lands here5%

Replicas ×1 · share cache misses, at most 48%

Machines in total6

Illustrative model, not benchmarks: an app server handles 1.2K requests/s, a database node 110 queries/s, a cache node 20K requests/s, and one region 100K requests/s. Reads are 90% of requests. Parts that scale out add machines at 70% load. A design is over-built when a smaller design also fits.

The four ranges below describe each design in detail.

Low QPS: 1-100 QPS

This range is typical for small startup applications, early-stage products, small-scale internal tools, and local community websites.

At this level, a traditional monolithic architecture can often suffice because data volumes and traffic remain low. One application server and one relational database are enough.

Common technology choices for this range include:

Backend Architecture: A unified backend framework such as Django (Python) or Express.js (Node.js) can handle this load without extra complexity.
Database: A single relational database such as PostgreSQL or MySQL manages data consistently.
Infrastructure: A single instance or server, such as an AWS EC2 instance or a DigitalOcean Droplet, is usually adequate to run the entire application.

For example, a local online store with a few hundred customers will typically have low QPS. A monolithic application can manage that load without a complex scaling strategy.

Medium QPS: 100-1,000 QPS

This range is typical for growing SaaS applications, popular blogs, and community-driven platforms.

When QPS grows, a monolithic architecture may struggle to meet demand, and database bottlenecks or insufficient compute resources may degrade performance. At this stage, horizontal scaling and decoupling become more important.

Common technology choices for this range include:

Backend Architecture: A modular approach such as microservices lets individual components scale independently to address bottlenecks.
Database Strategy: Techniques like sharding, replication, or caching handle higher load on data storage systems, such as MySQL read replicas or Redis caches.
Infrastructure Management: Horizontal scaling with containerized services and orchestration tools adds flexibility and resiliency, and tools like Docker and Kubernetes can manage these services.

For example, a small social networking site with frequent user interactions, such as posting or liking content, might reach medium QPS.

High QPS: 1,000-100,000 QPS

This range is typical for large e-commerce sites, popular SaaS products, and social media platforms.

At this level, services need high scalability and resilience to handle request surges. Microservices, message queues, redundancy, and distributed systems are commonly adopted to meet these demands.

Common technology choices for this range include:

Backend Architecture: microservices allow components to deploy and scale independently, though they require efficient inter-service communication to maintain performance.
Event-driven architecture: message queues such as Kafka buffer incoming requests and decouple components.
Data Management: distributed data stores like Cassandra or MongoDB scale horizontally, while streaming systems like Apache Kafka handle real-time processing.
In-memory caching: Heavy caching, such as caching search results, reduces database load.
Infrastructure Management: Container orchestration and load balancing distribute traffic at this scale, commonly using Kubernetes and AWS Elastic Load Balancing.

For example, an e-commerce platform handles many concurrent users, especially during sales events. Distributed databases, in-memory caching, and independently scaling services handle this volume together.

Very High QPS: 100,000+ QPS

This range is typical for global-scale platforms such as major social media sites, financial trading systems, and large streaming services.

At this scale, even small inefficiencies can cause severe issues. The architecture must be distributed and must keep working when a machine, a zone, or a region fails.

Common technology choices for this range include:

Globally replicated databases: Replicating data across regions reduces latency.
Multi-region deployments: Running applications across multiple data centers or cloud regions improves resilience.
Edge computing: Running code at edge locations, such as on CDNs, moves execution closer to users.
Serverless architecture: Functions scale automatically to handle unpredictable loads.
Multi-cloud environments: Using resources from multiple cloud providers adds redundancy.

For example, Netflix streams video to millions of concurrent users using a distributed architecture with multi-layer caching and heavy CDN usage. Netflix runs its own CDN, Open Connect, which places video caches inside internet provider networks.

Key idea. Up to about 100 QPS, one application server and one database are enough. From about 100 to 1,000 QPS, add servers, read replicas, and a cache. Above 1,000 QPS, add message queues and distributed data stores, and above 100,000 QPS, run in several regions behind a CDN.