Back-of-the-envelope Resource Estimation

Estimating resources for system design involves understanding the load parameters like daily active users (DAU) and translating them into computing and storage needs. It's crucial for identifying performance bottlenecks and designing scalable systems.

Key Load Parameters

  • Daily Active Users (DAU): Translates to requests per second for engineering calculations.

  • Requests Per Second: Helps estimate database and cache size requirements.

  • Read/Write Ratio: Influences the design of read and write paths, affecting decisions on caching, replication, and data store selection.

  • Concurrent Connections: Critical for applications like chat services, requiring strategies for managing web socket connections.

Estimating Storage and Performance

Design Considerations Based on Load

  • High Read/Write Ratio: Implement caching and replication for efficient read operations.
  • High Write/Read Ratio: Optimize for write operations using high-throughput data stores and eventual consistency.

Throughput and Latency

  • Throughput: Measures system capacity to handle data or transactions over time. Essential for assessing system performance.
  • Latency: Time taken for a request to travel from sender to receiver and back. Critical for real-time applications.

Response Time

  • Includes both latency and server processing time. Key for a smooth user experience across various applications.

Example: Online Content Platform

  • DAUs: 500,000
  • Reads/Writes per User: 5 reads and 1 write per day
  • Average Blog Post Size: 50 KB
  • Data Retention: 10 year

Requests Per Second:

  • Reads: ≈ 25/s
  • Writes: ≈ 5/s

Daily Storage Requirement:

  • ≈ 25 GB/day

10-Year Storage Requirement:

  • ≈ 93 TB

The usage of Estimator as bellow:

Resource Estimator Example

Additionally, the "Precision Mode" toggle is off by default, using 100,000 seconds per day and data sizes in 1000-byte units for rough estimates. When enabled, it switches to precise calculations with 86,400 seconds per day and 1024-byte units for data sizes.

Daily Active Users
1M
Read:write ratio
Write operations per user per day
Data retention (month)
Data per write request
Results:

Read RPS
Write RPS

Storage: 1GB of new storage every day, or around 31GB per month. And total: 31GB.

Check out Back-of-envelope-calculation guide to learn more.

Images

QualitySizeExample
Low10 KBThumbnail, small website images
Medium100 KBWebsite photos
High2 MBPhone camera photo
Very High20 MBRAW photographer image

Videos

QualitySizeExample
Low2 MB/min480p video
Medium20 MB/min1080p video
High80 MB/min4k video

Audio

QualitySizeExample
Low700 KBLow quality Mp3
High3 MBHigh quality Mp3

Bandwidth

BandwidthApplication
80 KbpsVoIP calling
150 KbpsScreen Sharing
0.5 MbpsLive streaming Webinars
3 Mbps720p video / zoom meetings
5 MbpsHD 1080p video streaming (youtube/netflix)
25 Mbps4k Ultra HD video

Datastore Latency

StorageLatency
Disk3ms
SSD0.2ms (15x faster)
Memory0.01ms (300x faster)