Design Rate Limiter

A rate limiter is a system is to prevent a user from making too many requests to a service within a certain time frame. This is used to protect services from abuse and can be implemented in various scopes like per user, per IP address, etc.

Functional requirement:

  • The client can be identified using either the cookie, user ID, or IP address.
  • If the throttling threshold is surpassed, the rate limiter will reject the request and send a "429 Too Many Requests" status code back to the client.
  • If the throttling threshold is not surpassed, the rate limiter will forward the request to the downstream server.

Scale requirement:

  • 100 million daily active users
  • Data retention is 1 day
  • Assuming each user makes 1,000 requests per day.
  • Assuming each request and response are 1 KB in size.
  • We need to store each user's remaining capacity and the timestamp of their last request. Let's say that takes 50 bytes.
1. Resource Estimation