Skip to main content
In interviews, you may be asked follow-up or “curveball” questions that change constraints in your initial design. The interviewer wants to see how you adapt without discarding the whole architecture. Below are three common follow-ups for a rate-limiter design, practical solutions you can describe, and trade-offs to consider. Question 1 — The limit isn’t one size fits all You initially assumed a single global limit (for example, 100 requests/minute). In production, different endpoints and principals often need different limits:
  • Public read-only endpoints can tolerate very high throughput (e.g., ~10,000 req/min).
  • Resource-creation endpoints consume more compute and should be throttled tighter (e.g., ~100 req/min).
  • Authentication endpoints must be the strictest to prevent credential-stuffing attacks.
Solution: implement configurable, rule-driven limits instead of a single hard-coded number. At request time, the gateway looks up the applicable rule based on attributes such as:
  • Endpoint path or route pattern
  • HTTP method
  • User role or subscription tier
  • API key or client identifier
Each applicable rule maps to a bucket or policy (token bucket, leaky bucket, fixed window, sliding window) that enforces the limit. Keep the set of rules small and easy to evaluate so lookups are fast. Example rule categories: Question 2 — One Redis isn’t enough at high scale A single Redis instance often cannot handle extreme loads. For example, a single instance might handle on the order of ~100k ops/sec (varies by instance type and command mix). At 1M req/sec you’ll need to shard across multiple Redis instances — roughly 10 shards in a simplified estimate. Rate-limiting is naturally partitionable by principal: Alan’s bucket is independent of John’s bucket. Use sharding keyed by principal (user ID, API key, client ID) so every request for the same principal routes to the same Redis shard. This preserves correctness without cross-shard coordination.
A stylized system diagram showing two users (Alan and John) sending requests through a load balancer to two gateways and corresponding Redis instances. Each Redis box shows per-user request quotas (Alan 74/100, John 100/100).
Operational consideration: an abusive client who has already exhausted their quota can still flood your gateways. If each incoming request triggers a Redis lookup, the shard will be hammered even though all requests are being denied. Sharding increases overall capacity but does not eliminate wasted Redis calls from aggressive clients. Question 3 — Can we avoid a Redis network call on every request? Yes. A common production optimization is to maintain a small local counter in each gateway process and periodically sync those counts to Redis. The gateway increments the in-memory counter on each request and flushes batches to Redis at a fixed interval (for example, every 100–500 ms). Benefits:
  • Fast local rejection: once the gateway determines a principal is blocked locally, it can reject further requests without hitting Redis.
  • Reduced Redis load: batching reduces the number of Redis operations significantly.
Trade-offs:
  • Slightly relaxed global enforcement during the sync window: multiple gateways may accept requests before their local counters are flushed to Redis. Choose the sync interval based on the acceptable error bound vs. performance.
  • Need to manage edge cases: gateway restarts or crashes can lose the in-memory counters unless you use a short sync interval or persist counters on exit.
Comparison of enforcement approaches
Using local counters with periodic synchronization is a practical optimization: it reduces Redis load and improves latency while introducing a small, bounded inaccuracy in enforcement. Tune the sync interval to balance performance and strictness.
Optional mitigations for abusive traffic
  • Early local deny lists: after a gateway locally observes sustained abusive behavior, add the principal to an in-memory deny list and apply a cooldown before re-evaluating in Redis.
  • Backoff and rate-limited error responses: instruct clients to back off (e.g., Retry-After) to reduce repeated hits.
  • Edge filtering: push basic throttling logic as close to the client as possible (CDN, WAF, or edge proxies) to filter obvious abuse before it reaches gateways.
Further reading and references Summary
  • Use configurable rules per endpoint/principal rather than a single global limit.
  • Shard Redis and route by principal to scale to high request rates while keeping correctness.
  • Employ local gateway counters with periodic sync to avoid a Redis call on every request, trading a small accuracy window for large performance gains.
  • Combine local optimizations with defensive measures (deny lists, backoff headers, edge filtering) to reduce wasted work from abusive clients.
This completes the discussion of common rate-limiter follow-ups. Take a moment to internalize these patterns: they are practical, widely used, and easy to explain during system design interviews.

Watch Video