> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Curveball Byte Rate Limiter
> Discusses rate limiter follow ups including configurable rules, Redis sharding, local gateway counters, trade offs, and mitigations for abusive traffic.
In interviews, you may be asked follow-up or “curveball” questions that change constraints in your initial design. The interviewer wants to see how you adapt without discarding the whole architecture. Below are three common follow-ups for a rate-limiter design, practical solutions you can describe, and trade-offs to consider.
Question 1 — The limit isn't one size fits all
You initially assumed a single global limit (for example, 100 requests/minute). In production, different endpoints and principals often need different limits:
* Public read-only endpoints can tolerate very high throughput (e.g., \~10,000 req/min).
* Resource-creation endpoints consume more compute and should be throttled tighter (e.g., \~100 req/min).
* Authentication endpoints must be the strictest to prevent credential-stuffing attacks.
Solution: implement configurable, rule-driven limits instead of a single hard-coded number. At request time, the gateway looks up the applicable rule based on attributes such as:
* Endpoint path or route pattern
* HTTP method
* User role or subscription tier
* API key or client identifier
Each applicable rule maps to a bucket or policy (token bucket, leaky bucket, fixed window, sliding window) that enforces the limit. Keep the set of rules small and easy to evaluate so lookups are fast.
Example rule categories:
| Resource type | Typical limit (example) | When to use |
| -: | -: | - |
| Public reads | `~10,000 req/min` | High-throughput, low-impact endpoints |
| Create / write | `~100 req/min` | Expensive, state-changing operations |
| Login / auth | `~10-20 req/min` | Prevent credential-stuffing and brute force |
Question 2 — One Redis isn't enough at high scale
A single Redis instance often cannot handle extreme loads. For example, a single instance might handle on the order of \~100k ops/sec (varies by instance type and command mix). At 1M req/sec you’ll need to shard across multiple Redis instances — roughly 10 shards in a simplified estimate.
Rate-limiting is naturally partitionable by principal: Alan’s bucket is independent of John’s bucket. Use sharding keyed by principal (user ID, API key, client ID) so every request for the same principal routes to the same Redis shard. This preserves correctness without cross-shard coordination.
Operational consideration: an abusive client who has already exhausted their quota can still flood your gateways. If each incoming request triggers a Redis lookup, the shard will be hammered even though all requests are being denied. Sharding increases overall capacity but does not eliminate wasted Redis calls from aggressive clients.
Question 3 — Can we avoid a Redis network call on every request?
Yes. A common production optimization is to maintain a small local counter in each gateway process and periodically sync those counts to Redis. The gateway increments the in-memory counter on each request and flushes batches to Redis at a fixed interval (for example, every 100–500 ms).
Benefits:
* Fast local rejection: once the gateway determines a principal is blocked locally, it can reject further requests without hitting Redis.
* Reduced Redis load: batching reduces the number of Redis operations significantly.
Trade-offs:
* Slightly relaxed global enforcement during the sync window: multiple gateways may accept requests before their local counters are flushed to Redis. Choose the sync interval based on the acceptable error bound vs. performance.
* Need to manage edge cases: gateway restarts or crashes can lose the in-memory counters unless you use a short sync interval or persist counters on exit.
Comparison of enforcement approaches
| Approach | Typical latency | Redis ops | Strictness | Notes |
| -: | -: | -: | -: | - |
| Single Redis check per request | Higher | High | Strict | Accurate but expensive at scale |
| Sharded Redis (by principal) | Higher | Higher, but parallelized | Strict | Scales capacity, reduces per-instance load |
| Local counters + periodic sync | Low | Low (batched) | Slightly relaxed | Good latency; bounded inaccuracy |
| Hybrid (local fast path + Redis fallback) | Low | Low | High if fallback used | Combine local checks with occasional Redis validation |
Using local counters with periodic synchronization is a practical optimization: it reduces Redis load and improves latency while introducing a small, bounded inaccuracy in enforcement. Tune the sync interval to balance performance and strictness.
Optional mitigations for abusive traffic
* Early local deny lists: after a gateway locally observes sustained abusive behavior, add the principal to an in-memory deny list and apply a cooldown before re-evaluating in Redis.
* Backoff and rate-limited error responses: instruct clients to back off (e.g., `Retry-After`) to reduce repeated hits.
* Edge filtering: push basic throttling logic as close to the client as possible (CDN, WAF, or edge proxies) to filter obvious abuse before it reaches gateways.
Further reading and references
* Redis official documentation: [https://redis.io/](https://redis.io/)
* Rate limiting patterns: [https://en.wikipedia.org/wiki/Rate\_limiting](https://en.wikipedia.org/wiki/Rate_limiting)
* Consistent hashing for sharding: [https://en.wikipedia.org/wiki/Consistent\_hashing](https://en.wikipedia.org/wiki/Consistent_hashing)
Summary
* Use configurable rules per endpoint/principal rather than a single global limit.
* Shard Redis and route by principal to scale to high request rates while keeping correctness.
* Employ local gateway counters with periodic sync to avoid a Redis call on every request, trading a small accuracy window for large performance gains.
* Combine local optimizations with defensive measures (deny lists, backoff headers, edge filtering) to reduce wasted work from abusive clients.
This completes the discussion of common rate-limiter follow-ups. Take a moment to internalize these patterns: they are practical, widely used, and easy to explain during system design interviews.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.