Rate Limiter — system design by AgileViper46

Latency target is stated but not defended end-to-end

You mention a sub-10ms decision target, but what happens when a request must evaluate multiple buckets and one of them still requires a Redis round trip under load? Without an end-to-end latency budget per step and a p99 target, it is hard to tell whether the design still meets the NFR during peak traffic, failover, or hot-key scenarios. You could improve this by explicitly budgeting gateway processing, local checks, Redis access, and config lookup behavior at p95/p99.

Availability target is not mapped to failure scenarios

You state 99.99% availability and mention circuit breaking and emergency quota, but what happens when a regional Redis cluster is unavailable, ETCD watches lag, or a gateway loses both Redis and fresh config at the same time? The design says fail-open/fail-closed is dynamic, but it does not define which limits choose which mode or how that preserves the availability target without violating enforcement too badly. You could improve this by specifying failure policies per rule type and the expected behavior during Redis, config, and cross-region outages.

Consistency model is only partially specified

The design explains eventual consistency for rule propagation and approximate consistency for global quotas, but what happens if a critical rule change must take effect immediately across all gateways? 'Force-refresh for critical changes' is directionally good, yet the consistency contract is still unclear: are per-user/API/tenant limits strongly enforced within a region, eventually enforced across regions, and globally approximate? Making that explicit would clarify what correctness guarantees clients and operators can rely on.

Request identity dimensions are not modeled as entities or relationships

Have you considered how a request maps to user, API, and tenant at the domain level? The requirements are specifically per-user, per-API, and per-tenant, but the entity list only has Clients and Rules. Without explicitly modeling those dimensions, it is unclear how rules attach to the right subject and how one request resolves all applicable limits.

Relationship between clients and rules is underspecified

What happens when one client is subject to multiple limits at once, such as a tenant rule plus an API rule plus a user rule? The design says all applicable rules are evaluated, but the core entities section does not define whether this is one-to-many, many-to-many, or how rule applicability is determined. Making that relationship explicit would remove ambiguity in the happy path.

Per-request operation count is not carried through to Redis sizing

Have you considered what happens when each request matches multiple rules? At 300K regional RPS, a request may need user + API + tenant checks, which can turn into roughly 900K bucket evaluations/sec before retries or config lookups. The Redis estimate of 4-5 nodes per region appears to assume one operation per request, so the fleet could be undersized once multi-rule evaluation is included. You could improve this by translating request mix into average bucket checks per request and sizing Redis from that derived ops/sec.

No storage or memory sizing for 100K rules and active buckets

What happens when the working set of active user, tenant, and API buckets grows during peak traffic? The design names ETCD and Redis, but there is no estimate for rule storage, in-memory rule snapshots on gateways, active token-bucket cardinality, TTL behavior, or replication overhead. Without that, it is hard to tell whether the chosen clusters fit the stated 100K rules and 10M unique clients. You could improve this by estimating active bucket count per region, bytes per bucket, retention/TTL, and replicated memory footprint.

Cross-datacenter capacity trade-off is not quantified

Have you considered what happens to global accuracy and quota utilization when multiple regions hold leases at the same time? Lease-based allocation is directionally correct, but the design does not quantify lease size, refill frequency, or worst-case overshoot/underutilization during failures or partitions. At 1M RPS, those parameters materially affect both correctness and the load on the global Redis. You could improve this by showing how lease size is chosen from target coordination QPS and acceptable burst error.

Request contract is underspecified for multi-dimensional limits

Have you considered what the client or gateway must send into evaluate(request) so the system can actually enforce per-user, per-API, and per-tenant rules? Without a clear request shape carrying identifiers like user ID, tenant ID, API key/path, and possibly method, different callers may compute different keys and the same request could be evaluated inconsistently. Define the required attributes and how missing identities are handled.

Multiple-rule failures are ambiguous to clients

What happens when a request violates more than one applicable rule, such as both tenant and API limits? The current response only says OK or 429 with generic headers, but it is unclear which limit the headers refer to. Clients may see misleading remaining/reset values if several buckets were checked. You should define whether headers represent the most restrictive rule, all matched rules, or a specific canonical rule, and keep that behavior consistent.

Error behavior beyond 429 is not defined

What does the client see when Redis is unavailable, rule config is missing, or the gateway cannot evaluate the request? The explanation discusses circuit breaking and fallback behavior internally, but the API contract does not say whether the caller gets fail-open allow, fail-closed deny, 503, or some structured error body. At senior level, the client-visible behavior during dependency failures should be explicit so integrators know whether to retry or surface an outage.

Retry guidance is incomplete for throttled clients

Have you considered how clients should retry after a 429? Retry-After is listed, which is good, but the contract does not say whether it is seconds vs HTTP-date, how it is computed when several rules apply, or whether clients can safely retry exactly at reset time. A clearer retry contract avoids synchronized retries and makes the API easier to consume correctly.

Cross-datacenter correctness depends on approximate reconciliation

Have you considered what happens if multiple regions lose connectivity to global Redis at the same time and continue serving from emergency quota? The system stays available, but global limits can be materially exceeded until reconciliation. That may be an acceptable trade-off, but the HLD should make that approximation explicit and bound the overshoot.

Redis-global is a likely bottleneck and failure concentration point

What happens when lease refresh traffic spikes across all gateways or Redis-global becomes slow? Even though it is off the per-request path, all regions still depend on it for replenishment of global and hot-tenant leases. Without clear sharding/replication/failover strategy for Redis-global, this becomes the first shared choke point for cross-DC enforcement.

ETCD topology across regions is unclear

Have you considered what happens if each region has its own ETCD cluster and rule updates race or diverge? The diagram shows an ETCD cluster in each region, but the explanation treats rules as a single source of truth. If these are independent clusters, config consistency becomes fragile; if they are one logical cluster stretched across regions, write latency and quorum behavior need to be considered.