Home / Blog / System Design Rate Limiting Interview

System Design Rate Limiting Interview

Published 2026-10-02 · Luna Interview

System design rate limiting interviews separate “add a rate limiter” from a design that explains who is limited, where the counter lives, which algorithm, and what the client sees when denied. This guide gives you a structured talk track for API and gateway limits without inventing product-specific magic.

Start with the job of the limit

A rate limiter protects downstream capacity and fairness under bursty or abusive traffic. Say the goal before algorithm names.

Talk track:

“I’ll limit by authenticated user (and IP for anonymous) at the edge so one client cannot melt the core API. Denied requests get 429 with Retry-After.”

If the problem is only internal fan-out, you may need per-service budgets, not just public API quotas. Internal limits often protect databases and third-party APIs with stricter ceilings than the public gateway.

Clarify before you draw

1. Unit of identity: user, API key, IP, tenant, or combination?
2. Scope: per endpoint, global, or per resource?
3. Window: per second, minute, day? Soft vs hard quota?
4. Consistency: approximate OK, or must every replica agree exactly?
5. UX: drop, queue, or degrade?

Missing the key (identity) is the classic junior tell—algorithm talk without a key is decoration. Also ask whether costed operations exist (search costs more than a cheap GET).

Where to enforce (pick deliberately)

Default interview sketch: gateway + shared store for counters, with a note that edge can shed obvious floods first. Do not draw four limiters in the first minute—add layers only when the threat model needs them.

Algorithms worth naming

• Fixed window: simple; burst at window edges (two windows meet)
• Sliding window log: accurate; memory-heavy at high QPS
• Sliding window counter: approximate; good interview default for “mostly fair”
• Token bucket: allows controlled bursts; refill rate + capacity
• Leaky bucket: smooths to a constant outflow

Talk track:

“Token bucket with capacity for short bursts and a steady refill; counters in Redis with a per-user key so all gateway instances share the budget.”

Do not claim one algorithm is “always correct.” Tie choice to burst tolerance and memory. If the interviewer asks for code-level detail, sketch INCR + expire or a Lua/transactional compare for tokens—keep it short.

Counter storage

Mention hot keys: one celebrity user or shared IP NAT can dominate. Mitigations: sharded keys, local token caches with periodic sync, or coarser tenant buckets. For multi-tenant SaaS, a tenant ceiling plus a per-user ceiling prevents one seat from burning the whole account budget.

Response contract

Spell the client-facing behavior:

• HTTP 429 (or gRPC resource exhausted)
• Headers: remaining quota, reset time, Retry-After
• Idempotency: limiting should not corrupt safe retries for reads; for writes, clients need clear guidance

Talk track:

“Clients see 429 with Retry-After; critical admin paths can use a separate higher bucket so ops is not locked out during abuse.”

Document whether partial work happened before the reject (usually reject before side effects).

Fairness and multi-tenant notes

• Separate buckets for free vs paid tiers
• Per-tenant ceilings so one customer cannot starve others
• “Cost” weights: expensive search costs more tokens than a cheap GET
• Optional warm-up: new clients get a smaller burst until trusted

Fairness is a product sentence, not only an infra sentence—say who you are protecting.

Failure modes you must not skip

• Limiter store down: fail open (risk overload) vs fail closed (risk outage)—name the tradeoff for the product
• Clock skew across nodes for window math
• Distributed race: two instances both grant the last token—approximate is usually OK; say so
• Retry storms after a 429 wave—mention jittered backoff on the client side
• Config mistakes: a global limit of “100” meant per-user—call out testing

First five minutes

1. Name the identity key and window
2. Choose gateway vs app enforcement
3. Pick token bucket or sliding window and why
4. Place shared counter store
5. Define 429 + Retry-After and store-down behavior

Worked mini-example (narrate, do not overbuild)

“Photo upload API: 30 uploads/hour per user, token bucket capacity 5 for short bursts. Key rl:upload:{userId} in Redis. Gateway checks before accepting the multipart body. Anonymous users share a stricter IP bucket. If Redis is down, fail closed on uploads (expensive) but fail open on public read GETs with a local coarse limit.”

That paragraph shows judgment. Interviewers remember judgment more than algorithm trivia.

Freeze recovery

1. Erase exotic algorithms
2. Write: key / window / algorithm / store
3. Draw one happy path + one denied path
4. Add burst and multi-instance only after the basic path is clear

Soft practice note

Rate limiting interviews are narration-heavy: keys, algorithms, and failure modes. If you want a Chrome side panel to practice this system design rate limiting talk track under timed pressure, Luna Interview is built for calm rehearsal. Use it to build structured communication—not to fake production scars or to violate interview rules.

Day-of checklist

• [ ] Identity key spoken before Redis appears
• [ ] Algorithm named with burst rationale
• [ ] Shared counter placement clear
• [ ] 429 / Retry-After mentioned
• [ ] Store-down fail open vs closed chosen
• [ ] Hot-key or multi-tenant fairness noted if relevant

Justify the limit—then draw it once, clearly.

Rehearse with an AI interview copilot, then browse commonly reported interview questions.