CodePark/ questions / Rate limiter design
START PRACTICINGPRACTICE
Backend question · System design & architecture › Rate limiter design

An API should allow an average of 10 requests per second per key, but let clients make short bursts of up to 50 requests. Which algorithm fits most naturally?

Token bucket with capacity 50 and a refill rate of 10 per second

A token bucket can store up to 50 unused tokens. So a client that was idle can send a burst of 50 requests. The refill rate keeps the long-run average at 10 per second. "Leaky bucket draining at 10 per second" is tempting, but it smooths output to a constant 10 per second. A burst is queued or dropped, not served right away.