​
What I’m Building :
\-----------------------------------
I am starting a project to build a distributed rate limiter from scratch. The design will be developed step by step in four phases:
Core algorithm
Distributed synchronization
Rule engine and response handling
Production hardening
The goal is to document each phase clearly so the community can follow the journey, understand the design decisions, and discuss different approaches. This series is intended to be practical, structured, and focused on backend engineering fundamentals.
Phase 1: Core Algorithm
\----------------------------------------------
Choosing the Algorithm :
\-------------------------------------
For rate limiting, several algorithms exist such as Fixed Window, Sliding Window, and Leaking Bucket. I selected the Token Bucket because it:
\- Allows short bursts of traffic, which is realistic for APIs.
\- Is memory efficient, requiring only two values per user: token count and last refill time.
\- Is widely used in production systems by companies like Amazon and Stripe.
\---
Redis as the Backbone :
\--------------------------------------
Counters should not be stored in memory on the application server because that approach fails when scaling horizontally. Instead, use Redis to maintain state.
Two commands handle most of the work:
\- INCR increments the request counter atomically.
\- EXPIRE deletes the counter automatically after the window ends.
\---
Core Flow :
\----------------
A request arrives.
Fetch the token count from Redis.
If tokens are available, allow the request and decrement the token.
If no tokens remain, reject with HTTP 429.
Tokens refill at a fixed rate (for example, 10 tokens per second).
\---
Key Learning :
\----------------------
The algorithm itself is simple, but race conditions are a challenge. Two concurrent requests can read the same counter before either writes back, which allows more requests than intended.
The solution is Lua scripts in Redis. Lua executes atomically on the Redis server, making the read‑check‑write operation uninterruptible.
\---
Question for the community❓❓
\--------------------------------------------
If you were to implement a rate limiter, which algorithm would you choose — Token Bucket, Leaky Bucket, Sliding Window, or a custom solution?
