Every production rate limiter has two layers. The first layer is the algorithm: token bucket, leaky bucket, fixed window, sliding window. The second layer — the one that decides whether your limiter works at all — is where the counters live. If your service runs on one machine, that’s a mutex and a map. The moment you run two or more replicas behind a load balancer, that map becomes a lie: each instance counts its own traffic, and your “100 requests per minute” limit quietly becomes 100 per minute per pod.
This post is about the second layer. We’ll build a distributed rate limiter in Go backed by Redis, using the GCRA algorithm that powers the redis_rate package. You’ll learn why the obvious counter-based approaches break under concurrency, how GCRA gets away with storing exactly one value per client, and the operational pitfalls — fail-open vs fail-closed, hot keys, key TTLs, IPv6 — that turn a correct algorithm into an incident.
Why the Obvious Approaches Break
Before GCRA, it’s worth being precise about why the common patterns fail in distributed settings, because each failure mode shapes the solution.
Fixed window counters
Keep a counter per (client, minute-of-epoch), increment it, expire it after a minute. It’s one INCR and one EXPIRE — cheap and simple. The problem is boundary burstiness. A client can send 100 requests at 10:59:59.9 and 100 more at 11:00:00.1, and both batches succeed. Your nominal limit is 100 per minute, but you just served 200 in two seconds. The scarcer the protected resource, the more a doubled burst hurts.
Sliding window log
Store a timestamp for every request and count the entries in the last window. This is perfectly accurate and perfectly unbounded: a client making 10,000 requests per minute makes you store and expire 10,000 entries per key. At any real scale, memory and round trips dominate. It’s a great reference model and a bad implementation.
Sliding window counter
The popular middle ground: keep the previous window’s count and the current count, and estimate the sliding rate as a weighted average based on how far into the current window you are. This bounds the burst to roughly twice the limit and costs two counters. The estimate is approximate, and the increment-then-check pattern is a read-modify-write race unless it’s atomic — which is exactly the constraint that pushed the industry toward token-bucket-style algorithms computed atomically in one place.
And that’s the deeper problem with all of them: check-then-set races. If your limiter reads a value, compares, then writes — across two requests arriving on different replicas — both can read the same value, both pass, and both write. Under load, races don’t happen occasionally; they happen constantly. The fix is to make the entire decision atomic, which in Redis-land means a Lua script or an algorithm designed so the mutation is a single primitive operation. GCRA is the latter.
GCRA: The Leaky Bucket With One Value of State
GCRA — the Generic Cell Rate Algorithm — comes from ATM network traffic shaping, which is a good sign: it was designed for hardware that couldn’t afford complex bookkeeping. The intuition is a leaky bucket with a twist. Each client’s requests drain from a bucket at a fixed rate (the emission interval — one request every 1/rate seconds). A request is allowed if the bucket isn’t already too full. The clever part: you don’t need to store the water level, or a list of timestamps, or counters. You store a single timestamp — the theoretical arrival time (TAT), which tracks how far “ahead of schedule” the client already is.
The decision procedure for a request arriving at time t:
- Read the stored
TAT(or initialize it totfor a first-seen client). - If
t < TAT - burst_offset, the client is too far ahead of schedule — the bucket would overflow. Reject. - Otherwise, allow, and atomically update
TAT = max(TAT, t) + emission_interval.
The burst_offset — burst size times the emission interval — is what makes this a bucket and not a metronome: it’s how much a client can be “ahead of schedule,” the burst allowance on top of the steady rate. Both the check and the update touch exactly one value, and because the read-and-write runs in one atomic Redis operation, there’s no race even across dozens of replicas.
The practical consequences are worth spelling out:
- O(1) memory per client — one key, one small value, a TTL. A client that hasn’t sent traffic in an hour costs nothing.
- One round trip — the entire decision is a single Redis operation, not a script looping over a sorted set.
- Rate and burst are independent knobs — “10 req/s sustained, bursts up to 50” is one configuration, unlike fixed windows where burst and rate are the same number.
- Retry-after is computable — the distance between the rejection boundary and TAT tells you exactly when the next request will pass, so your 429 can carry an honest
Retry-After.
GCRA in Go, Without Hand-Rolling It
The algorithm is small enough to implement in an afternoon, but the maintained implementation is better tested than anything most of us would write under deadline. The redis_rate package is a thin GCRA layer over go-redis, and its API is the shape you’d end up with anyway — allow, then get a decision plus remaining budget plus retry time:
package main
import (
"context"
"fmt"
"github.com/go-redis/redis_rate/v10"
"github.com/redis/go-redis/v9"
)
func main() {
ctx := context.Background()
rdb := redis.NewClient(&redis.Options{Addr: "localhost:6379"})
limiter := redis_rate.NewLimiter(rdb)
res, err := limiter.Allow(ctx, "project:123", redis_rate.PerSecond(10))
if err != nil {
panic(err)
}
fmt.Println("allowed", res.Allowed, "remaining", res.Remaining)
}Under the hood that call runs the decision we walked through: read the client’s TAT, compare against the burst boundary, atomically write the new TAT. The Result carries Allowed (how many requests pass now), Remaining (instantaneous burst budget left), and RetryAfter — which maps directly onto a Retry-After header when the answer is zero.
For custom tiers, construct a Limit explicitly. Sustained rate and burst are independent fields, which is exactly the GCRA sweet spot — “2,000 requests per hour per client, but allow short spikes of 50” is one struct, not a hack:
hourly := redis_rate.Limit{
Rate: 2000,
Period: time.Hour,
Burst: 50,
}
res, err := limiter.Allow(ctx, "apikey:"+key, hourly)Wiring It Into an HTTP Service
The limiter only earns its keep inside middleware. Three decisions matter here: what the key is, what headers you return, and what the failure mode is when Redis is unreachable.
package middleware
import (
"net/http"
"strconv"
"github.com/go-redis/redis_rate/v10"
"github.com/redis/go-redis/v9"
)
func RateLimit(rdb *redis.Client, limit redis_rate.Limit) func(http.Handler) http.Handler {
limiter := redis_rate.NewLimiter(rdb)
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
key := clientKey(r)
res, err := limiter.Allow(r.Context(), key, limit)
if err != nil {
// Redis is down: fail open or closed? See below.
next.ServeHTTP(w, r)
return
}
w.Header().Set("X-RateLimit-Limit", strconv.Itoa(limit.Rate))
w.Header().Set("X-RateLimit-Remaining", strconv.Itoa(res.Remaining))
if res.Allowed == 0 {
retry := int(res.RetryAfter.Seconds())
if retry < 1 {
retry = 1
}
w.Header().Set("Retry-After", strconv.Itoa(retry))
http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
return
}
next.ServeHTTP(w, r)
})
}
}
func clientKey(r *http.Request) string {
// Prefer an authenticated identity; fall back to IP only at the edge.
if k := r.Header.Get("X-Api-Key"); k != "" {
return "apikey:" + k
}
return "ip:" + r.RemoteAddr
}Note what this middleware does not do: it doesn’t parse bodies, doesn’t hit the database, doesn’t allocate per-request buffers. Rate limiting runs on the hot path of every request, so the limiter’s cost is a floor on your service’s overhead. One Redis round trip — sub-millisecond on a healthy network — is the entire budget, which is exactly why GCRA’s single-value design matters more than its elegance.
The Pitfalls That Bite in Production
Fail open or fail closed?
The snippet above fails open — if Redis errors, the request goes through. That’s the right default for most APIs: your limiter’s dependency shouldn’t take down your service. But it’s the wrong default wherever protection matters more than availability: login endpoints (brute-force protection should fail closed), payment endpoints, anything where the limit exists to stop abuse rather than to be polite. Make this decision explicitly per route, not globally, and alert on Redis errors from the limiter itself — a silent fail-open is a limiter that stopped existing.
Hot keys and the Redis single-thread tax
Every rate-limit decision is a Redis command, and Redis executes commands serially. A single extremely hot key — one API key doing 50,000 requests per second through your fleet — serializes every replica’s checks onto one Redis core. Redis is fast (well over 100k operations per second on commodity hardware), but it’s not infinite, and a hot key makes your limiter the service’s bottleneck. If you need per-client limits above a few thousand per second, either shard that client’s limit across several keys (each with a fraction of the rate) or move that tier of enforcement to a local token bucket with periodic sync.
Key TTL: the silent leak
GCRA stores one value per client, which sounds harmless — until you realize every unique API key, every rotating IP, every scrapable endpoint creates a key. Without TTLs, that’s an unbounded memory leak in Redis. With too-short TTLs, a client whose limit is “1,000 per hour” gets their state wiped mid-window. Rule of thumb: the TTL should cover at least the time a full bucket takes to drain, and it must be set on every write path, including rejections. A limiter that only sets TTLs on allowed requests slowly accumulates junk keys from rejected clients forever.
The IPv6 masking mistake
If you key on IP for unauthenticated traffic, remember that IPv6 gives every device an effectively unlimited supply of addresses. An attacker rotating through a /64 block can present 264 distinct “clients” to your limiter. Mask to a common prefix — typically /56 or /64 for IPv6, the full address for IPv4 — before keying, and accept the collateral: some legitimate users behind large carriers will share a bucket. This is one place where the ulule/limiter library’s options — client-IP header selection and IPv6 masking as configuration — show what production limiters end up needing that tutorials skip.
Which layer limits what
A single global limiter in front of everything is the wrong shape. Limits should be nested:
- Edge (per-IP, coarse) — stops scrapers and DDoS debris before they cost you anything. Fail open only if your edge already has other protection.
- Per-client (per API key, the business limit) — this is the limit on your pricing page. GCRA’s rate-plus-burst maps neatly onto “sustained throughput with short spikes.”
- Per-operation (expensive endpoints) — a report-generation endpoint might cost 10 units of the same client budget, or have its own tighter limit. Implement by scaling the rate, or by using a separate key namespace per operation class.
One last practical note: libraries exist and are good. redis_rate covers the GCRA-over-Redis case, and ulule/limiter offers a store abstraction with token-bucket backends and ready-made middleware for stdlib, Gin, and FastHTTP, if you need pluggable strategies or Redis Cluster awareness. Hand-rolling is worth it once, for understanding — then reach for the maintained implementation and spend your effort on the operational details above.
Wrapping Up
The algorithm is the easy part of rate limiting. What makes a limiter production-grade is deciding its failure mode per route, keeping its state bounded with TTLs on every path, keying on identity rather than spoofable headers, and knowing which layer owns which limit. GCRA earns its place as the default answer because it collapses all of that state to a single timestamp per client — one atomic write per decision, one round trip, honest Retry-After values for free. Build the toy version once to understand it; run the maintained one in production; and watch your limiter’s own metrics like any other dependency, because a rate limiter that fails silently open is a rate limiter you don’t have.