Rate limiting looks like a solved problem until the day your API runs on more than one machine. The in-memory token bucket that worked beautifully in staging silently fragments in production: five pods each enforce “100 requests per minute” against their own local counter, and a determined client gets 500. The limit you meant to express is global, but the enforcement is per-process. Closing that gap is what distributed rate limiting is about, and it is one of those features where the naive implementation works fine in a demo and falls apart under exactly the traffic it exists to control.
This post builds a distributed rate limiter in Go on top of Redis and Lua, covering the parts that tutorials usually skip: why the algorithm choice matters less than atomicity, how to write the Lua script so concurrent requests cannot corrupt it, what the middleware should return so well-behaved clients can back off, and — the decision that bites hardest in incidents — whether to fail open or fail closed when Redis itself goes down.
Why the naive approaches fail
Start with what not to do, because each failure mode motivates a real design decision.
Local counters. golang.org/x/time/rate is an excellent in-process token bucket, but its state lives in one process’s memory. With N replicas behind a load balancer, effective throughput is N times your limit, minus whatever rounding the balancer introduces. It remains useful for protecting a single service instance from a single abusive caller, but it cannot express “this API key gets 1,000 requests per minute, total.”
Fixed windows in shared storage. Storing a counter keyed by minute in Redis fixes the shared-state problem but introduces the boundary burst: a client that sends its entire quota at 10:59:59 and again at 11:00:01 has just pushed 2x the limit through in two seconds. Whether that matters depends on what you’re protecting — it matters a lot when the limit exists to shield a downstream dependency.
Check-then-set without atomicity. The most subtle failure. Suppose you read a counter, check it against the limit, and increment if allowed — as three separate Redis commands. Two concurrent requests both read the same count, both see headroom, both increment, and the limit is exceeded. Under load, this is not an edge case; it is the steady state. The fix is not locking (slow, deadlock-prone) but making the entire decision a single atomic operation, which is precisely what Redis Lua scripts are for.
Choosing the algorithm
Three candidates cover nearly every real requirement:
- Sliding window log — store a timestamp per request in a sorted set, trim entries older than the window, and compare count against the limit. Exact, but memory grows with request volume.
- Sliding window counter — interpolate between the previous and current fixed windows, weighted by how far into the current window you are. Near-exact, two integers of state, and the default recommendation for most APIs.
- Token bucket (GCRA-style) — the classic burst-tolerant limiter: tokens refill at a steady rate, each request consumes one, and a burst up to the bucket capacity is allowed. One hash of state per key.
Token bucket is the right default for public APIs because real clients are bursty: a mobile app batching uploads on launch, a CI system fanning out requests, a UI firing a screenful of parallel calls. Enforcing a strict average with zero burst tolerance generates support tickets. The rest of this post implements a token bucket; adapting it to a sliding window counter is a small change to the script.
The Lua script: one atomic decision
The refill-check-consume cycle must run as a single indivisible operation. Redis executes a Lua script atomically — no other commands interleave during its execution — which gives us the semantics of a critical section without locks. Here is the complete script:
-- KEYS[1]: bucket key
-- ARGV: rate (tokens/sec), capacity, now (unix nanoseconds), requested (tokens)
local key = KEYS[1]
local rate = tonumber(ARGV[1])
local capacity = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])
local state = redis.call('HMGET', key, 'tokens', 'ts')
local tokens = tonumber(state[1])
local ts = tonumber(state[2])
if tokens == nil then
tokens = capacity
ts = now
end
-- refill based on elapsed time, capped at capacity
local elapsed = math.max(0, now - ts)
tokens = math.min(capacity, tokens + (elapsed / 1e9) * rate)
local allowed = 0
local retry_after_ms = 0
if tokens >= requested then
tokens = tokens - requested
allowed = 1
else
retry_after_ms = math.ceil((requested - tokens) / rate * 1000)
end
redis.call('HSET', key, 'tokens', tokens, 'ts', now)
-- expire once the bucket could fully refill plus a buffer
local ttl = math.ceil(capacity / rate) + 60
redis.call('EXPIRE', key, ttl)
return {allowed, tostring(tokens), tostring(retry_after_ms)}Three details worth calling out. First, the caller passes now rather than the script calling Redis’s clock — this keeps the limiter deterministic and testable, and avoids surprises if your Redis deployment has clock quirks across replicas. Second, the TTL is derived from the bucket’s full-refill time: an idle client’s state self-deletes instead of accumulating forever, which matters when the key space is unbounded (per-IP limiting exposes you to memory-exhaustion via IP spoofing or NAT pools). Third, the script returns the remaining token count and a retry-after hint, so the API response can tell well-behaved clients exactly when to come back.
The Go middleware
On the Go side, go-redis handles script loading and EVALSHA caching automatically. A complete, self-contained middleware:
package ratelimit
import (
"context"
"net/http"
"strconv"
"time"
"github.com/redis/go-redis/v9"
)
var allowScript = redis.NewScript(`
local key = KEYS[1]
local rate = tonumber(ARGV[1])
local capacity = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])
local state = redis.call('HMGET', key, 'tokens', 'ts')
local tokens = tonumber(state[1])
local ts = tonumber(state[2])
if tokens == nil then
tokens = capacity
ts = now
end
local elapsed = math.max(0, now - ts)
tokens = math.min(capacity, tokens + (elapsed / 1e9) * rate)
local allowed = 0
local retry_after_ms = 0
if tokens >= requested then
tokens = tokens - requested
allowed = 1
else
retry_after_ms = math.ceil((requested - tokens) / rate * 1000)
end
redis.call('HSET', key, 'tokens', tokens, 'ts', now)
local ttl = math.ceil(capacity / rate) + 60
redis.call('EXPIRE', key, ttl)
return {allowed, tostring(tokens), tostring(retry_after_ms)}
`)
type Limiter struct {
rdb *redis.Client
rate float64 // tokens per second
capacity float64 // burst size
failOpen bool
}
func New(rdb *redis.Client, rate, capacity float64, failOpen bool) *Limiter {
return &Limiter{rdb: rdb, rate: rate, capacity: capacity, failOpen: failOpen}
}
func keyFor(r *http.Request) string {
// in practice: API key or authenticated user id first, IP as fallback
if k := r.Header.Get("X-API-Key"); k != "" {
return "rl:" + k
}
return "rl:ip:" + r.RemoteAddr
}
func (l *Limiter) Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 150*time.Millisecond)
defer cancel()
res, err := allowScript.Run(ctx, l.rdb, []string{keyFor(r)},
l.rate, l.capacity, time.Now().UnixNano(), 1).Result()
if err != nil {
// Redis unavailable: decide the failure policy explicitly
if l.failOpen {
next.ServeHTTP(w, r)
return
}
http.Error(w, "rate limiter unavailable", http.StatusServiceUnavailable)
return
}
vals, ok := res.([]interface{})
if !ok || len(vals) != 3 {
http.Error(w, "rate limiter error", http.StatusInternalServerError)
return
}
allowed := vals[0].(int64)
remaining, _ := strconv.ParseFloat(vals[1].(string), 64)
retryMs, _ := strconv.ParseInt(vals[2].(string), 10, 64)
w.Header().Set("X-RateLimit-Limit", strconv.FormatFloat(l.capacity, 'f', -1, 64))
w.Header().Set("X-RateLimit-Remaining", strconv.FormatFloat(remaining, 'f', 0, 64))
if allowed != 1 {
secs := (retryMs + 999) / 1000
w.Header().Set("Retry-After", strconv.FormatInt(secs, 10))
http.Error(w, "rate limit exceeded", http.StatusTooManyRequests)
return
}
next.ServeHTTP(w, r)
})
}The standard headers matter more than they look. Retry-After on a 429 lets SDKs and retry loops back off automatically instead of hammering you; X-RateLimit-Remaining lets well-behaved clients pace themselves before hitting the wall. If you expose a public API, returning a naked 429 with no metadata is effectively forcing every client to implement their own backoff heuristics against your infrastructure.
Fail open or fail closed?
The middleware above makes the Redis-outage decision explicit, and it deserves its own section because teams get it backwards. If the rate limiter’s datastore dies, do you let traffic through (fail open) or reject everything (fail closed)?
The honest framing: a rate limiter is a protective control, not a correctness control. When it fails, the risk of failing open is downstream overload — the exact thing the limiter prevents. The risk of failing closed is a self-inflicted total outage of your own API, triggered by a dependency whose only job was protection. For most public-facing services, failing open with a short request timeout (the 150ms context deadline above bounds how long each request waits on Redis) is the right default: a brief window of unlimited traffic is usually survivable, while rejecting 100% of traffic because a protective component hiccuped usually is not. Fail closed makes sense for genuinely finite or expensive resources — payment initiation, third-party APIs you pay per call, or anything where exceeding the limit causes contract violations.
Whichever you choose, make it a deliberate flag (as above), wire it into alerts, and remember that the timeout itself is part of the design: without it, a hanging Redis turns every request into a five-second stall and your limiter becomes the outage.
Production hardening
- Hot keys. A single API key doing 50k req/s turns one Redis key into a serial bottleneck, since every decision on that key queues behind the same atomic script. Mitigations: local pre-filtering with x/time/rate as a cheap first gate, or sharding the bucket across N keys and consuming 1/N of a token from a randomly chosen shard.
- Clock values as arguments. Passing
time.Now().UnixNano()from the app keeps scripts testable and sidesteps differences between app and Redis clocks. Do not callTIMEinside the script unless you specifically want server-authoritative time. - Unbounded key spaces. Per-IP limiting on a public endpoint invites memory-exhaustion attacks: millions of spoofed sources each create a small hash. The TTL caps lifetime but not simultaneity. Consider a global key-count budget or limiting only authenticated identities.
- Multiple limits. Production APIs typically enforce two or three tiers at once (per-second burst, per-minute sustained, per-day quota) — run the same script with different rate/capacity pairs and separate key prefixes, short-circuiting on the first rejection.
- Testing determinism. Because
nowis injected, unit tests can drive the bucket through time: exhaust it, advance the clock, assert refill math — no sleeps, no flakiness.
Wrapping up
Distributed rate limiting is less about picking a clever algorithm and more about getting four decisions right: make the decision atomic (Lua), choose burst semantics that match real clients (token bucket), give clients the metadata to behave (Retry-After and remaining-count headers), and pick a deliberate, timeout-bounded failure policy for when the shared state is unreachable. The implementation above is small enough to read in one sitting and holds up under production traffic patterns. From here, the natural extensions are tiered limits, per-route configurations, and exporting allow/reject metrics to your monitoring stack so you can see the limiter working before your users tell you it is.