The most expensive lie a system can tell is that it accepted work it cannot process in time. It usually starts innocently: a queue gets added between a producer and a consumer, the queue is unbounded, and suddenly overload stops failing fast. Requests that used to error in milliseconds now succeed at the door and time out thirty seconds later. Memory grows with the backlog. Eventually the consumer’s pod gets OOM-killed, the queue drains nothing, and the failure arrives all at once, at the worst possible moment. Backpressure is the alternative: a deliberate mechanism that makes consumers slow upstream, so overload shows up where it can be handled instead of hiding inside a buffer.
This post covers how backpressure actually works in Go — bounded channels as the core mechanism, the four producer policies for when the buffer is full, and load shedding at the HTTP edge — plus the trap that makes queues the most misunderstood tool in this space.
The Bounded Buffer Is the Whole Mechanism
Backpressure is not a framework feature; it falls out of one rule: never let in-flight work grow without limit. In Go the rule has a direct translation — give every handoff between stages a fixed capacity. A channel with a buffer fills up, and then the producer’s send blocks. That blocking is the backpressure: the producer physically cannot create a hundred thousand queued items, because the hundred-and-first send waits until a consumer takes one.
package main
import (
"fmt"
"sync"
)
type Job struct {
ID int
}
func main() {
jobs := make(chan Job, 64) // bounded: sends block once 64 are queued
var wg sync.WaitGroup
for w := 0; w < 4; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for job := range jobs {
process(job)
}
}()
}
for i := 0; i < 200; i++ {
jobs <- Job{ID: i} // blocks when the buffer is full
}
close(jobs)
wg.Wait()
}
func process(j Job) {
fmt.Println("processed", j.ID)
}The capacity is not decoration — it is a latency-versus-throughput tuning knob, and it should be sized deliberately. A larger buffer smooths bursts and keeps consumers fed, but it also inflates the worst-case wait for any item: a queue of 64 jobs at 10ms each is under a second of delay; a queue of 100,000 is seventeen minutes. Whatever number you pick, it becomes part of your latency budget, so treat it like a configuration decision with an owner, not a default nobody remembers choosing. The Go blog’s pipelines article covers the staging pattern in depth.
Blocking Is Only One of Four Producer Policies
Blocking the producer is the default, and it is often wrong. A slow consumer backing up into your HTTP handlers will eventually exhaust your connection pool or worker goroutines, and now the whole service is wedged — you have moved the failure, not prevented it. When the buffer is full, the producer needs an explicit, chosen response. There are four:
- Block — wait for space. Correct when the producer must not lose work and can afford to stall (an internal batch job, a replica applying events in order).
- Fail fast — return an error immediately. Correct when the caller can retry meaningfully and you want the rejection to happen at the cheapest possible layer.
- Drop newest — discard the incoming item. Correct for observability pipelines: losing one metrics point is far cheaper than stalling the emitter.
- Drop oldest — evict the head to make room. Correct when recency matters more than completeness — latest-value telemetry, a live price feed.
Whatever you pick, encode it in the API rather than leaving it implicit:
package main
import "errors"
var ErrOverloaded = errors.New("queue full, work rejected")
type Job struct {
Payload []byte
}
type Ingest struct {
jobs chan Job
}
func NewIngest(capacity int) *Ingest {
return &Ingest{jobs: make(chan Job, capacity)}
}
// Submit sheds load instead of waiting. The caller decides what a
// rejection means: retry later, defer to the next batch, or drop.
func (s *Ingest) Submit(job Job) error {
select {
case s.jobs <- job:
return nil
default:
return ErrOverloaded
}
}A zero-capacity channel changes the semantics again: every send synchronizes with a ready receiver, so there is no buffering at all — producer and consumer advance in lockstep. That is the strongest possible backpressure and occasionally exactly what a strict ordering requirement wants, but it couples the two stages’ throughput completely.
Propagate the Signal to the Edge
Backpressure that stops at the first internal queue has just relocated the backlog. The signal needs to reach whatever admits work into the system — the HTTP server — and turn into something a client can act on. At the edge this becomes load shedding: cap concurrency explicitly, and when the cap is hit, refuse quickly with a 429 and a Retry-After header instead of accepting the request into a queue that will fail it slowly later. A rejected request costs microseconds; an accepted one that times out costs a connection, memory, a client-side timeout, and usually a retry.
package main
import (
"log"
"net/http"
)
type server struct {
inFlight chan struct{} // capacity = max concurrent requests
}
func newServer(maxInFlight int) *server {
return &server{inFlight: make(chan struct{}, maxInFlight)}
}
func (s *server) shed(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
select {
case s.inFlight <- struct{}{}:
defer func() { <-s.inFlight }()
next.ServeHTTP(w, r)
default:
w.Header().Set("Retry-After", "1")
http.Error(w, "server overloaded, retry shortly",
http.StatusTooManyRequests)
}
})
}
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/jobs", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
})
log.Fatal(http.ListenAndServe(":8080", newServer(200).shed(mux)))
}Load shedding is deliberately cruder than rate limiting. Rate limiting answers “is this client allowed” — it is fair, per-identity, and steady-state policy. Shedding answers “is the server healthy enough to accept anyone” — it is a total-capacity check that fires only under distress and ideally treats all callers identically. You want both: the limiter first, the shed as the last line of defense. Two details make shedding survivable for clients: keep rejections fast (never add work to refuse work), and honor Retry-After semantics so well-behaved callers back off instead of turning one overload into a retry storm that guarantees the overload persists.
“Just Add a Queue” Moves the Problem, It Does Not Remove It
External queues deserve their own warning label. Kafka happily accepts messages far faster than any consumer group can process them — from the broker’s point of view there is no overload at all. The queue decouples availability, but it does not create capacity: if producers outpace consumers, the backlog grows without limit, and your overload is now invisible behind a green health check. The signal still exists — consumer lag and the age of the oldest unprocessed message — but somebody has to be watching it, because nothing fails until the lag crosses the retention window and messages start disappearing unread.
This is why consumer-side flow control matters as much as producer-side. Consumers that fall behind need a way to stop digging: pause fetching on a partition until the backlog shrinks, spill oversized work to slower storage, or push the pressure upstream through the same shed-or-block decision the HTTP edge uses. And the retry paths deserve design scrutiny equal to the happy path — unbounded retries with no backoff amplify the original overload, which is how a 30-second slow patch becomes an hour-long incident.
One metric makes all of this diagnosable, and it is not CPU or memory: the age of the oldest item in every queue. Queue depth tells you how much work is waiting; age tells you whether the system is winning or losing. Depth can be stable while age climbs steadily, and age is the number that maps directly to user-visible staleness. Alert on it, dashboard it, and cap it — an explicit maximum age is a backpressure policy expressed in units everyone understands.
Wrapping Up
Backpressure comes down to three decisions made on purpose rather than by default: every buffer in the path has a fixed capacity, every producer has a chosen response when that capacity is hit, and every internal slowdown can reach the edge and become a fast 429 instead of a slow timeout. Bounded channels give you the mechanism for free in Go — the discipline is refusing the unbounded queue when someone reaches for it, and watching queue age as the honest health metric. Systems that shed load degrade noisily and recover cleanly; systems that buffer everything degrade silently and fail all at once. Pick which kind you are running before the traffic spike picks for you.