You open your tracing backend to debug a slow checkout request and find five disconnected traces instead of one. Five services were involved, each one traced correctly on its own — but somewhere between the API gateway and the payment service, the trace context vanished. This is the most common failure mode in distributed tracing, and it is almost never a problem with your tracing backend. It is a problem with context propagation: the mechanism that carries trace identity across service boundaries.
This post explains how context propagation actually works under the W3C Trace Context specification, what the traceparent header contains byte by byte, and the specific places where propagation breaks in real systems — HTTP clients, background workers, message queues, proxies, and mixed vendor environments.
The traceparent header, byte by byte
The W3C Trace Context specification defines a single required header, traceparent, with a fixed format:
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01The four dash-separated fields are:
- Version (
00) — 1 byte, currently zero. - Trace ID — 16 bytes, shared by every span in the same trace. This is what you search by in the tracing UI.
- Parent Span ID — 8 bytes, identifying the span that produced this request.
- Trace Flags — 1 byte; the low bit indicates whether the trace was sampled.
Every W3C-compliant system reads this format identically, which is what makes cross-vendor tracing possible. The spec also defines a companion tracestate header for vendor-specific extensions, but traceparent is the interoperable core.
How OpenTelemetry moves context
OpenTelemetry implements propagation through two operations: extraction (reading context from an incoming request) and injection (writing context onto an outgoing request). Both are handled by a propagator, and OpenTelemetry’s default is a composite of the W3C Trace Context and W3C Baggage propagators.
When instrumentation is set up correctly, the happy path requires no manual work. In Go, a server wrapper extracts incoming context and starts a server span; a client transport injects context into outgoing requests:
package main
import (
"net/http"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/checkout", handleCheckout)
// Extracts traceparent from incoming requests, starts server spans.
wrapped := otelhttp.NewHandler(mux, "checkout-server")
client := &http.Client{
// Injects traceparent onto outgoing requests.
Transport: otelhttp.NewTransport(http.DefaultTransport),
}
_ = client
http.ListenAndServe(":8080", wrapped)
}The server wrapper and the client transport are two halves of the same contract. The failure cases below are all variations of one half being missing.
Where propagation breaks in practice
An uninstrumented HTTP client
The server extracts context fine, but the downstream call uses a plain http.Client with the default transport. No injection happens, so the next service sees no traceparent header and starts a brand-new trace. Your trace waterfall splits at exactly the hop you needed to debug. Every outbound client in the process needs the instrumented transport — including clients buried in SDKs and generated API clients, which usually accept a custom http.Client if you dig into their options.
Background workers and goroutines
Context is thread-bound. When a request handler kicks off a background job — a goroutine, a queue consumer, a deferred task — the active span context does not follow automatically. In Go, the fix is to pass the context.Context explicitly and start a new span from it inside the worker, which links the work back to the original trace:
func handleCheckout(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
tracer := otel.Tracer("checkout")
go func(ctx context.Context) {
// New span, but a child of the request span via ctx.
ctx, span := tracer.Start(ctx, "send-receipt")
defer span.End()
sendReceipt(ctx)
}(ctx)
}The bug to watch for: starting a span with context.Background() instead of the request’s context. That produces a valid, healthy-looking root span in the worker — and silently orphans it from the trace.
Message queues
A queue decouples the producer from the consumer in time as well as space, so context must travel inside the message. The pattern: the producer creates a span, injects context into a plain map, and attaches that map as message headers; the consumer extracts from the headers before processing. Instrumentation libraries exist for the common clients (Kafka, RabbitMQ, AMQP), but custom producers and consumers need the injection and extraction wired manually. The one-line summary: if the message does not carry the headers, the consumer cannot continue the trace.
Proxies and gateways that strip headers
Some load balancers, API gateways, and WAFs drop headers they do not recognize — and traceparent is not on every default allowlist. The symptom is characteristic: traces are intact on the client side of the proxy and fragmented on the service side. If hop one to two is connected but hop two to three is not, check what sits between two and three. The fix is usually an allowlist entry for traceparent and tracestate.
Mixed propagation formats
Legacy services often emit B3 (Zipkin) headers instead of W3C format. A service configured with only the W3C propagator will ignore incoming B3 headers and start a new trace. OpenTelemetry supports composite propagators that accept multiple formats:
import (
"go.opentelemetry.io/contrib/propagators/b3"
"go.opentelemetry.io/otel/propagation"
)
otel.SetTextMapPropagator(
propagation.NewCompositeTextMapPropagator(
propagation.TraceContext{},
b3.New(),
),
)Both sides of a boundary need to agree — injection format on the sender, extraction format on the receiver. During migrations, configure the union of formats everywhere and shrink the list as legacy services are converted.
Manual spans that skip extraction
When hand-writing instrumentation, it is easy to start a new span without first extracting the incoming traceparent. The result is a root span where a child belongs — a trace that restarts at your service. Always extract before you start; the span you begin will then become a child of the incoming context automatically.
Testing propagation deliberately
Propagation failures are invisible in unit tests because they only manifest across process boundaries. Two practices catch them early. First, log the trace ID at every service entry — when a request misbehaves, comparing logged trace IDs across services tells you instantly whether propagation broke and at which hop. Second, add an integration test that fires a sampled request through the full service graph and asserts the backend records exactly one trace, not N. It runs on every CI build and fails the moment someone adds an uninstrumented client.
Baggage deserves a brief mention: OpenTelemetry supports propagating arbitrary key-value pairs alongside trace context, useful for carrying tenant IDs or feature flags through the call tree. Keep baggage small, treat its contents as untrusted input at extraction (clients can set arbitrary values), and never use it as an authorization channel.
Wrapping up
Most “tracing is broken” incidents are propagation incidents, and they resolve down to a short list of causes: an uninstrumented HTTP client, context that never reached a background job, a message that did not carry its headers, a proxy stripping traceparent, mismatched propagation formats, or manual spans started without extraction. Each has a mechanical fix. Instrument every client, pass context explicitly to async work, put headers inside messages, verify what your proxies forward, and align formats on both sides of every boundary — and the single connected trace you expected will actually be there when you need it.