Getting a request into your system is the easy part. The hard question arrives at hop two: when the order service calls the payment service, who exactly is asking? If your answer is “whoever is inside the network,” you have a perimeter, not an architecture — and perimeters fail the moment one container is compromised, because the attacker inherits free transit to everything behind the same wall. Service-to-service authentication is the discipline of making every hop prove its identity, and it’s the foundation that zero-trust architectures are built on.
There are three mainstream approaches — network certificates, bearer tokens, and token exchange — and teams routinely pick the wrong one for their failure model. This post walks through each: how they work, what they actually protect against, and how they combine in production systems, with working Go code for the patterns that need it.
The Threat Model Comes First
Pick the mechanism after picking the adversary. Three questions determine everything:
- Can an attacker join the network? On a corporate VPN or a flat VPC, yes more easily than anyone wants to admit. On a Kubernetes cluster with strict NetworkPolicies, less so.
- Do you need to know the end user on internal hops, or just the calling service? Fraud checks and row-level permissions usually need the user; rate limits and audit trails usually need the service.
- Who operates the infrastructure? You control the cluster, or a platform team (or AWS) does — this decides whether you can terminate mTLS yourself.
The classic failure is answering all three questions with a single static API key shared across services. It authenticates nothing (anyone who exfiltrates it once owns it forever), it rotates badly, and it can’t distinguish service identity from user identity. Everything below is a refinement of replacing that key.
mTLS: Identity Baked Into the Connection
Mutual TLS upgrades the familiar server certificate dance into a two-way handshake: the client also presents a certificate, and both sides verify identity before a single application byte moves. Identity is bound to the connection itself — there is no credential in a header to leak, log, or forget to send.
The hard part was never the handshake; it’s certificate lifecycle. Every service needs a certificate with a short lifetime, issued by a CA your infrastructure trusts, and rotated automatically before expiry. Doing this by hand is how you get a 3 a.m. outage when a cert quietly hits its 90-day limit. The ecosystem answer is a workload identity mesh: Istio and Linkerd sidecars (or Istio’s newer ambient mode, which moves the mesh to node-level proxies) run the mesh, intercept connections, and rotate certificates behind your back — typically daily, from an internal CA.
What mTLS buys you: a compromised pod cannot impersonate another service, because it doesn’t hold that service’s key. What it doesn’t buy: end-user identity. The certificate says this connection comes from the orders service — it says nothing about which customer initiated the request. That gap is why mTLS alone is rarely the whole answer.
If you run plain Go services without a mesh, you can terminate mTLS yourself:
func mtlsServer(certFile, keyFile, caFile string) (*http.Server, error) {
caPEM, err := os.ReadFile(caFile)
if err != nil {
return nil, err
}
pool := x509.NewCertPool()
if !pool.AppendCertsFromPEM(caPEM) {
return nil, errors.New("CA file contains no certificates")
}
tlsCfg := &tls.Config{
ClientAuth: tls.RequireAndVerifyClientCert,
ClientCAs: pool,
MinVersion: tls.VersionTLS13,
}
return &http.Server{
Addr: ":8443",
Handler: mux,
TLSConfig: tlsCfg,
}, nil
// srv.ListenAndServeTLS(certFile, keyFile)
}
Once the server requires and verifies client certs, you can extract the verified caller identity from the connection state in your handler — r.TLS.PeerCertificates[0].Subject.CommonName is the service’s name, and at that point it’s cryptographically proven, not claimed. Your authorization layer can then make decisions on that identity directly.
Bearer JWTs: Simple, Portable, and Easy to Get Wrong
The token approach moves identity into the application layer: the caller presents a signed JWT, and the receiver validates the signature and claims. In the pure service-to-service flavor, the client is a client-credentials grant — the service authenticates to an authorization server with its own credentials and receives a token naming itself as the subject. Validation is stateless: the receiver needs only the issuer’s public keys, which it fetches once and caches.
The appeal is real: no mesh to install, works across network boundaries and cloud vendors, and carries rich claims. A Go validation handler is compact:
func validateJWT(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
authz := r.Header.Get("Authorization")
token, ok := strings.CutPrefix(authz, "Bearer ")
if !ok {
http.Error(w, "missing bearer token", http.StatusUnauthorized)
return
}
claims := jwt.RegisteredClaims{}
parsed, err := jwt.ParseWithClaims(token, &claims, func(t *jwt.Token) (any, error) {
// Pin the algorithm: never trust the token header alone.
if _, ok := t.Method.(*jwt.SigningMethodRSA); !ok {
return nil, fmt.Errorf("unexpected signing method: %v", t.Header["alg"])
}
return publicKey, nil // cached JWKS key from the issuer
}, jwt.WithIssuer("https://auth.internal.example.com"),
jwt.WithExpirationRequired(),
jwt.WithAudience("payments.internal"))
if err != nil || !parsed.Valid {
http.Error(w, "invalid token", http.StatusUnauthorized)
return
}
ctx := context.WithValue(r.Context(), callerKey{}, claims.Subject)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
Now the failure modes, which are all variations of “the token outlives its purpose”:
- The “alg: none” trap. A validator that trusts the token’s header to pick the algorithm will happily accept unsigned tokens. Pin the algorithm in code, as above — this single line has prevented more JWT vulnerabilities than any other.
- Overlong lifetimes. A service token valid for 24 hours is a 24-hour window for a replayed token. Keep access tokens in the 5–15 minute range; short lifetimes are what make stateless validation safe.
- Audience neglect. A token issued for the inventory service must not authenticate to payments. Validate
audon every request — without it, any internal token works on any internal service, and one leak compromises everything. - Signature check without claim check. A validly signed token that’s expired, revoked, or issued by the wrong tenant is still wrong. Signature proves integrity, not validity.
The structural limitation mirrors mTLS’s, inverted: a client-credentials token identifies the service well, but if you stuff end-user identity into the same token as custom claims, you couple two different trust domains — user-session tokens get the lifetimes and revocation semantics of service tokens, or vice versa. Which brings us to the pattern that resolves this.
Token Exchange: Two Identities, Two Tokens
The OAuth 2.0 Token Exchange standard (RFC 8693) solves the user-versus-service coupling directly: a service presents the incoming user token plus its own service credentials, and receives a new downstream token that encodes both identities — the user as the subject, the service as the actor. Each hop mints a fresh, audience-scoped token for the specific service it’s about to call:
curl -X POST https://auth.internal.example.com/oauth/token \
-d "grant_type=token-exchange" \
-d "subject_token=USER_JWT" \
-d "subject_token_type=urn:ietf:params:oauth:token-type:jwt" \
-d "requested_subject=orders-service" \
-d "audience=payments.internal"The response is a short-lived JWT whose act (actor) claim records the chain: the payment service receiving it can see that orders-service is acting on behalf of user user-9182. Three properties make this pattern the backbone of serious multi-service systems:
- Least privilege per hop. The downstream token is scoped to one audience with only the scopes that hop needs. Compromising the orders service doesn’t expose tokens that work against unrelated services.
- Auditable chains. The
actchain gives you a cryptographic answer to “which service acted for which user” — the question forensics teams always ask and static keys never answer. - Clean credential separation. User tokens never leave the hop that received them. Only freshly minted exchange tokens travel onward, so a leaked downstream token says nothing about the user’s original session.
The cost is a round trip to the authorization server per hop — real, but tamed by caching minted tokens until near expiry (exchanges are deterministic for the same inputs, so caching is safe) and by keeping token TTLs short. Kong’s OpenID Connect plugin, Okta, and similar gateways/IdPs expose exchange as a standard grant, so in practice you’re configuring, not implementing.
How They Combine in Production
Mature systems layer these rather than choosing one:
- mTLS everywhere as the floor. Every pod-to-pod connection is mutually authenticated via the mesh. This kills network-level impersonation and gives you a proven workload identity on every connection, regardless of what the application layer does.
- Token exchange on top for user-carrying requests. When a request carries end-user identity, each hop exchanges for a fresh downstream token. The mTLS layer proves which workload is connected; the exchanged token proves which user that workload is acting for.
- Pure service tokens (client credentials) for machine workloads with no user in the loop — cron jobs, schedulers, replication. Same validation code path, simpler minting.
This layering is exactly what NIST SP 800-207, the zero-trust architecture standard, describes abstractly: per-connection authentication below, per-request authorization above, nothing trusted for being “inside.”
Migration: Retiring the Shared Secret
If you’re starting from shared API keys, the migration that works is additive:
- Measure first. Log every internal request with the key used and its calling workload (you can derive this from network metadata or deployment tags). You can’t deprecate what you can’t see.
- Introduce dual acceptance. Receivers accept both key and token during the window, but treat key-authenticated requests as a degraded class: lower rate limits, extra audit logging.
- Migrate callers one service pair at a time, starting with the highest-value targets (payments, PII stores). This is the strangler pattern applied to credentials.
- Expire, don’t disable. Set a hard expiry date on the shared key, announce it, and let the 401s do the final arguing. A key with no expiry gets migrated never.
The end state is worth the sequence: every internal request carries a verifiable workload identity, user identity propagates through a cryptographically auditable chain, and the blast radius of any single compromised component shrinks to the scopes that component actually holds. That’s the difference between a network with a wall and an architecture with an identity system.