gRPC Server Reflection in Production: Gate It, and Use Health Checks Instead

Here is a debugging story that repeats across every gRPC team. A developer runs grpcurl -plaintext localhost:50051 list against a production service and it works. It works because the server has gRPC reflection registered — usually left over from development, where reflection is what makes tools like grpcurl and Postman “just work” without any client stubs. Then someone asks the obvious question: if an engineer on the VPN can enumerate every RPC and its message schema with one command, what exactly is stopping an attacker who can reach the port?

The answer, in most deployments, is nothing. Reflection is a metadata API that describes your entire API surface — service names, method signatures, request and response message definitions — to any client that asks. It is genuinely useful in development and genuinely dangerous when exposed. This post covers what the reflection service actually does, how to gate it by environment without breaking your tooling, and the health-checking protocol that is the right standard surface to leave enabled for probes and load balancers.

What the reflection service actually exposes

gRPC’s strength is binary Protobuf serialization: compact, fast, and completely opaque without the schema. That opacity is an operational nuisance — to hand-craft a request you would need to know every service, every message definition, and every type each message references. The reflection protocol solves this by having the server describe itself: a client calls the standard ServerReflection service, requests a file by name or by symbol, and receives the serialized protobuf descriptors it needs to encode requests and decode responses.

That is exactly what tools like grpcurl use behind the scenes. When grpcurl lists services on a server without you supplying a .proto file, reflection did it. The mental model: enabling reflection is like publishing your OpenAPI document on the same server that serves the API — except the “document” is served over RPC to anyone who can open a connection.

Three facts sharpen the security picture:

  • Reflection is opt-in. Servers do not expose it by default; the author registers it explicitly. The danger comes from it being enabled everywhere out of habit because dev tooling needs it.
  • It reveals structure, not secrets. Field names and types are not credentials — but for an attacker, an accurate API map eliminates the reconnaissance that binary protocols were supposed to make expensive. Internal service names like admin.AdminService are reconnaissance gold.
  • The gRPC docs flag it directly: if your API is reachable by untrusted clients, exposing the reflection service is a security consideration that requires an explicit trade-off decision, not a default.

One versioning detail worth knowing before you grep your codebase: the reflection protocol originally shipped as grpc.reflection.v1alpha, which is now deprecated in favor of stable grpc.reflection.v1. Recent versions of the Go library register both when you call reflection.Register, and there is a separate RegisterV1 for serving only the new version. Either way, both versions go through the same registration call, so one environment gate covers them both.

Gating reflection by environment in Go

The standard library registration is a single call. Here is the pattern as it usually appears — including the problem:

s := grpc.NewServer()
pb.RegisterOrderServiceServer(s, &server{})

// Registered unconditionally: dev convenience, prod exposure.
reflection.Register(s)

if err := s.Serve(lis); err != nil {
	log.Fatalf("failed to serve: %v", err)
}

The fix is to make reflection an explicit, environment-dependent decision. The simplest version uses an environment check so that even if the code is present, reflection never registers in production:

s := grpc.NewServer()
pb.RegisterOrderServiceServer(s, &server{})

if os.Getenv("ENV") != "production" {
	// Dev/staging only: powers grpcurl, Postman, grpc-ui.
	reflection.Register(s)
}

if err := s.Serve(lis); err != nil {
	log.Fatalf("failed to serve: %v", err)
}

For anything beyond a single service, prefer explicit configuration over environment sniffing. A config flag reads better in code review, shows up in your deployment manifests, and can be wired into the same config system that already controls TLS and ports:

type ServerConfig struct {
	Addr              string
	EnableReflection  bool
	EnableHealthCheck bool
}

func NewGRPCServer(cfg ServerConfig, svc orderpb.OrderServiceServer) *grpc.Server {
	s := grpc.NewServer()
	orderpb.RegisterOrderServiceServer(s, svc)

	if cfg.EnableReflection {
		reflection.Register(s)
	}
	if cfg.EnableHealthCheck {
		healthpb.RegisterHealthServer(s, health.NewServer())
	}
	return s
}

Then production sets enable_reflection: false while staging and local development set it true. The important property is not the mechanism — env var, config file, or build flag — it is that “reflection off in prod” becomes a reviewable, diffable fact instead of an accident of what the example code did.

Verify with grpcurl, not with hope

After disabling reflection, confirm it is actually gone — and learn which of your services were exposed in the first place:

# List services on a server with reflection enabled
grpcurl -plaintext localhost:50051 list

# After disabling: expect UNIMPLEMENTED, not a service list
grpcurl -plaintext localhost:50051 list

Against a server without reflection, list fails with an UNIMPLEMENTED status rather than enumerating services — that error is your confirmation. While you are at it, run the check against every environment including staging and any admin ports; reflection sprawl tends to cluster in the environments nobody audits.

Note that disabling reflection does not make gRPC tools useless against production — it makes them require the schema explicitly, which is the correct posture:

# Still works without reflection: bring your own proto
grpcurl -plaintext \
  -proto protos/order.proto \
  -d '{"order_id": "123"}' \
  localhost:50051 order.OrderService/GetOrder

Authenticated operators who need interactive exploration against an internal endpoint can use -proto with the same protos the clients were built from. The attacker without your schema files is back to guessing.

The standard surface you SHOULD leave on: health checking

Disabling reflection raises an immediate operational question: if tools can’t introspect the server, what can load balancers and Kubernetes probes call? The answer is the gRPC health checking protocol — a separate, deliberately minimal standard service (grpc.health.v1.Health) with a Check RPC that returns a serving status, plus a Watch RPC that streams status changes.

This service is designed for exactly the exposure reflection is not: it reveals one bit of information per service — healthy or not — and nothing about your API surface. The server keeps an internal status map, returns SERVING or NOT_SERVING per registered service, and answers NOT_FOUND for unknown service names, so it does not even leak your full service list. The empty-string service name reports overall server health for clients that do not care about a specific service.

In Go, wiring it up takes three lines plus whatever dependency checks you want:

s := grpc.NewServer()
healthcheck := health.NewServer()
healthgrpc.RegisterHealthServer(s, healthcheck)
pb.RegisterOrderServiceServer(s, &server{})

// Report overall status; flip to NOT_SERVING on shutdown
// or when a critical dependency (DB, queue) is down.
healthcheck.SetServingStatus("", healthpb.HealthCheckResponse_SERVING)

// During graceful shutdown, before s.Stop():
healthcheck.SetServingStatus("", healthpb.HealthCheckResponse_NOT_SERVING)

Setting the status to NOT_SERVING during graceful shutdown is the piece teams skip most often — without it, a load balancer can keep routing to a pod that is mid-shutdown. On the probe side, the grpc-health-probe binary is the standard tool for Kubernetes exec probes: ship the static binary in your container and point an exec probe at it, since Kubernetes’ native gRPC probes only cover the simple cases. (The tool now lives in the grpc-ecosystem organization.)

A short production checklist

  • Inventory: run grpcurl ... list against every gRPC endpoint you operate, in every environment. Anything that answers has reflection on.
  • Gate: make reflection registration conditional and default-off; require an explicit opt-in per environment.
  • Replace: register the health service and point your load balancers and Kubernetes probes at grpc.health.v1.Health/Check — that is the supported, minimal surface for liveness.
  • TLS everywhere: reflection behind TLS is still enumeration behind TLS. Transport encryption and reflection gating solve different problems; you want both.
  • Authorize what remains: the health service itself can sit behind the same interceptors as everything else if your threat model requires it — the protocol intentionally reuses normal RPC semantics so standard auth applies.

Reflection is a good tool that got a default it never deserved. Keep it where it earns its keep — local development and staging — and let the health protocol do the one job production probes actually need. The five minutes it takes to gate one registration call is cheap insurance against handing every attacker who can reach the port a complete map of your internal APIs.

Leave a Reply

Your email address will not be published. Required fields are marked *