The problem that pushed me to the proxy
A realistic MCP estate looks like this: fourteen servers, three of which you wrote. The rest are somebody else's container image — a vendor's ticketing bridge, a couple of community filesystem servers, an internal team's Python server nobody has touched since March, a hosted endpoint you only reach over TLS with an API key. When a user says "the agent is being slow," the honest answer for eleven of those fourteen is that you have no idea, because you cannot add a span to code you do not own and cannot redeploy.
That is the whole case for proxy-layer MCP observability. SDK instrumentation is better telemetry when you can have it, and I covered that in the MCP server monitoring guide — if every server is yours, go do that instead. The moment the estate is mixed, the only surface with uniform coverage is the L7 hop all fourteen conversations pass through.
Why the proxy is the only full-fidelity vantage point
Three properties make the gateway hop uniquely good for MCP.
Coverage is structural, not negotiated. Every MCP exchange over HTTP goes through the proxy by construction. You do not have to convince a vendor to ship OTel or wait for a maintainer to merge your instrumentation PR. A fifteenth server appears in your dashboards the moment you add the route.
The protocol is uniform even when the servers are not. MCP encodes messages as JSON-RPC 2.0, so a tools/call against a vendor's server has the same wire shape as one against your own. Per the JSON-RPC 2.0 specification, every request object carries jsonrpc, method, an optional params, and an id. One extraction rule works across all upstreams.
You see both directions of the failure. Server-side instrumentation shows what the server did with requests it received. It cannot show the request that never arrived because a connection was reset, or the SSE stream a client abandoned mid-flight. The proxy sees the abort, the truncated stream, the retry storm — and a meaningful share of "the agent is broken" reports live there. It is the same argument that justifies a gateway in front of model endpoints, which I worked through in the piece on inference API gateways.
What the transport actually looks like on the wire
The shape of MCP over HTTP is unusual enough to break naive instrumentation, so get this straight before writing config. The current transport is Streamable HTTP, defined in the MCP transports specification; it replaced the HTTP+SSE transport from protocol version 2024-11-05. What matters to a proxy operator:
- One endpoint path, two methods. The server exposes a single MCP endpoint — say
/mcp— supporting both POST and GET. Client-to-server JSON-RPC messages are POSTs; a GET opens an SSE stream so the server can push without being asked. - A POST can answer two different ways. For a JSON-RPC request body, the server may respond with
Content-Type: application/jsonand a single object, orContent-Type: text/event-streamand an SSE stream. Your metrics pipeline has to handle both. - Notifications and responses get 202. A POST body that is a notification or response, not a request, gets 202 Accepted with no body. A status-code-only dashboard reads a healthy stream of 202s as something odd. It is not.
- Session identity lives in a header. A server may assign a session at initialize and return it in
Mcp-Session-Id; clients send it on subsequent requests. This is the one field you get for free without touching the body, and it makes per-conversation analysis possible. - Protocol version is also a header. As of the 2025-06-18 revision, clients must send the negotiated version in
MCP-Protocol-Version. Cheap to log, invaluable when chasing version skew. - Batching is gone. The 2025-06-18 changelog removed JSON-RPC batching, so one POST body is one logical operation again. Older clients pinned to 2025-03-26 may still batch, and an array body defeats a naive "read
.methodfrom the JSON root" rule.
What to extract at L7
Split the fields by cost. Headers and connection facts are nearly free. Body inspection is not.
Free or near-free: HTTP method, response status, Content-Type of the response (this is how you classify JSON versus SSE without parsing anything), Mcp-Session-Id, MCP-Protocol-Version, request and response byte counts, upstream connect and header times, whether the client aborted.
Requires reading the request body: the JSON-RPC method — initialize, tools/list, tools/call, resources/read, and so on — plus the tool name, which for tools/call lives at params.name, and the request id if you want to correlate a response to its request.
Requires reading the response stream: SSE event count, time to first event, total stream duration, and the JSON-RPC error.code when a server returns a protocol-level error inside an HTTP 200.
That last one is the trap nobody warns you about. A JSON-RPC error is not an HTTP error. A server that returns {"jsonrpc":"2.0","id":7,"error":{"code":-32601,"message":"Method not found"}} returns it with HTTP 200. If your MCP error rate is built on 5xx counts, it will read zero during an outage in which every single tool call is failing.
The hard parts, stated honestly
Path-based metrics are useless here
Every MCP request hits the same path. All of your traffic is POST /mcp. The dimension you actually care about — which method, which tool — is in the body. Standard gateway dashboards, which key everything off route and status, will show you one enormous bucket and tell you nothing. Any useful MCP metric requires a label sourced from the payload, and that requirement is what drives every design decision below.
SSE destroys request-duration semantics
Request duration on a streamed response measures how long the stream stayed open, not how long the work took. A 90-second SSE stream is not a 90-second latency problem; it may be a perfectly healthy notification channel. If you feed SSE responses into the same latency histogram as JSON responses, your p99 becomes a measure of how chatty your servers are.
Split the metric. For streams, the number that behaves like latency is time to first event; total duration belongs in a separate histogram, ideally alongside an event count so you can distinguish a long-and-productive stream from a long-and-stalled one.
Body buffering has a real cost
To read method you must hold the body in memory, which costs latency and memory per in-flight request. Two mitigations: cap the bytes you inspect (method and params.name sit near the front of essentially every real MCP request, so a few kilobytes is plenty), and never buffer response bodies for streaming content types — classify off Content-Type and let the stream pass through.
Session affinity is a routing constraint, not just a metrics field
If a server holds state for a session, requests carrying that Mcp-Session-Id must land on the same upstream instance. Round-robin across replicas produces session-not-found errors that look like random flakiness. Hash on the session header, and make unrecognised-session requests an explicit metric rather than a mystery.
NGINX: cheap and surprisingly far
Plain NGINX proxies MCP correctly once you disable the things that break streaming. There is no MCP-aware NGINX module I can point you to, and I will not invent one — what you have is the standard streaming-reverse-proxy pattern plus a scripting layer for body inspection.
location /mcp {
proxy_pass http://mcp_upstream;
proxy_http_version 1.1;
# SSE requires these. Without them the stream is buffered
# and the client sees nothing until the server closes.
proxy_buffering off;
proxy_cache off;
proxy_set_header Connection "";
# Long-lived GET streams must not be reaped as idle.
proxy_read_timeout 1h;
proxy_send_timeout 1h;
# Session and protocol version are pure header work: free.
proxy_set_header Mcp-Session-Id $http_mcp_session_id;
proxy_set_header MCP-Protocol-Version $http_mcp_protocol_version;
}
With nothing more than that, a custom log format gives you a usable first layer of MCP telemetry:
log_format mcp escape=json
'{"t":"$time_iso8601",'
'"status":$status,'
'"method":"$request_method",'
'"resp_ct":"$sent_http_content_type",'
'"session":"$http_mcp_session_id",'
'"proto_ver":"$http_mcp_protocol_version",'
'"req_bytes":$request_length,'
'"resp_bytes":$body_bytes_sent,'
'"dur":$request_time,'
'"upstream_header_time":"$upstream_header_time"}';
$sent_http_content_type does the quiet heavy lifting: text/event-stream versus application/json is your stream-versus-unary classifier, and $upstream_header_time is your time-to-first-byte proxy for streams. Most of the value, zero body inspection.
The JSON-RPC method needs scripting. The njs module (ngx_http_js_module) is the first-party option: read the request body, expose derived values as variables, reference them in the log format or pass them upstream as headers. OpenResty's Lua module is the other well-trodden route. Same pattern either way — read a bounded prefix, parse, pull method and params.name, guard for the array case, and fail open, because a parse error must never become a 500.
Where NGINX stops: you are hand-rolling. No MCP-native concept in the config language, no built-in JSON-RPC-aware metric, no per-tool authorization primitive. You write and maintain the extraction logic and own the buffering trade-offs.
Kong: an actual MCP-aware plugin
Kong ships the AI MCP Proxy plugin, and it is not merely a logging shim. Per Kong's documentation it acts as a protocol bridge between MCP and HTTP, with a mode parameter selecting whether it proxies MCP requests to upstream MCP servers, converts RESTful APIs into MCP tools, or exposes grouped tools as an MCP server. It requires Kong Gateway 3.12 or later and is part of AI Gateway Enterprise — a licensing detail that matters for planning.
The observability-relevant claim in Kong's docs: because the plugin sits in the MCP request flow between client and server, it captures MCP traffic independently of any LLM request flow, and the gateway's existing logging and tracing plugins then apply to it. One documented constraint you should not discover the hard way — do not configure AI MCP Proxy alongside other AI plugins on the same Service or Route.
services:
- name: vendor-mcp
url: https://mcp.vendor.example/mcp
routes:
- name: vendor-mcp-route
paths:
- /mcp/vendor
plugins:
- name: ai-mcp-proxy
- name: prometheus
config:
status_code_metrics: true
latency_metrics: true
- name: http-log
config:
http_endpoint: http://collector.internal:8080/mcp-logs
The practical read: Kong is the shortest path if you are already a Kong shop and can license AI Gateway Enterprise, because the protocol awareness becomes somebody else's maintenance burden. It is also where MCP behaviour is most tightly coupled to a commercial tier. Kong's companion AI MCP OAuth2 plugin covers the authorization side, overlapping the ground in MCP enterprise authorization — worth reading together, since auth failures and observability gaps surface as the same vague ticket.
Envoy: MCP as a first-class gateway concern
Two distinct things live under the Envoy name, and conflating them wastes an afternoon.
Envoy AI Gateway documents MCP Gateway support with an MCPRoute API. From their docs: full support for MCP's streamable HTTP transport aligned with the June 2025 spec, aggregation of multiple MCP servers behind one endpoint, tool filtering, OAuth enforcement with JWT-claim and CEL-based access control, upstream API-key injection, and OpenTelemetry tracing plus Prometheus metrics for MCP requests on the same stack used for LLM traffic. Two architectural details matter for dashboards: the gateway creates unified sessions by encoding multiple backend session IDs and handles SSE reconnection via Last-Event-ID, and tool names are prefixed with the backend name (their example is github__issue_read) to route calls to the right upstream. That prefixing is a gift for metrics — backend attribution is already in the tool label, so you get per-server breakdowns without a join.
apiVersion: aigateway.envoyproxy.io/v1beta1
kind: MCPRoute
metadata:
name: mcp-route
namespace: default
spec:
parentRefs:
- name: aigw-run
kind: Gateway
group: gateway.networking.k8s.io
path: "/mcp"
Plain Envoy is the path when you are not adopting the AI Gateway. Envoy's HTTP filter model runs an ordered chain per request, and each filter can inspect or mutate headers and body. The Lua HTTP filter is the accessible entry point: request and response callbacks where you read a bounded body, extract the JSON-RPC method, and stamp it into dynamic metadata that access logging and stats then consume.
http_filters:
- name: envoy.filters.http.lua
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
default_source_code:
inline_string: |
function envoy_on_request(handle)
local body = handle:body()
if body == nil then return end
local raw = tostring(body:getBytes(0, math.min(body:length(), 4096)))
local m = string.match(raw, '"method"%s*:%s*"([^"]+)"')
local tool = string.match(raw, '"name"%s*:%s*"([^"]+)"')
handle:streamInfo():dynamicMetadata():set(
"mcp", "method", m or "unknown")
handle:streamInfo():dynamicMetadata():set(
"mcp", "tool", tool or "none")
end
The string match above is deliberately crude and I would not ship it as-is — it will happily pick up a name field from elsewhere in a nested payload. Real deployments should decode the prefix properly. The point is the shape: bounded read, extract, stamp metadata, fail open.
Where Envoy stops: plain Envoy gives the most control and the most work. Lua-filter buffering needs care around streaming responses, and high-cardinality labels from tool names will grow your stats surface unless you bound them.
If your MCP servers run on Cloudflare Workers, the edge is your proxy layer, and Cloudflare documents MCP support in its Agents platform for building and hosting remote MCP servers — worth reading before you commit to running your own gateway tier.
The metric set
| Metric | Type | Labels | Why it matters |
|---|---|---|---|
mcp_proxy_requests_total | Counter | server, method, tool, http_status, transport | The base rate. Without method and tool labels every MCP dashboard collapses into one bucket. |
mcp_proxy_jsonrpc_errors_total | Counter | server, method, error_code | Catches protocol errors returned inside HTTP 200. This is the metric most teams are missing. |
mcp_proxy_unary_duration_seconds | Histogram | server, method, tool | Real latency for non-streamed responses only. Keep SSE out of it. |
mcp_proxy_stream_ttfe_seconds | Histogram | server, method | Time to first SSE event — the only stream number that behaves like latency. |
mcp_proxy_stream_duration_seconds | Histogram | server, method, close_reason | Stream lifetime. Paired with close reason it separates healthy long streams from stalls. |
mcp_proxy_stream_events_total | Counter | server, method | A long stream with zero events is a hang. Duration alone cannot tell you that. |
mcp_proxy_request_bytes / _response_bytes | Histogram | server, method, tool | Response size is your best proxy-side signal for context bloat and downstream token cost. |
mcp_proxy_sessions_active | Gauge | server | Derived from distinct Mcp-Session-Id values. Leaked sessions show up here first. |
mcp_proxy_session_missing_total | Counter | server | Requests with an unrecognised session. Usually a broken affinity rule, not a client bug. |
mcp_proxy_body_parse_failures_total | Counter | server, reason | Your own blind-spot meter. Rising values mean your labels are silently degrading to unknown. |
Bound tool cardinality deliberately — an allowlist built from tools/list responses, everything else folded into other. The RPC semantic conventions map cleanly onto MCP if you treat rpc.system as jsonrpc, rpc.method as the JSON-RPC method, and carry the tool name as a separate attribute. That is the bridge I would use to reconcile proxy spans with the model-side traces described in OpenTelemetry AI inference tracing.
Alerting rules, and the reasoning behind each number
These thresholds are starting points derived from reasoning about the protocol, not measurements from a benchmark. Tune them against your own baseline before paging anyone.
JSON-RPC error ratio above 5 percent for 5 minutes, per server. A healthy server's steady-state protocol error rate should be close to zero — errors come from client bugs or broken tools, not normal operation. Anything sustained above a few percent means something structural changed. Five minutes rides out a rolling restart.
error_code of -32601 (Method not found) appearing at all on a server that previously returned none. A version-skew or capability-removal signal, not a load signal, so rate does not matter — the first occurrence is the event. Pair it with the MCP-Protocol-Version label and the cause is usually immediate.
Time to first SSE event p95 above 5 seconds. A judgement call about agent UX: beyond that an interactive agent looks hung to a human. Batch workloads should set this much higher.
Streams open longer than 10 minutes with zero events. Ten minutes of nothing is almost certainly a leaked connection or stalled upstream, not slow-but-working. Deliberately generous so quiet notification channels do not trip it; tighten it if your servers heartbeat.
session_missing above roughly 1 percent of requests. Near zero when affinity works. A steady low-single-digit percentage is the classic fingerprint of balancing across replicas that do not share session state.
body_parse_failures above 0.1 percent. This alerts on your instrumentation, not your traffic. Parse failures should be effectively nonexistent; when they are not, your method and tool labels are filling with unknown and every other alert becomes less trustworthy.
Response byte p99 above 3 times the 7-day baseline, per tool. Relative rather than absolute, because payload sizes vary wildly between a filesystem read and a status check. A tool suddenly returning three times as much data is about to blow up somebody's context window and token bill.
What this does not give you
The ceiling matters more than the pitch.
- Nothing about stdio servers. MCP's stdio transport is a subprocess pipe — no HTTP hop, no proxy, none of this applies. Locally launched servers need endpoint-side instrumentation.
- No internal attribution. You learn a
tools/calltook 4 seconds. You do not learn that 3.8 of those were a slow query inside the vendor's database. The proxy sees the black box's outline, not its contents. - No semantic correctness. A tool returning confidently wrong data returns HTTP 200 with a plausible byte count. Every metric above reads green.
- No model-side context. The proxy does not know which user turn triggered the call or what the model did with the result. Correlating that needs trace context propagated from the client, which third-party clients may not send.
- Encrypted payloads defeat body extraction. End-to-end encryption above the transport puts you back to header-only telemetry, and TLS termination is a prerequisite for any body inspection — a security conversation before an observability one.
The realistic end state is layered: proxy metrics for uniform coverage, SDK spans on the servers you own, shared trace context stitching them together. Where a commercial platform fits is a separate question I worked through in the LLM observability tools comparison.
Where I would start
Staring at a mixed MCP estate, the highest-value first move is not a new platform. It is a custom access-log format recording response Content-Type, Mcp-Session-Id, MCP-Protocol-Version, byte counts, and upstream header time. Header-only, free at request time, and it separates streams from unary calls — the distinction that makes every subsequent metric meaningful.
Body inspection for the JSON-RPC method comes second, once you have decided how many kilobytes you will buffer. The JSON-RPC error counter comes third, and it is the one that will surprise you, because it tends to reveal failures your status-code dashboards have been reporting as perfectly healthy traffic for months.
Affiliate Disclosure: This article contains affiliate links to tools and services we recommend. We may earn a commission at no additional cost to you if you sign up through our links.