Agentic Observability: Monitoring Multi-Agent LLM Systems in Production

Introduction: The Monitoring Gap No One Is Talking About

Your LLM application worked fine until it started using tools.

Before agents, monitoring was straightforward: latency, error rate, token count, cost per request. You had a prompt in, you had a completion out, and you could measure every dimension of that interaction. Then your system started calling APIs, executing code, querying databases, spawning sub-agents, and making decisions across multiple reasoning steps. Now your most critical failure modes are invisible to your existing dashboards.

An agentic system doesn't fail the way a web service fails. It doesn't return a 500 error. It returns a subtly wrong answer after taking a hundred tool calls and burning $40 in GPU time. The API was healthy. The model was responsive. The failure was in the reasoning chain — and you had no visibility into it.

This is the agentic observability gap. Closing it requires rethinking what "observability" means for systems that reason, plan, and act. This article is a practical guide to doing that.

Why Standard LLM Monitoring Falls Short

Before building an observability strategy for agentic systems, it's worth understanding precisely what's missing from conventional LLM monitoring.

Token-level metrics miss reasoning quality. Standard LLM dashboards track tokens per minute, time to first token, and cost per 1K tokens. These tell you how efficiently the model is generating text. They tell you nothing about whether the reasoning that preceded that text was sound. An agent can produce coherent, token-efficient output while making a catastrophic planning error three steps upstream.

Single-request tracing misses multi-turn state. Conventional tracing follows a single prompt-completion pair. Agentic systems maintain state across arbitrarily long interaction windows — working memory that persists across steps, tool results that inform subsequent reasoning, shared context across parallel sub-agents. If you're only tracing at the request level, you're blind to failures in inter-step state management.

Uptime monitoring misses reliability degradation. An agentic system can be technically "up" — responding to requests — while producing increasingly unreliable outputs as retrieval quality degrades, tool APIs drift, or model behavior shifts. Traditional uptime metrics don't capture this kind of graceful degradation.

Cost dashboards miss loop risk. Standard cost monitoring tracks spend at the transaction level. Agentic systems introduce unbounded cost risk: a misconfigured agent can enter a reasoning loop, burning tokens with each iteration until it hits a hard limit or your budget runs out. By the time your cost alert fires, you've already paid for the damage.

The core problem is that agentic systems introduce a new class of failure modes — reasoning failures, planning failures, tool-call failures, state corruption — that exist at a different level than the metrics conventional monitoring captures. You need a monitoring architecture designed for this specific failure mode topology.

The Four-Layer Agentic Observability Stack

A production-ready observability strategy for agentic systems covers four distinct layers, each requiring different instrumentation and tooling.

Layer 1: Trace Observability — Following the Reasoning Chain

The foundation of agentic observability is step-level tracing — capturing every reasoning step, tool call, and state transition in the execution graph.

This goes beyond standard OpenTelemetry tracing in one critical way: the semantics of the span matter, not just its timing. A span representing a tool call to a search API is fundamentally different from a span representing an internal reflection step. Your observability layer needs to capture not just that a step happened, but what type of step it was, what inputs it received, and what outputs it produced.

What to instrument:

• Step type taxonomy — Classify every step in the agent loop: reasoning, tool_call, tool_result, plan_update, sub_agent_spawn, sub_agent_result, artifact_generation, error, circuit_breaker_triggered. This taxonomy is the foundation for everything else.
• Step inputs and outputs — Log the full input/output of each step, not just its duration. For reasoning steps, capture the prompt and completion. For tool calls, capture the full request and response. This data is essential for debugging failures.
• Step-level latency — Track latency for each step independently, including time-to-first-token for reasoning steps and wall-clock time for tool calls. Correlate step latency with total end-to-end latency to identify bottlenecks.
• Branching and parallelism — When an agent spawns parallel sub-agents or makes parallel tool calls, trace each branch independently and capture the dependency graph. Total latency is the critical path through this graph, not the sum of all branches.

Recommended tooling: OpenTelemetry with custom span attributes is the minimum viable foundation. For higher-level abstraction, LangSmith, Phoenix (by Arize), or Weights & Biases Weave give you purpose-built agent tracing with reasoning step reconstruction. For production-scale systems with custom agent runtimes, building on top of OpenTelemetry's SDK with a custom collector pipeline gives you the most flexibility.

Layer 2: Cost Observability — Tracking the Token Meter

Agentic systems make cost management genuinely hard. A single user request can trigger dozens of LLM calls — the initial reasoning, tool-call summaries, sub-agent summaries, final synthesis. Multiply this by concurrent users and you have a cost structure that's nearly impossible to reason about without instrumentation.

Per-step token accounting. Track tokens at the step level, not just the request level. This requires instrumenting your LLM calls to log input and output token counts per step, then aggregating by step type, tool, and sub-agent. The goal: for any running session, you can answer "where is the token spend happening?"

Cost attribution by task type. Not all tasks cost the same. A reasoning-heavy step that uses a frontier model costs orders of magnitude more than a tool-call summary step handled by a smaller model. Tag each LLM call with the task type and model used, then build cost dashboards that show spend by task type. This is the data you need to make routing decisions.

Loop detection and circuit breakers. The most dangerous cost risk in agentic systems is the unbounded loop — a reasoning chain that never terminates, either because the agent keeps re-planning without converging or because a tool call keeps returning partial results that trigger retries. Instrument a "steps in current reasoning cycle" counter that increments with each LLM call and resets when a user-visible output is produced. Set alerts at configurable thresholds. Implement hard circuit breakers: if the counter exceeds N steps, halt execution and return an error rather than continuing to burn budget.

Cost-per-outcome tracking. This is the FinOps metric that matters most for agentic systems: not cost per token, but cost per successful task completion. Measure this at the session level and track it over time. If cost-per-outcome is trending up without a corresponding improvement in outcome quality, you have an efficiency problem that cost-per-token metrics would miss.

Layer 3: Reliability Observability — Detecting Reasoning Failures

This is the hardest layer to instrument, because it requires evaluating the quality of the agent's reasoning, not just its efficiency.

Task completion rate. Track whether the agent successfully completed the task it was given, across sessions. This is a coarse metric but essential: if 30% of your agentic sessions are ending in failure, that's a signal that needs investigation regardless of what your latency or cost dashboards show.

Tool-call failure rate and categorization. Instrument every tool call to capture: whether it succeeded, failed, or returned a partial result; the error type if it failed; and whether the failure was transient (network timeout) or deterministic (authentication error, invalid input). Aggregate these by tool to identify unreliable integrations. A tool with a 15% failure rate is a production reliability problem regardless of how fast it is when it works.

Retrieval quality drift. For agents that depend on retrieval (RAG agents, agents with knowledge bases), monitor retrieval quality over time. Track metrics like the fraction of retrieved context chunks that are actually referenced in the agent's reasoning steps — if this ratio drops, it suggests the retriever is returning increasingly irrelevant context. Embedding drift detection (comparing query embedding distributions over time) can serve as an early warning system.

Hallucination and fabrication detection. The agentic equivalent of standard LLM hallucination detection — but compounded by the fact that the agent may be building on false tool-call results or corrupted intermediate state. Techniques include semantic similarity between agent claims and retrieved context, cross-validation against known-ground-truth queries, and LLM-as-judge evaluation on a sample of production outputs.

Layer 4: Security Observability — Watching for Agentic Attack Surfaces

Agentic systems introduce a new class of security concerns that compound traditional LLM vulnerabilities.

Tool-call anomaly detection. An agent that suddenly starts calling tools it hasn't called before — or calling familiar tools with unexpected parameters — may be exhibiting the early signs of a prompt injection attack. Instrument a baseline of normal tool-call patterns (which tools, which parameter patterns, which sequences) and alert on statistically significant deviations.

Resource consumption monitoring. Agentic systems can consume resources in unexpected ways — writing large files, making excessive API calls, spawning unbounded sub-agents. Monitor process-level resource usage (CPU, memory, file descriptors, network connections) per agent session. Set baseline profiles and alert on deviations.

State isolation verification. For agents handling multiple concurrent sessions, verify that session state is properly isolated — that one session's memory and tool access can't bleed into another. This requires instrumentation at the session management layer, not just the agent layer.

Practical Implementation: Building the Instrumentation

Now for the implementation question: how do you actually instrument a production agentic system? The answer depends on your agent framework.

If you're using LangChain: LangSmith provides built-in step-level tracing with minimal code changes. You get reasoning step reconstruction, token accounting, and tool-call tracing out of the box. The trade-off is vendor lock-in and cost at scale.

If you're using LangGraph, AutoGen, or a custom agent runtime: You'll likely need to build on top of OpenTelemetry. The key is designing a tracing context that persists across the full agent session — not just a single LLM call. This requires threading a context object through your agent loop and using OpenTelemetry's context propagation to maintain continuity across async boundaries and sub-agent calls.

The custom instrumentation checklist:

1. Define your step type taxonomy before writing any instrumentation code. Every step in your agent loop should map to one type in your taxonomy.
2. Instrument at the agent loop level, not inside individual tools. Tools should be dumb — the agent loop is where you capture the step metadata that makes observability meaningful.
3. Emit spans for every step with at minimum: step type, step number in the reasoning chain, parent step (for nested reasoning), input tokens, output tokens, latency, and a summary of the step output.
4. Track a session-level context object that persists across all steps: session ID, user ID, task description, accumulated cost, step counter, and circuit breaker state.
5. Build aggregation queries first. The raw trace data is only as useful as your ability to query it. Design the queries you want to run — cost by step type, latency by tool, failure rate by session — before finalizing your instrumentation schema.

The Minimum Viable Agentic Monitoring Stack

If you're building this for the first time and need to ship something production-ready without a six-month observability project, here's the minimum stack:

Step 1 — Capture step-level traces. Use OpenTelemetry or LangSmith to log every reasoning step, tool call, and sub-agent result with input/output summaries and token counts. This is non-negotiable.

Step 2 — Build a cost-per-session dashboard. Take your per-step token data and aggregate it at the session level. Show: total tokens, total cost, cost by step type, and cost by model used. Add an alert when cost-per-session exceeds a threshold you define based on observed baselines.

Step 3 — Implement circuit breakers. Set a maximum step count per session (start conservative — 50 is reasonable for most use cases). When the counter hits the limit, halt execution, log the failure, and return an error. Track how often circuit breakers fire — a rising rate is an early warning sign.

Step 4 — Add tool-call reliability metrics. Track success/failure rates per tool, categorized by error type. Alert on tools whose failure rate exceeds your threshold. This alone will catch most of your agentic reliability problems.

Step 5 — Instrument loop detection. Track when the agent makes the same tool call with similar parameters within a configurable lookback window. This catches reasoning loops before they burn through your budget. Alert and halt when detected.

This stack won't catch every failure mode. But it will give you visibility into the three most common and most expensive ones: cost overruns from unbounded reasoning, reliability failures from flaky tool integrations, and reasoning degradation from retrieval quality problems.

Conclusion: Observability Is the Infrastructure

The uncomfortable truth about agentic systems is that they fail in ways that are qualitatively different from the failures we've learned to monitor in traditional software. A reasoning chain that takes a hundred steps to complete an objective can fail at any of those hundred steps — and often the failure is invisible until it produces a wrong answer.

The teams that will operate agentic systems successfully in production are the ones that treat observability as a first-class infrastructure concern, not an afterthought. That means instrumenting the reasoning chain, tracking cost at the step level, monitoring tool reliability, and building circuit breakers before the first production deployment — not after the first $10,000 bill from an unbounded loop.

The good news: the primitives exist. OpenTelemetry, purpose-built agent tracing tools, and the standard observability stack you're already running can be extended to cover agentic workloads. The hard part is designing the instrumentation taxonomy that makes the trace data actually useful for debugging agentic failures — and that's a design problem, not a tooling problem.

The monitoring stack for the agentic era is being built right now. The teams that build it right will be the ones who understand that the most important metric for an agentic system isn't how fast it responds — it's whether it was thinking correctly.