AI Agent Observability: What to Log When Things Go Wrong at 2 a.m.
2026-07-22 · 6 min
A logging and trace schema for production agents — correlation IDs, synthesis outcomes, tool gates, and the signals that separate model bugs from workflow bugs.
At 2 a.m., nobody wants your 400-line stack trace. They want to know: what did the agent think it knew, what did it do, and where did the workflow stop being recoverable?
Standard APM tells you the service was up. It does not tell you whether the agent confidently recommended a trade on stale filings or escalated a false positive because retrieval returned an empty set disguised as success.
Agent observability is not "log the prompt." It is designing evidence for postmortems — structured, correlatable, and safe to retain. Finance and security teams feel this first because their incidents have auditors attached.
The difference between a two-hour incident and a two-week investigation is usually logging discipline, not model intelligence. When synthesis outcomes, gate decisions, and evidence versions are missing, every on-call becomes a guessing game about what the agent believed.
The unit of observability is the decision, not the request
HTTP request logs are necessary. They are insufficient.
Define a decision_id that spans retrieve, synthesize, gate, and act. Every log line for that workflow carries it. When Slack screams, you grep one ID and rebuild the story.
Without decision_id, you have shards: a retrieval log in one service, a tool call in another, a chat transcript in a third. Postmortems become archaeology.
Propagate decision_id to the UI when humans intervene — approvals, overrides, dismissals. The human action is part of the decision story, not a separate Slack thread lost to retention policy.
Log layer one: inputs frozen
Timestamp the data slice. Version the retrieval index. Hash the evidence object. Record model and reasoning engine versions.
If you cannot replay inputs, you cannot explain outputs. Examiners treat unfrozen inputs as unfrozen conclusions.
For finance: log which tickers, which filing versions, which market data cutoff. For cyber: log which alert IDs, which rule versions, which threat intel feeds.
Include environmental context: deployment region, feature flags active, agent version. Incidents that ignore environment chase red herrings for hours.
Redact secrets at the field level, not by skipping logs. Incidents without inputs are worse than incidents with careful redaction.
Store input hashes even when raw payloads cannot be retained long-term. Hash plus version metadata often suffices to prove what the system saw without hoarding sensitive blobs.
Log layer two: synthesis outcomes
Log whether the reasoning layer returned proven, inferred, or inconclusive — as enums, not prose.
Capture trace step counts, conflict flags, and numeric binding completeness. "Three figures inferred" is actionable. "Seemed unsure" is not.
When Interdot or similar services attach logic traces, persist trace IDs and export hashes to your system of record. Traces are legal artifacts, not debug fluff.
Alert on inconclusive spikes and confident-wrong reversals. Those signals precede user-visible incidents.
Segment synthesis metrics by workflow and user cohort. A global inconclusive rate hides a broken retrieval index affecting only one desk or one SOC playbooks bundle.
Log layer three: gates and side effects
Log risk tier, gate decision, approver identity, and time-to-approve. If an agent attempted a high-tier action, log blocked versus executed explicitly.
Tool calls need parameters, outcomes, and retry policy — especially financial transfers and security containment actions.
The worst failures are silent allows: gate configured wrong, tool executed, logs show success because the tool succeeded even though the decision was wrong.
Add synthetic canaries: scheduled test decisions that should always block or always inconclusive. If a canary executes, page immediately — your gate drifted without anyone deploying code.
Log layer four: human and model disagreement
When humans override agent recommendations, log the override, the prior trace ID, and structured reason codes.
This is training gold and governance gold. Patterns of override reveal bad retrieval, bad synthesis, or bad UX — distinguishable only if you log systematically.
Teams that treat overrides as embarrassment hide the signal that would fix the system.
Weekly review overrides with product and domain experts — thirty minutes, top reason codes only. This meeting prevents quarter-end surprises when compliance asks why humans distrust the agent.
What not to log (and what to log instead)
Do not log raw prompts with PII to unstructured files. Log structured evidence schemas with field-level redaction.
Do not log only final natural language answers. Log machine-readable conclusion objects.
Do not rely on chat transcripts as audit logs. Transcripts are narratives. Logs are contracts.
Retention policy must match compliance, not disk cost. Short retention saves money until the subpoena arrives.
Align observability retention with your system-of-record retention. If traces expire in seven days but audits look back seven years, you built a mirage of governance.
Dashboards that help at 2 a.m.
Panel one: inconclusive rate by workflow, 15-minute granularity.
Panel two: gate block rate versus execute rate for high-tier intents.
Panel three: retrieval empty-success rate — queries that returned 200 with no usable evidence.
Panel four: p95 end-to-end decision latency with synthesis tail highlighted.
Panel five: override rate with top reason codes.
If your on-call runbook links to these five panels, you have observability. If it links to "check the model," you have hope.
Keep dashboards boring and stable. Incident response is not the time to discover someone renamed a metric last sprint. Treat observability schema changes like API changes — versioned and announced.
Runbooks tied to signals
Inconclusive spike → check retrieval freshness and index deployment.
Confident-wrong reversal → pull trace IDs, diff evidence bindings, patch synthesis policy.
Gate bypass → freeze high-tier tools, audit RBAC, incident bridge.
Latency tail → profile synthesis and retrieval separately; do not blame the network by default.
Vendor outage → verify cached trace behavior and fallback inconclusive paths; never let agents improvise conclusions because the reasoning API timed out.
Game-day these runbooks quarterly. Logs nobody practices reading are logs nobody reads during incidents.
Include a "trace replay drill" in game days: on-call must reconstruct a material decision from logs alone in under ten minutes. Fail the drill, fix the schema — not the people.
Observability is empathy for future you — exhausted, blamed, and under time pressure. Build logs that tell the truth fast, without leaking secrets, without requiring the vendor on the bridge. Agents that cannot be observed cannot be trusted in finance or cyber. Full stop.
Treat observability schema as a product surface: version it, document it, and train new hires on it the same way you train them on on-call rotation. The log line you skip today is the missing sentence in the postmortem that costs you a quarter of trust tomorrow.
Finance and cyber leaders who invest in observability early sleep better — not because incidents disappear, but because incidents become legible before the narrative hardens.