What LLM Observability Means and Why It Matters for AI in Production

LLM observability refers to runtime visibility into a large language model or agent system, capturing traces, metrics, and logs to make production failures diagnosable rather than mysterious. A trace records every model call, tool call, and retrieval within a single request, along with inputs, outputs, token usage, latency, and cost. Unlike evaluation, which measures output quality across a dataset, observability focuses on documenting exactly what the system did during a specific run. The discipline exists largely because LLMs are non-deterministic — the same input can produce different outputs, tools, or execution paths, making local reproduction of production failures unreliable. Capturing what happened in real time is therefore the only reliable way to debug agent systems operating at scale.