Developer Self-Hosts SigNoz to Trace Multi-Step LLM Agent Pipeline Using OpenTelemetry

A developer built and instrumented a four-step LLM research agent — covering plan, retrieve, generate, and synthesize stages — using OpenTelemetry and the self-hosted observability platform SigNoz. The setup was deployed locally via SigNoz's new Foundry CLI tool, which spins up the full stack including ClickHouse, Postgres, and an OTel collector using a minimal configuration file. The agent emitted traces, metrics, and logs with GenAI semantic-convention attributes on every LLM call, enabling visibility into model usage, token counts, and estimated costs per step. After 60 simulated requests, SigNoz automatically surfaced RED metrics — latency, error rate, and throughput — for the service without any manual configuration. The author noted that a key gotcha during setup was that telemetry ingestion is blocked until an admin account is created in the UI, after which data begins flowing within roughly 30 seconds.