10 Hard Lessons From Building a Multi-Agent AI System on Azure and NVIDIA

A developer building their first multi-agent customer support system on Azure AI Foundry and NVIDIA NIM documented ten unexpected technical lessons from the project. Key findings included that token count is a poor cost proxy since different model sizes carry vastly different per-token prices, and that verbatim hash caching is ineffective for natural language workloads, achieving zero cache deflection instead of the predicted 25–40%. Several pitfalls involved observability tooling, such as Azure Monitor failing to capture OpenAI SDK calls without explicit HTTPX instrumentation, and silent version conflicts in OpenTelemetry dependencies breaking trace exports. The developer also found that NVIDIA's reasoning models like Nemotron Nano require a minimum token budget even for simple classification tasks, as low limits cause the model to exhaust tokens on internal reasoning without producing usable output. Additional lessons covered mismatches between catalog model names and actual API strings, the need for dedicated router decision logging, and the difficulty of testing graceful degradation mechanisms within sequential benchmarks.