NVIDIA Dynamo's SLA Planner Automates GPU Scaling for LLM Inference on Kubernetes

Static GPU allocation struggles with LLM inference because request patterns and sequence lengths vary, leading either to wasted compute or to missed latency targets. NVIDIA Dynamo's SLA Planner addresses this by profiling workloads and autoscaling GPU worker replicas to meet time-to-first-token and inter-token latency goals. A new guide explains how to deploy the planner on Kubernetes, covering Prometheus monitoring, DynamoGraphDeploymentRequest configuration for automated profiling, and Grafana dashboards for tracking scaling decisions. Prerequisites include a cluster with at least two NVIDIA GPU nodes, the GPU Operator, Helm 3, and NGC and Hugging Face access tokens. The result is an LLM deployment that scales GPU workers automatically to hold SLA targets.