How One Team Uses Terragrunt and YAML to Manage Multi-Workspace Databricks on AWS

A engineering team has shared the infrastructure-as-code architecture behind their Databricks AI platform on AWS, detailing how they use Terragrunt and YAML-driven configuration to manage multiple workspaces at scale. The setup separates reusable Terraform modules from per-environment directory trees, ensuring that engineers edit only YAML files rather than writing Terraform or Terragrunt code directly. A key design decision involves Terragrunt automatically injecting environment prefixes — such as dev_ or prd_ — into resource names, eliminating the risk of cross-environment naming collisions. Each workspace is represented as a folder within the environment tree, allowing the platform to scale without duplicating code. The approach also addresses a Databricks-specific challenge: since every workspace has its own API host URL, provider configurations are generated dynamically using a shared variable map rather than being hardcoded.