Dagster and dbt: Do You Need Both?
A common question is whether teams using Dagster also need dbt, since both draw lineage graphs, schedule work, and run tests. The author explains they are different kinds of tools: dbt is a SQL transformation framework that orders and tests SQL models but does not schedule itself, while Dagster is an orchestrator that handles scheduling, retries, partitions, and arbitrary Python. The overlap lies in lineage and testing, but Dagster's graph covers the whole pipeline, including Python loading and model training. dbt remains valuable when transformations are mainly SQL in one warehouse, when analysts write models, or when portability across orchestrators matters. The author's own production pipeline runs on Dagster without dbt, which suits teams where transformations are mostly Python and one engineer manages everything.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in