New Python package brings zero-code OpenTelemetry tracing to Dagster pipelines
A developer has released opentelemetry-instrumentation-dagster, a package that automatically adds distributed tracing to Dagster pipelines without requiring any code changes to pipeline files. The tool works by patching Dagster's decorator factories — such as op, asset, and multi_asset — before user code is imported, ensuring every operation gets a tracing span automatically. It integrates with the standard OpenTelemetry Python launcher, meaning pipelines are instrumented simply by swapping the run command rather than modifying definitions. The package complements the author's earlier dagster-otel library, which offered explicit opt-in tracing via a decorator, targeting users who prefer zero-touch instrumentation instead. A noted technical challenge is cross-process execution, where Dagster's multiprocess executor spawns fresh interpreters per step, requiring the patch to be reapplied in each new process.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in