SShortSingh.
Back to feed

New Python package brings zero-code OpenTelemetry tracing to Dagster pipelines

0
·8 views

A developer has released opentelemetry-instrumentation-dagster, a package that automatically adds distributed tracing to Dagster pipelines without requiring any code changes to pipeline files. The tool works by patching Dagster's decorator factories — such as op, asset, and multi_asset — before user code is imported, ensuring every operation gets a tracing span automatically. It integrates with the standard OpenTelemetry Python launcher, meaning pipelines are instrumented simply by swapping the run command rather than modifying definitions. The package complements the author's earlier dagster-otel library, which offered explicit opt-in tracing via a decorator, targeting users who prefer zero-touch instrumentation instead. A noted technical challenge is cross-process execution, where Dagster's multiprocess executor spawns fresh interpreters per step, requiring the patch to be reapplied in each new process.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Why Database Schema Diagrams Become Outdated and How to Fix It

Schema diagrams frequently fall out of sync with actual databases because they are static files that no one updates after migrations run. When a developer adds or renames a column, the diagram stored in Confluence, Figma, or a repo image remains unchanged, creating a misleading source of reference. Responsibility for updating diagrams typically falls between teams, so updates are perpetually deferred. The recommended fix is to auto-generate diagrams directly from the live database on every deployment or pull request, rather than maintaining them manually. For teams wanting version control, the generated output can be committed as an SVG or Markdown file and enforced via CI checks that fail if the diagram does not match the current schema.

0
ProgrammingDEV Community ·

Developer Builds Custom Geo-fencing Engine After Google API Proved Too Imprecise

A developer building an app called Muffle — designed to automate phone sound profiles based on location and context — initially used Google's Geofencing API but abandoned it due to poor precision and unpredictable latency. The API failed to detect zone transitions accurately, sometimes delaying exit events until the user had moved several blocks away. To gain finer control, the developer switched to a custom engine built on Android's FusedLocationProviderClient, using the Haversine formula for distance calculations within a persistent Foreground Service. An adaptive polling system was implemented, reducing location update frequency when stationary and increasing it upon detected movement to conserve battery. The shift away from Google's high-level API introduced new challenges around Android's fragmented power management behavior across device manufacturers.

0
ProgrammingDEV Community ·

Framework Proposes Five Autonomy Levels to Govern AI Agents Safely in Production

A developer has published an open-source framework for governing AI agent autonomy, inspired by Garry Kasparov's 2005 observation that process quality determines outcomes more than raw capability. The framework defines five autonomy levels with sixteen controls mapped to OWASP Agentic and ISO/IEC 42001 standards, along with a scoring worksheet and promotion rules requiring evidence before granting greater autonomy. The project was motivated by a Harvard and BCG study of 758 consultants showing the same AI tool produced 40% better results within its competency range but 19 points worse results just outside it. The author argues the key question for AI deployment is no longer human versus no human, but rather how much human involvement each specific use case requires and how safety can be verified. The draft framework is publicly available on GitHub under a CC BY 4.0 license, and the author is seeking feedback from teams running agents in production to help calibrate its thresholds.

0
ProgrammingDEV Community ·

Why Organizations Must Switch to Phishing-Resistant MFA Now

Traditional multi-factor authentication methods such as SMS codes and push notifications have proven vulnerable to real-time relay and man-in-the-middle attacks, rendering them insufficient for modern security needs. Attackers can intercept or manipulate these legacy MFA mechanisms without requiring users to click a malicious link, creating a false sense of security. Phishing-resistant MFA addresses these weaknesses by cryptographically binding authentication sessions to specific domains, preventing credential reuse on spoofed sites. The two leading standards for phishing-resistant MFA are FIDO2/WebAuthn, which uses public-key cryptography and hardware or biometric authenticators, and certificate-based authentication, commonly deployed in enterprise environments. Security experts now consider migrating to these stronger protocols a critical imperative rather than an optional upgrade for organizations handling sensitive data.