Developer builds Prometheus exporter for Dagster using GraphQL polling instead of Pushgateway
A developer has released dagster-prometheus-exporter, an open-source Go binary that collects Dagster pipeline metrics by polling Dagster's GraphQL API at regular intervals. The tool was created to address a key limitation of Dagster's officially recommended approach, where metrics are pushed to a Prometheus Pushgateway from within a running job — meaning crashed, OOM-killed, or queued runs never report any data. The exporter maintains in-memory state and decouples metric collection from serving, so the /metrics endpoint continues returning last-known data even if a GraphQL call fails. It tracks metrics including active run counts and durations, completed run statistics, queue backlogs, schedule and sensor statuses, and code location load errors. The project is available on GitHub and supports deployment via Docker or a Helm chart for existing Dagster installations.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in