Engineer Builds On-Demand GitHub Actions Runners Using Temporal and Nomad

·10 views

A developer running a personal homelab solved a self-hosted GitHub Actions runner scaling problem by combining Temporal workflows with Nomad's parameterized batch jobs. The core issue was that outside a GitHub organization, each self-hosted runner can only be registered to a single repository, requiring one persistent runner per repo and wasting cluster resources during idle time. To fix this, a Temporal schedule polls each repository every 30 seconds, detects queued CI jobs, mints fresh registration tokens, and dispatches ephemeral Nomad runner jobs on demand. Each runner registers with GitHub, processes exactly one job, then deregisters and exits, with restarts and rescheduling explicitly disabled to prevent stale runners from lingering. Configuration and secrets are managed through Consul KV and Vault, with scoped Nomad ACL tokens injected at runtime via Vault's workload identity mechanism.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer builds self-parking car simulation using genetic algorithms

A developer created a self-parking car simulation using a genetic algorithm, documented in a 2021 blog post on trekhleb.dev. The project demonstrates how evolutionary computing techniques can teach a simulated vehicle to park autonomously without explicit programming of parking rules. Genetic algorithms mimic natural selection by iterating through generations of solutions, progressively improving performance. The post resurfaced on Hacker News, attracting modest engagement with 5 points and 2 comments.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

Data Science vs Data Engineering: Key Differences and How to Choose Your Path

Data Science and Data Engineering are two distinct but complementary fields within the modern technology ecosystem. Data Scientists use statistics, programming languages like Python and R, and machine learning to extract insights and build predictive models from data. Data Engineers, on the other hand, focus on designing and maintaining the infrastructure — including data pipelines, warehousing, and distributed systems — that makes data accessible and reliable. Without Data Engineers providing clean, structured data flows, Data Scientists would lack the quality inputs needed for meaningful analysis. Professionals are advised to choose between the two paths based on whether they prefer building robust data systems or solving analytical and predictive challenges.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Understanding Nested Arrays in JavaScript: Structure, Access, and Use Cases

Nested arrays in JavaScript are arrays that contain other arrays as elements, allowing related values to be organized into distinct groups within a single structure. A regular array uses a single index to access its values, while a nested array requires multiple indices — one for each level of depth. For example, accessing a value in a two-level nested array uses the syntax array[firstIndex][secondIndex], where the first index selects the inner array and the second retrieves the specific value within it. This approach is practical for grouping related data, such as student names or sets of numbers, in a structured and readable way. Reading indices from left to right helps clarify which group and which value is being accessed at each level.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Silent retry bug left automated trading positions unprotected despite passing all health checks

A developer running an automated paper-trading system discovered that its self-healing recovery routine had been silently failing for hours, leaving open positions without stop-loss protection. The bug stemmed from a deterministic client order ID generated by appending a fixed suffix to an existing identifier, which the broker rejected on every retry after the first attempt because the ID was no longer unique. Every individual component of the recovery chain was logically correct in isolation, yet the combined retry sequence permanently failed without raising any alerts. A 55-point health check passed throughout the entire incident because it verified structural integrity — whether processes were running and files were loaded — but not logical correctness, such as whether broker responses matched expected outcomes. The issue was ultimately uncovered not through code review but by manually running a sync command and reading the raw error returned by the broker.

0 comments Read more at DEV Community