Developer fixes recurring job timeouts by splitting tasks with sub-step idempotency

·1 views

A developer running an automated vault-auto-ingest job faced repeated timeouts over three consecutive days when a single processing step failed to complete within its 40-minute window. The step responsible for digesting 28 hours of Claude and Codex conversation logs was too large to finish reliably during busy periods, causing a completion marker to never be set and all four daily retry slots to fail in sequence. The fix involved splitting the problematic step into two independent sub-tasks, each with its own marker file and a 25-minute timeout, so that a timeout in one would not force the other to re-run. A shared composite marker is only written once both sub-steps succeed, preserving the existing outer retry logic. This approach, termed sub-step idempotency, ensures that already-completed units of work are skipped on subsequent retry attempts within the same day.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Public ATS APIs Let Developers Pull Normalized Job Postings Without Scraping

Major applicant tracking systems like Greenhouse, Lever, and Ashby expose public JSON APIs that return live job postings without requiring authentication. A developer writing on DEV Community highlights that the core challenge is not access but normalization, since each platform returns data in a different structure with inconsistent fields for location, compensation, and remote status. A third-party Apify actor called Job Postings API addresses this by auto-detecting which ATS a company uses and returning a standardized record per job listing. The post notes that Ashby boards stand out for publishing structured compensation data, with all 727 current OpenAI roles reportedly carrying salary ranges accessible via a simple GET request. The author cautions that fields like salary and remote status are only populated when the underlying ATS data includes them, meaning the tool normalizes shape but cannot generate missing information.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

OpenAI and Hugging Face Security Breach Exposes Risks in Model Evaluation Pipelines

OpenAI and Hugging Face disclosed a security incident involving a breach that occurred during model evaluation, initially characterised by the companies as a minor issue. The breach exposed a structural vulnerability in eval-as-a-service architecture, where malicious input payloads were able to interact with the environment running evaluation code. Security experts warn that most automated evaluation frameworks operate in permissive, unsandboxed environments because they require tool access, data access, and environment persistence — making them susceptible to remote code execution attacks. The incident highlights that proprietary test datasets fed into third-party evaluation APIs are at risk of exposure, particularly when differential privacy measures are absent. Engineers are being urged to run evaluation pipelines in ephemeral, isolated containers, validate model outputs as untrusted inputs, and treat evaluation workflows with the same security rigour as production systems.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

CPSC Product Recall API Has Broken Filters and No Pagination, Developer Finds

A developer testing the U.S. Consumer Product Safety Commission's public Recalls API on July 20, 2026, found that several documented search filters silently return empty results instead of matching data. The Hazard filter, along with UPC and ManufacturerCountry parameters, consistently returns zero records despite matching data existing in the corpus of 9,905 recall entries. The root cause appears to be that the server-side filter matches against empty internal fields rather than the populated display fields. The API also offers no pagination support, though downloading the entire dataset unfiltered takes under half a second due to its small size. The official programmer's guide, last updated in September 2018, documents these filters as functional without noting any known limitations.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Choose the Right RDP Client on macOS for Windows Remote Access

macOS does not include a built-in RDP client, leaving developers who switch from Windows to seek third-party solutions for remote access. Microsoft offers an official RDP client via Windows App, which handles basic connections to Windows desktops and servers but can feel limited when managing multiple machines. Developers often work across several protocols simultaneously — including SSH, VNC, SFTP, and RDP — making a fragmented multi-app setup inefficient. Factors such as image quality, clipboard behavior, input latency, and multi-monitor support also significantly affect day-to-day usability. For users with simple, occasional needs the official Microsoft client may suffice, while those managing complex multi-protocol environments may benefit from a unified remote management tool.

0 comments Read more at DEV Community