Developer Fixes Auto-Publish Pipeline That Posted a Two-Year-Old Story as Breaking News

·1 views

A solo developer running a fully automated content pipeline built on Cloudflare Workers and D1 discovered in July that the system had published a two-year-old news story as if it were current. The error occurred because an aggregator resurfaced the old article with a fresh feed timestamp, and two existing safeguards — a recency check and an LLM scoring judge — both passed it through, each blind to the staleness in different ways. The recency check only applied to product items, not news, while the LLM judge scored the story highly because its rubric never asked whether the content was current. After a routine quality check caught the mistake within days, the developer replaced a planned single-line fix with a three-layer solution: widening the deterministic date check to all content types, adding a currency criterion to the LLM rubric, and extracting the actual event date from article text to cross-check against feed metadata. The incident highlighted how automated pipelines inherit only the failure modes their builders have already imagined, making layered, overlapping safeguards more reliable than any single guard.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Two API Settings Tripled GPT-5.6 Sol's ARC-AGI-3 Benchmark Score, OpenAI Says

OpenAI reported that enabling two Responses API settings — retained reasoning and compaction — raised GPT-5.6 Sol's score on the ARC-AGI-3 public benchmark from 13.3% to 38.3%. The initial evaluation harness prevented the model from carrying forward reasoning across steps of the benchmark's interactive 2D puzzle tasks, limiting its performance. Alongside the score improvement, the new configuration also reduced output token usage by approximately six times. OpenAI emphasized that this outcome highlights how API configuration, not just model capability, can significantly influence both agent performance and token efficiency. The findings serve as a practical reminder that benchmark scores reflect the combined effect of the model, its settings, and the evaluation setup rather than model weights alone.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Define Your Data Grain Before Picking a Chart, Analysts Warned

A data modelling guide published on DEV Community argues that the most common error in business reporting is not poor chart design but mismatched data grain — the unit that each row in a dataset represents. Mixing invoice-level, line-level, payment, and financial-summary data in a single table causes totals to multiply and filters to answer the wrong question. The article illustrates the risk with a concrete example: joining a £1,000 invoice total across five line items and summing the result incorrectly yields £5,000. Analysts are advised to state the reporting question in plain language first, then select the appropriate grain — such as one row per invoice, per product line, or per payment — before building any visualisation. The guide also recommends keeping separate fact tables for each natural grain and documenting non-additive measures to ensure dashboards remain accurate when data is refreshed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Popular AI memory benchmark excludes 22% of questions and bars 'I don't know' answers

A widely cited AI memory benchmark, LOCOMO, has a significant methodological gap: its published evaluation harness skips all 446 adversarial questions in Category 5, which are designed to test whether a system can correctly abstain from answering. The harness also instructs the AI judge to 'COMMIT AND ANSWER' and never say 'not specified,' effectively eliminating any measurement of calibrated abstention. The LLM judge used for scoring has been found by an independent audit to accept roughly 63% of deliberately wrong answers, and LOCOMO's own answer key contains errors that cap the realistic maximum honest score at around 93.6. This means a recently reported score of 92.5 — touted as near-perfect memory performance — sits at the ceiling of a flawed benchmark, not at the ceiling of actual capability. The core issue is not fraud but misrepresentation: the metric measures answer accuracy on answerable questions only, yet is routinely interpreted as an overall measure of memory system quality.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Restructure a Go CRUD API Using Packages for Better Maintainability

As Go projects grow beyond simple examples, keeping all code in a single main.go file makes applications harder to navigate and maintain. The solution is to reorganize code into separate packages, each with a distinct responsibility such as data models, HTTP handlers, and server startup. In this approach, a models package holds data structures like the Task struct, while a handlers package manages HTTP request logic, importing from models as needed. This pattern, known as separation of concerns, makes it easier to locate code, fix bugs, and add new features over time. The restructured project uses only Go's standard library, making it accessible to developers learning idiomatic Go project organization.

0 comments Read more at DEV Community