NBA Prop Model Scored 63.7% in Testing, Flopped at 0.51 AUC in Production

·1 views

A machine learning model built to predict NBA and WNBA player prop outcomes showed a 63.7% hit rate and 0.56 AUC in offline backtesting, but performed no better than a coin flip in live deployment. Investigators initially suspected a bug in the serving code, but recomputation confirmed the predictions matched to floating-point precision. The real cause was a train-serve mismatch: the training dataset had been exported from a narrow spring snapshot of a database, and neither the query nor the date window was ever documented. Without recorded provenance, there was no way to detect that the training population differed from what the model encountered in production. The team has since parked the model, made dataset provenance mandatory in all changelogs, and will only promote its replacement using forward-settled live data — no backtests allowed.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Catalog-Based Context System Helps Coding Agents Load Only Relevant Code Notes

A developer has proposed replacing the all-or-nothing AGENTS.md approach for AI coding agents with a lightweight catalog system called pi-context. Instead of loading all project documentation at session start, the agent receives only a table of file paths and short trigger-style descriptions, then selectively reads individual notes when their description matches the current task. Each feature in a project gets its own small markdown note stored in a dedicated folder, and the catalog is regenerated every 20 commits or at sprint end to stay current with the codebase. The system uses a strict description format — phrases like 'Load when working on…' rather than summaries — so the language model can make precise, reliable decisions about which notes to pull. The author reports that indexing 20 features costs roughly 300 tokens at startup, making the approach significantly more token-efficient than loading full documentation into every session.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How RGB Triplets Convert to HEX Codes and Where the Process Can Go Wrong

Converting an RGB color value like rgb(255, 99, 71) to a HEX code such as #FF6347 is a straightforward base-10 to base-16 conversion performed separately on each of the three color channels. Because each channel fits exactly into two hexadecimal characters, the conversion itself is mathematically lossless with no rounding involved. However, problems can arise from out-of-gamut inputs, float-normalized values, or mismatched color spaces such as Display P3 versus sRGB. For example, copying a wide-gamut color picker value directly into a CSS stylesheet without conversion can produce visibly different colors than intended. Developers building color pipelines are advised to validate input ranges, choose a consistent rounding strategy for float inputs, and confirm the source color space before converting.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

React 19 Prerendering Bug Causes Duplicate Title and Meta Tags in Static HTML

A developer discovered that prerendered pages built with Vite and React 19 were shipping with multiple duplicate title, meta, and canonical tags in the final HTML files. React 19's new feature of hoisting head elements from anywhere in the component tree into the document head causes stale tags from transitional route components to persist during prerendering. An initial fix that attempted to remove duplicates via DOM manipulation failed silently, because React's reconciler runs after the cleanup and reinserts the deleted nodes before the page is serialized. The root cause is that page.evaluate() and page.content() are separate browser round-trips, giving React time to recommit removed elements between the two calls. The working solution bypasses DOM mutation entirely, instead reading the resolved title and canonical values and applying deduplication as a pure string transformation on the raw HTML output.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why Dart Isolates Can Slow Your Flutter App If Used Incorrectly

A Flutter developer discovered that parsing an 18 MB JSON file on the main thread caused a 900ms UI freeze on a mid-range Android device, prompting a deeper look at Dart isolates. The key distinction is between I/O-bound tasks, which are best handled with async/await on the main isolate, and CPU-bound tasks like JSON parsing or image manipulation, which genuinely require isolates to avoid dropped frames. Dart's event loop efficiently manages async waiting, but it cannot yield during a tight synchronous loop, meaning no amount of async/await will unblock the UI thread for pure computation. Common tools like compute and Isolate.run exist for offloading CPU-heavy work, but misapplying them to I/O-bound tasks adds latency without benefit. The author warns that deferring work with Future() does not offload it to another thread and that many perceived concurrency problems are simply cases of using the wrong async API.

0 comments Read more at DEV Community