SShortSingh.
Back to feed

Goodhart's Law: Why Hitting a Benchmark Target Often Misses the Point

0
·1 views

Goodhart's Law, rooted in economist Charles Goodhart's 1975 paper on monetary policy, holds that once a measure becomes a target, optimizing the number and improving what it represents become two separate pursuits. The principle gained its popular phrasing from anthropologist Marilyn Strathern in 1997, though it is frequently misattributed to Goodhart alone. Real-world examples span industries: Volkswagen's emissions software passed lab tests while polluting at up to 40 times legal limits on public roads, and a 2005 survey found 78% of financial executives would sacrifice real economic value to report smoother earnings. Schools, hospitals, and support teams exhibit the same pattern — teaching to the test, parking patients outside emergency doors, or closing tickets without solving problems. The distortion is structural rather than intentional, arising whenever those being measured also influence the conditions of measurement.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Solo Dev Tackles AI Context Gap When Specs Live Across Multiple Repos

A developer building personal projects as separate microservices ran into a recurring problem: AI coding tools like Cursor can only access files within the currently open workspace, making cross-repo spec references difficult. Two broad approaches exist — adding all relevant repos to a multi-root workspace, or having the AI fetch files by path on demand — but each carries trade-offs in noise, token usage, or context bloat. Cursor's April 2026 update improved multi-root workspace support for cross-repo edits, yet the core issue of indexing entire repos just to read a few spec files remains. The developer also evaluated self-hosted documentation tools and external SaaS options, finding them either too rigid, too costly, or ill-suited for a solo workflow. Unable to find a satisfactory off-the-shelf solution, the developer began exploring building a lightweight custom tool to bridge the gap between isolated repos and AI context windows.

0
ProgrammingDEV Community ·

Engineer migrates live IoT smart cane platform to Google Cloud with zero forced app updates

In January 2024, a backend engineer took over WeWALK, a smart cane platform serving 27,000 visually impaired users, and migrated it from a largely inaccessible self-hosted Rancher/Kubernetes cluster to Google Cloud Run, Cloud SQL, and Firestore. The core challenge was ensuring no user was ever forced to update their app, since the cane is a mobility aid and any interruption could leave someone stranded. To preserve existing AWS WebSocket endpoints baked into older client binaries, the engineer built a Lambda-to-Cloud-Run proxy that silently forwarded traffic to the new infrastructure without clients noticing the change. The migration also involved reconciling heterogeneous data stores — MongoDB, PostgreSQL, and Firestore — while handling a continuous stream of 102-byte device diagnostics arriving every ten seconds from canes in the field. The entire cutover was completed with no reported support tickets related to the migration.

0
ProgrammingDEV Community ·

Four-Part Prompt Structure Found to Outperform Vague Prompts on Costly AI Models

Developers shipping AI features have identified a consistent pattern that distinguishes reliable prompts from unreliable ones. The framework requires every prompt to specify a role, a single task, its constraints, and an exact output format. This structured approach proved so effective that a precise prompt on a cheaper model regularly outperformed a vague prompt on a more expensive one. The finding challenges the assumption that model quality alone determines output reliability. The pattern has been validated across use cases including classification, extraction, chain-of-thought reasoning, and structured JSON generation.

0
ProgrammingDEV Community ·

Google's AI Overviews Cut Top-Ranked Site's Organic Traffic by 59%, Data Shows

A website holding the number-one Google ranking for 11 months saw its click-through rate plunge from 27% to 11% after Google rolled out AI Overviews broadly in the US market, resulting in a 59% traffic drop on affected queries. Analysis across multiple clients revealed a consistent pattern: queries where AI Overviews deliver a complete answer saw CTR fall by 60–80%, while queries requiring user action saw only a 10–25% decline. The findings expose a growing disconnect between search rankings and actual site traffic, as ranking tools continue to show top positions while clicks quietly erode. Notably, once AI Overview-affected queries were separated from others, the same client's remaining organic traffic had actually grown 12% in the same period. In response, the SEO team has begun shifting content strategy away from purely informational articles toward interactive tools and resources that give users a concrete reason to click through.