SShortSingh.
Back to feed

Debugging AI failures requires asking 'which layer' broke, not just 'where'

0
·66 views

Software engineer Derek Wang argues that when an AI system fails, the instinct to pinpoint a single broken location is a fundamental debugging trap. He proposes a three-layer analysis framework: L1 (the visible phenomenon), L2 (structural flaws in the code), and L3 (flawed design assumptions underlying the system). A key rule in his approach forbids labelling something a 'data problem' as a root cause, since that label merely restates the symptom without explaining why the data went wrong. Wang also warns that fixing only the identified layer often triggers failures elsewhere, as engineering systems are deeply interconnected. The framework functions as a mandatory gate — no fix is permitted until all three layers have been examined and documented.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Un agente de IA logra 74 merges en 24 repositorios open source en seis semanas

Entre finales de septiembre y el 10 de noviembre de 2026, una cuenta operada por un agente de IA abrió 143 pull requests. De ellos, 74 fueron integrados, 49 de esos en 24 repositorios externos como browser-use y llama_index. Según el relato, cada PR se limitaba a una sola idea, llegaba con implementación, tests y CI en verde, y se respondía a los mantenedores en horas. Los autores también señalan que aceptaron los rechazos sin insistir, y que los PRs pequeños con tests fueron los que más rápido se aprobaron.

0
ProgrammingDEV Community ·

Daily Cognito Audit Job Could Silently Inflate AWS MAU Billing

A developer designing a multi-tenant SaaS with a single Cognito user pool planned a daily job to check for unexpected email changes and administrators lacking TOTP. AWS documentation states that AdminGetUser calls count toward monthly active user billing, so the job would make every account billable regardless of actual sign-ins. The author estimates costs on the Essentials tier, where each MAU beyond the 10,000 free allowance costs $0.015, and notes the cost scales with signed customers rather than usage. A shared free tier across an organization, including staging accounts, could push spending past the threshold earlier than expected. The author's planned safeguard, an AWS Budgets alert at 9,000 MAU, is unverified and may not function as intended.

0
ProgrammingDEV Community ·

Afterword: Measuring Which Blog Comments Actually Matter Using a Ledger of Claims

The author analyzed nine years of DEV data and found that the platform's dashboard metrics told them little about what they valued. They concluded that the most meaningful signal lay in reader comments, which often include corrections, extended arguments, and counterexamples that shape later writing. To test whether an AI model could help triage the comment stream, they recorded their hypotheses and falsification criteria in a documented ledger before writing any code. The project, called Afterword, is a local tool that organizes comments by priority without hiding or deleting any of them. Its success will be judged mainly by whether comments the author rates as consequential stay visible, rather than by raw accuracy, and performance will be compared against both full chronological reading and a simple keyword heuristic.

0
ProgrammingDEV Community ·

Bitcoin and 24/7 Stock Perpetuals Move Together in US Hours, Study Finds

Binance now lists perpetual futures on US stocks, gold, silver, and crude oil that trade around the clock, allowing minute-by-minute comparisons with Bitcoin. A study of 4 and 5 November showed Bitcoin rising while stocks were flat, then both falling, with stocks dropping far more on the second day. Over 139 trading days from April to November 2026, Bitcoin and the stock basket repriced together within the same minute during regular US hours, with neither clearly leading. When US markets were closed, the stock perpetuals followed Bitcoin by about a minute, though this relationship was weak. Daily correlation rose from near zero in late August to 0.84 recently, and 5 November was the largest divergence in the sample.