Article URL: https://www.hackingleadership.com/p/why-the-people-who-could-fix-it-are-the-first-to-leave Comments URL: https://news.ycombinator.com/item?id=48951147 Points: 8 # Comments: 0
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
A developer built a multi-agent code review pipeline using Claude and Gemini as parallel adversarial reviewers on GitHub Actions, tracking results across 95 pull requests over six weeks on a real multi-tenant codebase. A key discovery was that Gemini had inadvertently configured its own GitHub Action to only read diffs, not the full repository — a limitation it set up itself. Once given full repository access, Gemini's reviews became less reliable, shifting from honest uncertainty to confidently fabricating file names, functions, and API calls that did not exist. Over the trial, both reviewers together caught 200 real issues with zero escapes and zero false consensus when they agreed, but their findings overlapped only about 12 percent of the time. Claude's findings were accurate roughly 81 percent of the time, while Gemini's accuracy stood at approximately 45 percent, even after excluding the weeks it was reviewing without repository access.
Most engineering teams in 2026 are deploying AI agents with heavy, continuous human oversight, requiring more attention than the manual processes they were meant to replace. A growing argument in the developer community holds that this approach reflects a fundamental design failure, not responsible governance. The proposed alternative is an exception-based monitoring model, where teams define measurable outcomes, constraints, and escalation thresholds upfront, then only intervene when those conditions are breached — similar to how CI/CD pipelines and observability platforms already operate. The core shift is to treat AI agents as process executors whose outcomes need verifying, rather than autonomous decision-makers whose every step needs approval. Critics of this model acknowledge that the real challenge lies in the upfront work of defining precise, automatable success criteria — an investment most teams currently avoid by defaulting to dense oversight instead.
A developer created QuizPaste after repeatedly finding that popular AI quiz generators produced questions based on facts not present in the source material he provided. Unlike conventional tools that simply prompt a language model to generate questions, QuizPaste requires the model to cite the specific sentence from the user's text that each question is based on. The system then programmatically checks whether that sentence actually exists in the original input, discarding any question it cannot verify before the user ever sees it. Users can paste notes or YouTube transcripts and receive a quiz with flashcards in roughly five seconds, with no account required for the first two daily sessions. The tool, launched about four days before the article was written, supports export to Anki-compatible TSV and CSV formats and is free to try at quizpaste.com.
Browser tests can show green results while masking real failures caused by environment mismatches, outdated mocks, or undocumented feature flags. A test may pass locally but fail in staging simply because a feature flag altered the component tree or UI flow without changing the URL. Experts recommend logging the exact configuration state — including active flags, service versions, and seed data — alongside every test run rather than relying on generic environment labels. Adding a lightweight environment verification phase before the main test suite can catch setup issues early and prevent teams from wasting time debugging phantom failures. The core problem is that reliable browser testing demands proof that the environment, application state, and execution path match what the team actually intends to test.
Discussion (0)
Log in to join the discussion and vote.
Log in