Judged passage labels, not clicks, should gate RAG retrieval releases

·7 views

The author argues that a RAG retrieval evaluator for a documentation-monitoring alert system should rely on judged relevance labels rather than click proxies as its release gate. Recall@K is computed against a versioned set of question-to-passage labels, and builds fail when agreed slices regress, while clicks serve as a separate online signal. Averaging across all queries can hide complete misses on high-risk cases such as removed configuration constraints, so per-slice reporting by change type, template, language, and freshness is recommended. Labels should be defined at the same granularity the retriever returns, and the corpus should be frozen during evaluation. The approach does not by itself measure ranking quality, citation correctness, or answer faithfulness.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Coding agents copy your worst test files, so cleaning them up matters

A developer argues that the existing tests in a codebase act as a prompt, since coding agents imitate their patterns closely. Simon Willison has observed that agent-written tests tend to match the quality of a project's clean baseline tests. A common flaw he points to is heavily duplicated test setup, which he addresses by asking the agent to refactor with parametrize or shared fixtures. He notes that even a few well-written examples can steer an agent toward a preferred testing style. The author also notes that Anthropic and OpenAI docs recommend keeping instruction files short and letting conventions live in the code itself.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Four Laravel Query Fixes That Make Slow Database Code Much Faster

A Laravel developer explains that slow pages and jobs usually stem from everyday query habits rather than complex issues. Common culprits include wrapping columns in functions that block index use, computing totals in PHP instead of the database, and loading large row sets into memory just to aggregate them. Benchmarks on a one-million-row MariaDB table show dramatic gains, such as sums dropping from 5.8 seconds to 0.15 seconds and grouping falling from 25.8 seconds to 1.1 seconds. The author also recommends wrapping bulk inserts in a single transaction to avoid paying the commit cost on every row. The fixes center on letting the database do the heavy lifting and returning only the results that are actually needed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Estonia's September 2026 Incidents Prompt Call for Repeatable Public-Sector Exposure Baselines

Estonia's Information System Authority (RIA) reported several security failures in September 2026, including DDoS attacks on multiple public-service sectors and over 86 million requests against the national authentication service. A health-data breach at a dental IT processor and an e-bookstore intrusion through an unpatched flaw were also recorded. An analysis argues that organisations should maintain a documented, re-runnable exposure baseline with defined scope, verbatim queries, units, time windows and interpretation limits. The analysis measured Estonian exposure externally in mid-October 2026, counting 263 government-sector assets, 21,335 HTTPS domain assets, 135,740 hosts on port 443 and 98,788 SSH matches. It cautions that visibility does not equal compromise and that counts from overlapping layers should not be summed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

A Ten-Second Formatting Habit That Speeds Up Code Reviews

A developer describes how they now normalize code formatting before reviewing any pull request, after noticing that formatting noise had slowed them down and made their reviews worse. They paste code into a browser-based formatter, run it, and copy the result back, which takes roughly ten to fifteen seconds and leaves a diff showing only real changes. The author applies the same step before submitting their own pull requests, finding that reviewers respond faster and comment more on design. Because they often work on machines that aren't their own, they prefer a no-install web tool over a configured formatter, while still recommending CI linting for repositories they control. The piece argues that removing formatting noise lets reviewers focus attention on logic and design.

0 comments Read more at DEV Community