Media archive reranking: limit chunk versions and candidates before model calls

·1 views

The guide recommends retrieving around 20 candidate chunks and reranking them against the original query, sending only the top few into the prompt. It argues that storage costs, driven by records, chunks per record, retained versions, and retention days, usually matter more than tuning the final result set. Using illustrative figures for a publisher handling 100,000 records daily, it shows that keeping every version for 180 days could raise live chunks from 12 million to 72 million. It recommends tracking four counters and keeping compact decision traces, with full candidate text sampled for diagnostics. It also advises chunking along existing semantic boundaries and applying retention rules before reranking so obsolete revisions do not win on wording alone.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Coding agents copy your worst test files, so cleaning them up matters

A developer argues that the existing tests in a codebase act as a prompt, since coding agents imitate their patterns closely. Simon Willison has observed that agent-written tests tend to match the quality of a project's clean baseline tests. A common flaw he points to is heavily duplicated test setup, which he addresses by asking the agent to refactor with parametrize or shared fixtures. He notes that even a few well-written examples can steer an agent toward a preferred testing style. The author also notes that Anthropic and OpenAI docs recommend keeping instruction files short and letting conventions live in the code itself.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Four Laravel Query Fixes That Make Slow Database Code Much Faster

A Laravel developer explains that slow pages and jobs usually stem from everyday query habits rather than complex issues. Common culprits include wrapping columns in functions that block index use, computing totals in PHP instead of the database, and loading large row sets into memory just to aggregate them. Benchmarks on a one-million-row MariaDB table show dramatic gains, such as sums dropping from 5.8 seconds to 0.15 seconds and grouping falling from 25.8 seconds to 1.1 seconds. The author also recommends wrapping bulk inserts in a single transaction to avoid paying the commit cost on every row. The fixes center on letting the database do the heavy lifting and returning only the results that are actually needed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Estonia's September 2026 Incidents Prompt Call for Repeatable Public-Sector Exposure Baselines

Estonia's Information System Authority (RIA) reported several security failures in September 2026, including DDoS attacks on multiple public-service sectors and over 86 million requests against the national authentication service. A health-data breach at a dental IT processor and an e-bookstore intrusion through an unpatched flaw were also recorded. An analysis argues that organisations should maintain a documented, re-runnable exposure baseline with defined scope, verbatim queries, units, time windows and interpretation limits. The analysis measured Estonian exposure externally in mid-October 2026, counting 263 government-sector assets, 21,335 HTTPS domain assets, 135,740 hosts on port 443 and 98,788 SSH matches. It cautions that visibility does not equal compromise and that counts from overlapping layers should not be summed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

A Ten-Second Formatting Habit That Speeds Up Code Reviews

A developer describes how they now normalize code formatting before reviewing any pull request, after noticing that formatting noise had slowed them down and made their reviews worse. They paste code into a browser-based formatter, run it, and copy the result back, which takes roughly ten to fifteen seconds and leaves a diff showing only real changes. The author applies the same step before submitting their own pull requests, finding that reviewers respond faster and comment more on design. Because they often work on machines that aren't their own, they prefer a no-install web tool over a configured formatter, while still recommending CI linting for repositories they control. The piece argues that removing formatting noise lets reviewers focus attention on logic and design.

0 comments Read more at DEV Community