SShortSingh.
Back to feed

Adding LLM Descriptions to 1,245 Database Tables Made Search Retrieval Worse

0
·51 views

A developer catalogued a 1,245-object database schema by generating LLM descriptions for every table, expecting improved search retrieval, but instead saw recall drop significantly. The problem stemmed from BM25 scoring mechanics: when a term like 'contact' appears in over 1,000 of 1,245 documents, its inverse document frequency collapses to near zero, stripping it of ranking power. Compounding this, BM25's length normalisation penalised the most important tables — which naturally have the most columns — pushing them further down rankings. Attempts to fix this by down-weighting prose fields helped marginally but sacrificed retrieval of rare, valuable terms like obscure view names. The author's solution was to maintain three separate BM25 indexes — one for identifiers, one for prose descriptions, and one combined — then fuse their rankings using reciprocal rank fusion.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Zoom Windows clients flagged for high-severity certificate validation flaw, CVE-2026-108184

Zoom disclosed CVE-2026-108184 on October 13, 2026, as bulletin ZSB-26021, describing improper certificate validation in certain Windows clients. The issue carries a CVSS 3.1 score of 7.4 and can be triggered over the network without privileges or user interaction, though attack complexity is high. Zoom says the flaw may allow unauthenticated information disclosure, and the bulletin rates confidentiality and integrity impact as high without detailing the exact attack path. The affected products include Zoom Workplace for Windows, Zoom Workplace VDI Client, and Zoom Rooms for Windows across listed version ranges. The article cautions that a ZoomEye count of 347,617 Zoom fingerprints does not indicate how many clients are vulnerable, and recommends authenticated endpoint inventory to decide which systems need patching.

0
ProgrammingDEV Community ·

A Method for Reading Code You Didn't Write: Run, Enter, Follow, Break

Reading unfamiliar code is often approached by opening files at random, which wastes days and rarely builds real understanding. The article argues that reading is a learnable skill that matters more than writing, since most engineering work involves understanding existing code. It proposes a four-step method: run the program first, locate its entry point, trace a single execution path, and deliberately break something to see what fails. A fifth habit, consulting the project's history, is also recommended. The approach is illustrated with serverctl, a Go terminal UI for administering Linux servers over SSH, whose 107 Go files make it a useful example.

0
ProgrammingDEV Community ·

Repetitive Tasks, Not Low Productivity, May Be the Real Drag on Teams

The article argues that many teams struggle less with overall productivity than with repetitive, rule-based work. Tasks such as moving data between tools, sending routine follow-ups, compiling reports, and triaging requests consume significant hours without needing human judgment. The author, an automation developer, points to n8n, an open-source, low-code workflow platform, as one way to address this. Highlighted advantages include integrations with hundreds of apps, self-hosting for data control, and built-in support for AI models and agents. The piece suggests that the real benefit of automation is freeing staff for strategy, creativity, and client relationships.

0
ProgrammingDEV Community ·

Java String.format() Behaves Differently Under German Locale Settings

A developer found that String.format("%.2f", 0.5) produces "0,50" on a laptop with German regional settings, which then causes Double.parseDouble to throw a NumberFormatException. The article explains that string formatting in Java builds text from templates whose format specifiers such as %d, %s and %.2f are replaced by argument values. It compares the methods that produce this output, including String.format(), String.formatted() added in Java 15, System.out.printf() and the Formatter class. It also notes that all of these share the same template syntax and that locale settings such as German or French change the decimal separator from a point to a comma.