Study Finds Citation Checks Miss Fabricated Claims in Automated Research Pipeline

·1 views

A researcher built an automated research agent whose claims are checked against their cited sources, and it scored perfectly on its own benchmark. The author then tested it with poisoned sources, planted instructions, swapped statistics, and questions the sources could not answer. Nearly all source-borne attacks succeeded, and citation accuracy stayed at 100% even for fabricated claims. The critic only verifies that a claim faithfully quotes its source, not whether the source itself is reliable. The work argues that structural citation alone does not guarantee trustworthy research.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

AI Models Map San Francisco From Memory, With Mixed Accuracy

Researchers tested six AI chat models from Anthropic, OpenAI and Google by asking what lies at 41,526 coordinates across San Francisco. The models answered without maps or internet access, and each answer was checked against OpenStreetMap. The best models, run at high reasoning settings, labeled the city's water and park areas correctly about 86% of the time. Their results were close to one another and beat a baseline of 61% from simply guessing neither water nor park. Finer-grained tests in Crissy Field, the Marina and the Palace of Fine Arts showed the models could partly distinguish dense street grids from parkland, though small parks were often misidentified.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Google Maps closure data often lags reality, test of 400 US places finds

A study checked 400 random US business listings from Overture Maps against the Google Maps status for each place. An automated enrichment tool marked 93 as closed, and the researchers examined those cases by checking public listings and independent sources. After verification, 86 of the 93 flagged closures were confirmed, while 3 were wrong and 4 remained unresolved. Roughly a quarter of the confirmed closures, 23 of 86, still appeared as open or were missing on Google Maps. The authors caution that the test only covered the closures they flagged and is not a complete measure of accuracy.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Shared Git-Backed Memory Helps AI Coding Agents Avoid Rejected Approaches

A developer describes how scattered Markdown notes caused Claude Code, OpenAI Codex, and Google Antigravity agents to revisit approaches the team had already abandoned. He built KYB, an open-source shared store backed by Git, which keeps versioned records with lifecycle states such as active, archived, and retracted. The system lets users query current state, history, or a specific time slice, and allows wrong information to be replaced rather than appended. The author notes his experience is anecdotal, with no controlled benchmarks, and the mechanisms are pinned to KYB version 0.3.3.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Five Tools for Testing Cross-Zone Behaviour Locally, From Kernel netem to Chaos Mesh

A DEV Community article compares tools for emulating cross-zone network conditions on a developer's machine, grouping them by the layer they act on. Kernel-level tc netem shapes packets on a device, while Toxiproxy operates as a TCP proxy whose toxics are configured through an HTTP API. Pumba applies network effects to containers from outside their namespace, and Kubernetes-focused tools such as Chaos Mesh select pods by labels. The author notes the trade-off that lower layers offer more control but know less about the system, whereas higher layers need more installed. The piece is part five of a series and stresses that the author's own failure-scenario runs are still planned.

0 comments Read more at DEV Community