SShortSingh.
Back to feed

AI Models Silently Drop Critical Data in Up to 22% of Tasks, Study Finds

0
·21 views

A new study tested four AI models — two public open-source models and two compressed versions — on their ability to accurately handle 40 common business identifiers such as tax IDs, card numbers, and patient records. Each model was asked to copy, extract, and summarize these identifiers across 120 checks per model, repeated eight times to establish reliable error rates. One model silently omitted or withheld data in 21.7% of cases, while another failed 3.3% of the time, revealing a sixfold difference in silent data loss between models. Crucially, the failures produced no error messages or refusals — outputs appeared complete and well-formed, meaning standard automated checks would not detect the missing information. The researchers argue that silent data omission should be a primary criterion when selecting AI models for regulated industries like banking, healthcare, and insurance, and that the testing methodology is simple enough for any organization to replicate.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

H-1B holders may enter up to 10 days early but cannot begin work before the petition start date

Under U.S. immigration regulations, H-1B workers and their H-4 dependents can be admitted up to 10 days before their approved petition start date. The same window extends up to 10 days after the end date, and USCIS can grant the status inside the country on similar terms. Employment, however, is permitted only during the petition's validity period unless another work authorization applies. The rule, found at 8 CFR 214.1(l)(1), was set by DHS in a 2016 final rule that took effect in January 2017. The extra days are meant to allow time to enter, prepare for employment, or depart and pursue other lawful status options.

0
ProgrammingDEV Community ·

Author stress-tests a containerized task system and explains why hard isolation was left unbuilt

The final post in a series on agent execution infrastructure assembles earlier fixes and runs a load test of 20 concurrent one-minute tasks against a small container pool. The test confirmed that no tasks were lost, surplus work queued, users received queue positions instead of errors, and progress stayed queryable. The author stresses that this load level is too small to reveal real failure modes, and that failure-injection experiments need a multi-node cluster the local laptop setup cannot provide. He also admits that tasks held by a dead container are not automatically re-dispatched, which would require a task-state table and timeout rules. The post closes by noting that stronger isolation, such as the shared kernel's role in container boundaries, was designed on paper but not implemented.

0
ProgrammingDEV Community ·

Experts urge AI agent risk registers covering seven areas beyond prompt injection

A DEV Community article argues that treating prompt injection as the sole AI security concern leaves most risks unaddressed, since roughly 60 risks span data, models, software security, agent actions, human operations, governance, and societal harm. It notes that falling costs of AI-assisted vulnerability discovery, illustrated by recent PHP flaws found with reasoning models, weaken the defense that attackers won't bother. The author uses a hypothetical consultant who granted a summarization assistant deletion and sharing rights it did not need, showing how excessive permissions enlarge the damage an injected instruction can cause. The piece draws on OWASP, NIST, the EU AI Act, and MIT's AI Risk Repository to offer a seven-group map as a checklist for building a risk register. It stresses limiting agent permissions to what each task requires, since the blast radius of an attack depends on granted authority rather than the prompt alone.

0
ProgrammingDEV Community ·

Generating Sitemaps, Canonicals and OG Images Dynamically in Nuxt

The author argues that many SEO issues on Nuxt projects come from infrastructure rather than content, such as build-time sitemaps that go stale, canonical tags that multiply across query-string variants, and OG image URLs that return HTML 404 pages. The recommended approach is to produce all three from the backend's database at request time, with caching. The guide describes serving sitemap.xml from a Nitro server route tied to an API, setting a single normalised canonical per page with useHead, and serving OG images from a cached endpoint that returns image/png. The author supports this with curl output from GitHub and Nuxt's own documentation site, noting that correct status codes, content types and explicit cache policies are the key signals to copy. A comparison table weighs static files, build-time generation, Nuxt SEO modules and custom Nitro routes by freshness, scalability and operational effort.