Developer builds open-source tool to verify if AI answers match retrieved evidence

·1 views

A developer has created MEVA (Medical Evidence Verification Agent), an open-source project designed to test whether AI-generated answers are actually grounded in the evidence the AI retrieves. The tool works with synthetic FHIR medical records, allowing an AI agent to fetch evidence via MCP tools before a deterministic Python verifier — not another LLM — checks each claim. MEVA classifies results into four categories: Supported, Contradicted, Unsupported, or Unverifiable. A public Streamlit sandbox lets users test the verification process using synthetic Synthea patient data, with no real patient data involved and no diagnostic intent. The project is in early stages, and the developer is seeking feedback from those interested in AI evaluation, FHIR, local LLMs, or Python.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer spent 11 days optimizing search rankings invisible to all users but himself

A developer publishing automation tools on the Apify marketplace spent nearly two weeks investigating why his 23 tools had only one user despite strong search rankings. He conducted extensive demand, naming, and age-cohort analyses, renaming tools and improving quality scores, but nothing changed user numbers. The root cause turned out to be that his search ranking measurements were always taken while logged in as the tool author, masking a critical visibility issue. An anonymous API query on August 13 revealed his tools were entirely absent from public search results due to an unverified identity status on the platform. His tools were technically live and well-ranked, but excluded from the index served to all logged-out visitors until KYC verification was completed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

One-Line Encoding Bug Silently Disabled Python Security Hook on Non-ASCII Paths

A developer running Python-based security hooks to guard an AI coding agent discovered that one hook had been silently failing for an unknown period of time. The bug caused the hook to allow file reads it was supposed to block whenever the file path contained non-ASCII characters, such as Japanese folder names. On Windows, Python defaults to reading stdin using the system locale encoding (cp932), which silently corrupted the incoming UTF-8 path string instead of raising an error. The corrupted path then failed an existence check, triggering a fail-open safety valve that exited with code 0, indistinguishable from a normal allow decision. Fixing the issue required just one line — explicitly opening stdin with UTF-8 encoding — after which the hook correctly blocked reads across both ASCII and non-ASCII paths.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How UI Overdraw Quietly Degrades WebGL Game Performance

UI overdraw occurs when the GPU renders the same screen pixel multiple times within a single frame, a problem that becomes significant in WebGL games with layered interfaces. Elements like full-screen backgrounds, transparent overlays, panels, buttons, and decorative images may appear simple visually but can stack up to process millions of pixels repeatedly. WebGL applications face additional performance constraints because they run inside a browser, making unnecessary GPU workload more costly than in native desktop apps. Unity offers an Overdraw visualization tool and a UI Profiler to help developers identify which interface elements are contributing most to fill-rate bottlenecks. Developers are advised to profile on the target platform before optimizing, paying particular attention to large transparent elements that cover substantial portions of the screen.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Minisforum MS-R1 Ships With Vhost Disabled, Blocking Android VM Support

A developer attempting to run multiple native ARM64 Android virtual machines simultaneously on the Minisforum MS-R1 desktop discovered that its stock kernel lacks vhost subsystem support. The MS-R1 is built on the CIX P1 SoC and ships with kernel version 6.6.10-cix-build-generic, which has the entire vhost subsystem disabled rather than just specific modules. This omission prevents Google's Cuttlefish virtual device platform from functioning, as it relies on vsock for all host-to-guest communication. The vendor's design choice appears intentional, as their documented Android support path uses containerized Android via Redroid, which shares the host kernel and does not require vhost. The developer rebuilt the kernel to restore standard virtualization support, documenting four distinct obstacles encountered along the way.

0 comments Read more at DEV Community