SShortSingh.
Back to feed

Structured Outputs Fix JSON Format, Not AI Hallucinations, Developer Finds

0
·1 views

A developer building ShapeCraft, an open-source structured output library for OpenAI, Groq, Ollama, and Anthropic, found that JSON Schema validation and factual accuracy are two entirely separate guarantees. While structured outputs reliably ensure valid JSON, correct data types, and required fields, they cannot verify whether the extracted information actually matches the source document. The developer observed that different AI providers implement structured outputs in distinct ways — ranging from native server-side enforcement to best-effort prompt-and-retry approaches — which affects architectural decisions. ShapeCraft exposes a 'guaranteeLevel' field so developers know how confidently a response matches the schema structurally. The developer argues that the next frontier in AI tooling must address the harder problem of information trustworthiness, including source attribution, confidence scores, and semantic verification.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Developer finds deterministic rules outperform LLM in log classification after rigorous eval

A software developer built an evaluation harness to test whether a large language model could reliably classify integration failures from a legacy system generating over 4,000 parsed log errors. Testing across 58 hand-reviewed cases showed rule-based classification scored 89.7% accuracy versus the LLM's 87.9%, and three rounds of prompt engineering produced no improvement. The developer also found that regex matched the LLM's structured data extraction at 90–97% agreement, disproving an earlier assumption that extraction was a natural fit for AI. The final architecture uses deterministic rules for classification and structured extraction, reserving the LLM solely for writing plain-English incident summaries, where it scored 4.62 out of 5 for faithfulness. The entire pipeline runs locally via Ollama at zero API cost, processing each case in roughly 8–10 seconds.

0
ProgrammingDEV Community ·

Developer Fixes Null Check Bug in Hermes Agent That Crashed Permission System

A full-stack developer from Kolkata, Aniruddha Adak, identified and fixed a critical bug in NousResearch's open-source hermes-agent framework. The flaw resided in the permission bridge, where the function request_permission could silently return None when an ACP client sent an empty response, causing an AttributeError crash instead of a safe denial. Because the existing code accessed result.decision without first checking for None, the agent would crash mid-session rather than defaulting to a secure deny state. Adak reproduced the issue locally, then submitted a minimal patch adding a guard clause that returns 'deny' immediately whenever the response is None. The fix, merged as PR #13457, follows a standard fail-safe security pattern and includes a dedicated unit test to prevent regression.

0
ProgrammingDEV Community ·

Developer with Zig background shares key lessons from 1.5 months learning C

A developer with two years of Zig experience began learning C about six weeks ago as a requirement for an upcoming job, using a PNG decompression project as a practical exercise. Despite Zig's heavy inspiration from C, the transition proved harder than expected due to differences in standard library structure, keywords, and header file organization. The developer found the C community on Reddit to be helpful rather than hostile, with experienced programmers pointing out errors and suggesting improvements. Compiler flags such as -Wall, -Werror, and -fsanitize=address,undefined proved especially valuable in catching memory leaks and out-of-bounds errors quickly. After the experience, the developer acknowledges still having much to learn, particularly around idiomatic pointer usage, but notes that C turned out to be far more distinct from Zig than initially assumed.

Structured Outputs Fix JSON Format, Not AI Hallucinations, Developer Finds · ShortSingh