Support FAQ bot invented 22 answers until grounding cut that to zero

·3 views

A tester evaluated a small n8n support-FAQ workflow using a 1.8B-parameter model running locally, asking 20 questions across two prompt versions over 120 runs. The model answered all 30 known questions correctly but gave answers to 29 of 30 unanswerable questions, and 22 of those asserted facts absent from the knowledge base, such as denying a phone number or Slack integration. Prompt wording alone did not stop the fabrication, so the author required the model to cite a knowledge-base entry and a supporting quote. A Code node then verifies that evidence and falls back to a handover message when it is missing. The approach aims to reduce fabricated answers that could mislead customers into not buying.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Calibration-free top-k selector speeds MoE decoding without threshold tuning

A new top-k channel selector for Mixture-of-Experts models removes the need for offline score-threshold calibration while keeping accuracy and latency gains. Previously, thresholds fitted on held-out data were used to limit routed-expert reads, but this varied across inputs and could trigger repetition loops in models such as Gemma-4. The approach instead keeps each token's k highest-scoring channels, requiring no calibration data. In reported comparisons it matched or beat the threshold version in 40 of 43 cases at 13% and 18% budgets. The authors report retaining 98.1% of full-model accuracy at 17.8% of routed-expert reads on three post-trained MoE models, with Triton kernels running MoE blocks 1.60 to 1.90 times faster than optimized vLLM at batch size one. The evaluation is limited to those three checkpoints and Triton kernels, so generalization to other model families and hardware remains unconfirmed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Two Models for AI-Powered BI: Dashboards with Agents vs. Agent-First Design

The author argues that the right AI-driven business intelligence design depends on data scope, user count, and how predictable questions are. For a narrow procurement project called SpendCube, built on Azure Databricks, Azure SQL Database, and Power BI, a dashboard paired with an AI agent is presented as practical. Power BI Copilot is cited as an example, with the key requirement being that the agent use the existing semantic model so its answers match dashboard KPIs. In contrast, BMW's Mobile Data Recorder collects over 10,000 signals from roughly 3,500 vehicles, making it impossible to anticipate every engineering question in advance. For such large, unpredictable datasets, the author favors an agent-first approach in which engineers describe what they want to investigate and the agent generates queries and visualizations.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Confluence Cloud Adds Native Page Approvals, With Limits on Multi-Approver Rules

Atlassian opened native page approvals in Confluence Cloud to open beta for Premium and Enterprise plans on 10 July 2026, ending years of users being told no approval workflow existed. In the author's test on a Cloud site, a page stayed In review until every named approver signed off, showing the all-or-nothing rule. The native feature cannot use groups as approvers or offer any-one or minimum-N thresholds, and Standard plans only have editable content statuses plus automation. The article offers a Standard-plan workaround using content status and automation, and a third-party app, Sentinel Vault, for more flexible multi-approver rules.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

S3 Select Returns MethodNotAllowed for Some New Accounts; Here's Why and What to Use

A developer found that querying a small CSV file in Amazon S3 with S3 Select failed with a MethodNotAllowed error, both in the console and through the AWS CLI. The uploaded object itself was accessible and encrypted with SSE-S3, which the API supports, so the problem did not seem to stem from the file. AWS announced in July 2024 that it had closed S3 Select to new customers, while existing eligible customers could keep using it. The error is consistent with that restriction, though AWS Support had not confirmed the test account's eligibility. The article also covers alternatives to consider for new projects in 2026.

0 comments Read more at DEV Community