Live AI Chatbot Audit Reveals Fake Identity, Fabricated Stats, and a 55/100 Score

·1 views

A stress-test audit of a live AI chatbot built on Groq's API, conducted using the BotCritic evaluation tool, exposed multiple serious hallucination failures across five simulated customer personas. The bot falsely claimed a 'Meta AI Research team' reviewed user data — an entirely fabricated response with no basis in the actual system setup. It also invented a name, 'Rohan', when asked its identity, and cited specific ROI statistics attributed to Gartner, Forrester, IBM, Amtrak, and Domino's Pizza — all of which were made up. In a separate test, the bot repeatedly suggested a user contact customer support despite being told their ticket was already closed, looping three times before conceding it was unhelpful. The chatbot ultimately scored 55 out of 100, receiving low marks in Accuracy, Robustness, and Safety/Compliance, raising concerns about the risks of deploying AI agents without rigorous pre-launch testing.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

CAP Theorem Explained: How Distributed Systems Balance Consistency and Availability

CAP theorem, introduced by Eric Brewer in 2000, defines three core properties of distributed systems: Consistency, Availability, and Partition Tolerance. Since network partitions are unavoidable in real-world distributed environments, engineers must choose between prioritizing consistency (CP) or availability (AP). CP systems like ZooKeeper and HBase suit use cases such as financial transactions, while AP systems like Cassandra and DynamoDB are preferred for social media and e-commerce workloads. The BASE model — Basically Available, Soft State, and Eventually Consistent — has emerged as a practical complement to CAP for modern distributed architectures. Choosing the right consistency model depends on business requirements, technology stack, and acceptable operational complexity.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

JetBrains Releases Mellum2, a 12B Open-Source MoE Model for Developer Tasks

JetBrains launched Mellum2 on June 1, 2026, an open-source large language model built on a Mixture-of-Experts architecture with 12 billion total parameters, of which only 2.5 billion are activated per inference. This selective activation makes inference more than twice as fast as comparable full-scale models, substantially lowering deployment costs. Rather than competing with frontier AI models, Mellum2 is designed as a lightweight specialist for high-frequency developer tasks such as code completion, prompt classification, tool selection, context compression, and sub-agent planning validation. The model supports only text and code, intentionally omitting multimodal capabilities to stay lean and suitable for private enterprise deployments handling sensitive code or confidential data. Released under the Apache 2.0 license, model weights are available on HuggingFace and a technical report has been published on arXiv under ID 2605.31268.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer builds AI farm advisory chatbot for Nigerian smallholder farmers using Gemma 4

A developer has built an agentic farm advisory assistant using Google's Gemma 4 model via Google AI Studio's Gemini API, designed to support smallholder farmers and agro-logistics platforms in Nigeria. The chatbot can diagnose crop diseases from uploaded photos, check weather conditions for safe planting or spraying, retrieve live market prices, and log farm activities to farmer records. It supports responses in Nigerian Pidgin and local phrasing, making it accessible to a wider rural audience. All factual data — including weather and market prices — is sourced through real backend function calls rather than the model's own assumptions, reducing the risk of misinformation. The project was prototyped entirely in the browser using AI Studio before being deployed as an Express backend with a chat interface.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Deploy Tracking, Not Smarter AI, Was the Key to Trustworthy Incident Analysis

A development team building an AI-powered incident detection system found that their first version, which lacked system context like deploy tracking and service topology, correctly identified incident causes only 35% of the time. The tool frequently blamed the most recently deployed service regardless of the actual root cause, and within two weeks three engineers had abandoned it entirely. The team identified that effective incident response depends on rapidly connecting relevant information — including deploy events, dependency graphs, and historical postmortems — rather than generating plausible-sounding explanations. After integrating CI/CD deploy data directly into the incident context, engineer agreement with the system's top hypothesis rose from 35% to 70%. The core lesson was that AI summarization should explain what a well-grounded retrieval system surfaces, not substitute for the underlying contextual data.

0 comments Read more at DEV Community