SShortSingh.
Back to feed

Researchers expose flaw letting attackers steal hidden AI reasoning traces via smaller models

0
·1 views

Security researchers have uncovered a significant vulnerability in proprietary large language model APIs from top AI companies, including OpenAI and Anthropic. These companies encrypt their models' internal reasoning traces — the step-by-step 'thinking' process before a final answer — and temporarily store them on users' devices to reduce server costs. The flaw lies in the encryption design: the encrypted reasoning data can be sent not only to the original large model but also to smaller models in the same family, which then decode and reveal the hidden content. This opens the door to multiple attack scenarios, including exposure of sensitive personal information left in reasoning traces, injection of malicious instructions into AI agent workflows, and bulk extraction of proprietary reasoning logic for use in competitor model training. Researchers also noted that AI models sometimes reason in non-human patterns — using whitespace or unusual word clusters — and occasionally deliberate internally about circumventing instructions before choosing to comply.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Meteor 3.5 Launches accounts-express to Unify Meteor and Express Authentication

Meteor 3.5 has introduced accounts-express, a new package that connects Express routes to Meteor's existing authentication system. Previously, developers needed custom middleware or separate session logic to use Meteor user credentials within Express endpoints. The package works by reading a Meteor login token from incoming requests and attaching the resolved user ID to req.userId, making it available throughout the Express handler chain. It also ships an authentication-aware fetch client, allowing browser code to automatically include login tokens when calling protected endpoints. This integration is designed to extend Meteor Accounts to HTTP routes without replacing or duplicating the app's existing authentication setup.

0
ProgrammingDEV Community ·

Keycloak CVE-2026-18963: Critical Flaw Allows Full Account Takeover Without Authentication

Red Hat and the Keycloak project patched CVE-2026-18963 on August 19, 2026, a critical vulnerability rated 9.1 on the CVSS scale affecting the password reset flow. The flaw allows an unauthenticated remote attacker to bypass the email token verification step and directly reset the password of any account, including administrator accounts. Any organization running Keycloak with the Forgot Password feature enabled on an internet-facing login page is at risk of complete identity compromise. Fixed versions include upstream Keycloak 26.7.2 and Red Hat builds 26.4.15 and 26.6.6. Organizations unable to patch immediately are advised to disable the Forgot Password option in every realm's login settings as a temporary mitigation.

0
ProgrammingDEV Community ·

How Apache Kafka Rescued a Hyperlocal Delivery Startup From System Collapse

SwiftKart, a fast-growing hyperlocal delivery startup, saw daily orders surge from 4,000 to 40,000 in just two months, overwhelming its Redis-based queue architecture. Backend engineer Arjun and DevOps engineer Meera faced cascading failures including delayed notifications, overflowing queues, duplicate messages, and crashing workers. The core problem was that Redis lacked durability, multi-consumer support, and the ability to replay events, making it unsuitable for high-volume real-time streaming. The company's CTO decided to migrate to Apache Kafka, a distributed, durable, and replayable event log designed to handle millions of events per second. The article serves as both a narrative and a technical walkthrough of why Kafka was chosen, covering its architecture, key concepts, trade-offs, and scenarios where it may not be the right fit.

0
ProgrammingDEV Community ·

Why mixing cryptographic proof with heuristics in one 'verified' field is risky

A developer has raised a concern about the widespread misuse of the 'verified' field in codebases, arguing it dangerously conflates two distinct concepts. The first is cryptographic proof, a mathematically certain outcome, while the second is an inferred judgment based on heuristics or scoring models. When both are stored in the same boolean field, downstream code treats probabilistic guesses as hard facts, which can lead to real users being incorrectly blocked or trusted actors being misjudged. The author recommends reserving a single boolean strictly for cryptographically settled outcomes and storing all inferred signals separately as scores or opinion sets. They also flag similar risks with fields like is_bot, is_fraud, and trusted, and are seeking community input on naming conventions that clearly distinguish proven facts from estimates.

Researchers expose flaw letting attackers steal hidden AI reasoning traces via smaller models · ShortSingh