SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Researchers Debate Safety Risks in Astra's Recurrent AI Architecture

0
·1 views

A discussion on LessWrong has raised questions about potential concerns surrounding the recurrent architecture used in Astra, an AI system. The post examines whether the design of this architecture warrants significant safety or alignment scrutiny. The topic has drawn attention from the AI safety community, prompting debate about the implications of recurrent models. The conversation reflects broader ongoing concerns about evaluating risks in advanced AI system designs.

Read the full story at Hacker News

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Developer Rewrites Flutter Linux App as C++ Daemon After Three Silent System Bugs

A developer building a Linux screen-capture tool in Flutter encountered three distinct silent failures involving the tray icon, global hotkey registration, and window compositor rendering that produced no errors or logs. The tray menu stopped responding after the first click due to an interaction between GNOME 46's ubuntu-appindicators extension and the older appindicator activation pattern used by Flutter plugins. A separate Flutter hotkey plugin silently reported success even when the underlying keybinder_bind call failed, leaving hotkeys unregistered with no exception to catch. A third issue caused hidden Flutter windows to reappear with frozen, stale frame content under certain show/hide timing with the Mutter compositor. To resolve all three, the developer split the app into a long-running C++ daemon handling tray, hotkeys, and IPC natively, and an on-demand Flutter UI launched only when a window is needed, communicating over a Unix domain socket.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer builds 12-state tracking system to hold AI agents accountable for promises

A developer has built COGEXT, an accountability layer designed to track commitments made by AI agents through 12 distinct states, from detection to fulfillment or failure. The system addresses a core weakness in current AI agents: they generate intent readily but have no mechanism to track whether those commitments are ever fulfilled. Every extracted commitment is stored in PostgreSQL and transitions between states via a single atomic function, ensuring the audit log always matches the recorded status. A two-stage pipeline first classifies agent output to distinguish genuine commitments from suggestions or hypotheticals, then uses Meta's Llama 3.3-70B model to extract structured fields like action, recipient, and deadline. Fulfillment is verified through a weighted field-coverage model rather than simple keyword matching, requiring evidence to cross a confidence threshold before a commitment is marked complete.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Why AI Prompt Filters Fail and How Execution-Path Enforcement Fixes It

Security researcher Simon Willison identified a 'lethal trifecta' that makes coding agents inherently risky: simultaneous access to private data, untrusted content, and outbound capabilities like network calls. Most AI guardrails attempt to intercept threats by scanning prompts or model outputs, but these checks are merely advisory and can be bypassed by cleverly crafted injected text. A tool called Doberman takes a different approach by routing every tool call — shell commands, file writes, and API calls — through a single mandatory decision engine before execution, ensuring no action can bypass the checkpoint. The system operates on two core principles: it fails closed, meaning any uncertainty results in a denial rather than a default allow, and it is raise-only, meaning its policies can only be tightened automatically but require human authentication to loosen. Additionally, a 'taint floor' mechanism tracks sensitive data accessed during a session and automatically blocks any subsequent attempt to transmit that data outbound, directly countering the data-exfiltration risk.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer builds open-source bridge to revive legacy Diebold thermal printers on modern POS systems

A developer has created an open-source translation layer called diebold-bridge that allows legacy Diebold Procomp IM453H thermal printers — which use a proprietary command set incompatible with the industry-standard ESC/POS protocol — to work seamlessly with modern point-of-sale systems. The core problem was that most POS software sends ESC/POS commands, which the IM453H firmware does not understand, resulting in garbled output and no QR code printing. Rather than translating commands one-by-one, the solution interprets incoming ESC/POS or PDF data and re-renders the entire receipt as a 576-pixel-wide bitmap image, which is then sent to the printer using its native raster commands. The bridge accepts input via a Windows print queue, a local TCP socket on port 9100, or a watched folder, requiring no changes to existing POS software. Existing tools such as ACBr and Diebold's own SDK addressed parts of the problem but none provided a complete, install-and-run translation bridge for raw ESC/POS streams.

0 comments Read more at DEV Community
Researchers Debate Safety Risks in Astra's Recurrent AI Architecture · ShortSingh