Deterministic Simulation Testing Can Reproduce and Shrink Elusive Agent Bugs

·5 views

A class of hard-to-catch software bugs in AI agents only surfaces when faults occur in a specific sequence — for example, a retry firing after a side effect causes a customer to be charged twice. Deterministic simulation testing (DST) addresses this by routing all sources of nondeterminism, such as faults, timing, and randomness, through a single seed, making any failure perfectly reproducible. The technique, used by systems like FoundationDB and TigerBeetle, also supports shrinking, which strips a complex failing scenario down to its minimal root cause. A Python demonstration showed that standard happy-path tests missed a double-charge bug, while seeded fuzzing caught it, replayed it identically, and reduced a four-fault sequence to the single fault responsible. DST is gaining broader attention as a practical method for testing agents that operate in unpredictable, fault-prone environments.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Google Renames NotebookLM to Gemini Notebook, Adds Thinking Steps and Workspace Sync

Google has rebranded and expanded its notebook research tool as Gemini Notebook, integrating it more deeply with the Gemini app, Google Search, and Google Workspace. A key new feature allows eligible subscribers to view the model's thinking steps during chats and notebook operations, offering greater transparency into how responses are generated. Google has also added in-notebook code execution while retaining existing features like citation-backed responses, Audio Overviews, and Mind Maps. The rollout is phased, with initial access for Google AI Ultra and Workspace AI Ultra Access users, followed by broader availability for Pro subscribers on the web. Google cautions that the visible thinking steps are experimental and should be treated as an inspection aid rather than confirmation of output accuracy.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Measure True LAN Speed Between Devices Using iperf3

iperf3 is an open-source, client-server network testing tool maintained by ESnet that measures actual throughput between two devices on a local network, something internet speed tests cannot do. Unlike services such as Speedtest, iperf3 isolates a specific link — such as phone to NAS or mesh node to access point — by controlling both endpoints directly. One device runs as a server using 'iperf3 -s', while the client connects and pushes traffic for a set duration, reporting sustained throughput, retransmits, jitter, and packet loss. The tool is available via standard package managers on Linux and macOS, and native apps exist for iPhone, iPad, and Android to allow mobile testing across different rooms or locations. Walking through a building while running tests reveals real-world Wi-Fi performance gaps that signal-strength indicators alone cannot expose.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Neural Networks Process Text: Tokenization, Matrix Math, and Deep Learning Explained

Natural language processing models cannot work with raw text directly, so words are first converted into numerical token IDs before being fed into a neural network. Each layer of the network takes an input vector, multiplies it by a matrix of learned weights, adds a bias, and passes the result through a non-linear activation function like ReLU. Stacking many such layers — the basis of deep learning — allows networks to build progressively richer representations of the original input. Recurrent neural networks extend this by combining the current input vector with a running hidden state at every timestep, though the fixed size of that state limits how much information can be retained over long sequences. This matrix-based structure is also why GPUs are central to deep learning, as they are purpose-built to execute large-scale matrix multiplications efficiently.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AbilityGuard v1.1 adds persistent logs, advanced filters, and watchlist alerts

The WordPress plugin AbilityGuard has released version 1.1, significantly expanding its monitoring capabilities for the WordPress Abilities API. The update replaces the previous rolling 100-entry log cap with configurable or unlimited log retention, ensuring audit history is no longer silently lost. Advanced filtering options — covering status, risk level, user ID, date range, and more — now work independently of each other, and users can save filter combinations for reuse. Filtered log views can be exported directly to CSV, making the data easier to use in compliance reports or security reviews. A new sensitive ability watchlist lets users flag specific high-risk abilities for closer attention, shifting the tool from a passive dashboard to an active monitoring system.

0 comments Read more at DEV Community