How to Stress-Test an AI Coding Agent's Memory With a Simple Migration Scenario

·1 views

A structured evaluation method for AI coding agent memory involves feeding the agent conflicting project instructions across separate sessions to see how it handles updates. The test uses two fictional projects, Atlas and Beacon, with Atlas receiving a command change from 'npm test' to 'pnpm test' on a specific date, while Beacon's command stays the same. Evaluators then probe the agent across fresh sessions to check whether it returns the current command, recalls historical state, distinguishes between projects, and correctly flags missing information. When an agent fails a question, the recommended approach is to isolate the failure to one of three stages: the write path, the retrieval layer, or the final prompt context sent to the model. The exercise is designed to be storage-agnostic, applicable to memory files, relational tables, vector stores, or knowledge graphs.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Gitea Emerges as Top Choice for Self-Hosted Git Server Over GitLab and Gogs

Developers looking to move away from cloud-based git providers can self-host a git server using one of several free options, including Bonobo, Gogs, GitLab, and Gitea. Bonobo is limited to Windows only, while GitLab is feature-rich but resource-intensive as a commercial product with a free tier. Gogs is lightweight but lacks a built-in container registry, making it less suitable for complete development workflows. Gitea, deployable via Docker Compose with a PostgreSQL backend, runs on approximately 260MB RAM at baseline and around 420MB after hosting over ten repositories. It also supports both push and pull repository mirroring and offers built-in HTTPS configuration, making it a well-rounded option for on-premises self-hosting.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Open-Source Tool 'Ghost Browser' Gives AI Agents Real Browser Control to Act Online

A developer has open-sourced a tool called Ghost Browser that addresses a core limitation of AI agents: the inability to perform real, logged-in actions on websites. While much debate around AI agents focuses on reasoning and memory, the author argues the bigger gap is execution — an LLM can decide to post or message someone but has no native way to do it. Ghost Browser solves this by running a real Chromium instance that uses a visual technique called Set-of-Mark, which overlays numbered boxes on interactive page elements and lets the agent act by referencing those numbers rather than fragile CSS selectors. Sessions are established once by the user and stored in isolated profiles, so the agent operates as a genuinely logged-in user without repeated authentication. Consequential actions like posting or following require explicit human approval by default, with a built-in guard that intercepts such clicks and converts them into proposals for review.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Node.js Microservices Without Express: A Zero-Dependency Approach

A developer has demonstrated that Node.js ships with all the tools needed to build communicating microservices without installing Express or any third-party packages. Using only node:http, the global fetch API, and node:test, two functional services — a user service and an order service — were built and tested. The exercise highlights real distributed systems failure modes, such as unhandled promise rejections crashing a service when a downstream call fails without a try/catch block. A key architectural warning is also raised: splitting code into separate services while sharing the same database schema delivers the costs of distributed systems without the benefits. The author argues developers should understand these fundamentals before defaulting to Express and Docker setups.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Terraform Remote State: Why It Is a Critical Consistency and Security Boundary

Terraform's state file is not a simple cache but a precise mapping between declared infrastructure and real provider objects, including sensitive credentials and dependency metadata. Remote state transforms this mapping into a shared consistency boundary for teams, while backend locking prevents competing automation runs from causing lost-update anomalies. Encryption, access controls, object versioning, and audit logs are all necessary safeguards, since marking outputs as 'sensitive' only affects display, not how data is stored. Monolithic state files increase lock contention and blast radius, but over-fragmentation creates brittle cross-state dependencies, so partitioning should follow ownership, lifecycle, and failure domains. State should never be manually edited under pressure; instead, teams must stop all writers, preserve existing versions, and use supported operations like import or moved blocks, validating backups through tested recovery exercises.

0 comments Read more at DEV Community