How a Forgotten 34-Line Test File Accidentally Built an AI Testing Discipline

·1 views

A developer building PheronAgent, a local 9-billion-parameter macOS AI model, discovered that two separate test frameworks had been independently created for the same project three weeks apart, with no shared scenarios and no awareness of each other. The first file — 34 lines of basic sanity checks — sat untouched inside an unrelated infrastructure commit for 27 days before a 324-line Python test harness was rebuilt from scratch in a different folder. A deliberate rule established during the first full test run — log everything, fix nothing mid-flight, review evidence before acting — became the foundational methodology for the entire project. The process revealed how fragile software project memory can be, with real bugs logged, some intentionally left unresolved, and all session transcripts and commit histories preserved as documentation. What began as an accidental duplication eventually evolved into a structured testing discipline the developer considers worth sharing broadly.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Codeberg Bans AI-Generated Code After Members Vote 71% in Favor

Codeberg, a Berlin-based non-profit Git platform with a democratic membership structure, voted between July 9–22, 2026 to restrict AI-generated code uploads, with 358 votes in favor and 144 against out of 516 total. The new Terms of Use prohibit content primarily composed of code written by generative AI tools such as Claude or OpenAI Codex. Enforcement will be handled case-by-case rather than through automated mass deletions, and older repositories predating large language models are not affected. Supporters argue the policy addresses a practical imbalance: while AI makes code generation cheap, human volunteer reviewers still bear the full cost of evaluating every pull request. The Zig programming language community has since migrated its infrastructure to Codeberg, citing GitHub's push toward AI-assisted workflows as a key reason.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

One Developer Built a 131-Test AI Eval Harness After a Chatbot Hallucinated a Discount

A solo developer building AI agents for WhatsApp discovered that passing all unit tests was no guarantee of safe production behavior after a lead qualification bot invented a non-existent 15% discount for a client. The incident caused no direct revenue loss but damaged trust, and the root cause was traced to subtle drift in the LLM's internal reasoning rather than any code bug. In response, the developer built a four-layer evaluation harness now comprising 131 tests, running on Oracle Cloud Infrastructure at just $0.03 per full run. Unlike standard unit tests, which verify deterministic code outputs, the harness also evaluates semantic intent, catching cases where an LLM might misclassify user messages after a model update. The developer now mandates that no new AI agent feature ships without first passing through this evaluation framework.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Gateway Pattern Lets Developers Use One Key for Claude, GPT, and Gemini

Managing multiple AI model providers typically requires separate API keys, SDKs, and billing accounts for each, adding complexity unrelated to the core product. The AI gateway pattern solves this by routing all requests through a single OpenAI-compatible endpoint using one credential, regardless of the target model. On Neon's serverless platform, enabling the gateway via a config flag automatically injects the required token and base URL into the deployed function, eliminating manual key management. A developer tested the setup by sending identical chat requests through a single handler to GPT, Claude, and Gemini, all returning correct responses with the same code. One noted compatibility issue is that GPT-5 models require the max_completion_tokens parameter, while other providers use max_tokens.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why storing company data as static columns erases historical truth

A software developer argues that storing company attributes like CEO names and headquarters as simple database columns is fundamentally flawed, because these facts change over time and static fields cannot capture that history. When a value is overwritten, the previous data is permanently lost, making it impossible to answer questions tied to a specific point in time. The proposed fix is an event-based schema where each change is recorded as a new row with a valid_from date, a valid_to date, and a source URL, preserving the full timeline. The author also highlights the need to store date precision alongside dates, since sources often only specify a year or quarter rather than an exact day. The core principle offered is that any company fact capable of changing should be modelled as a dated, sourced event rather than a column.

0 comments Read more at DEV Community