How to Build a Reproducible QA Harness for AI Image Editing Features

·1 views

AI image editing features that perform well in demos often break down in real-world use when users upload low-contrast photos, trigger duplicate actions, or request complex edits like overlapping object removal. A structured QA harness built around fixed test fixtures, explicit invariants, and repeatable run metadata can make such failures easier to reproduce and diagnose. The approach defines a task contract with clear, testable conditions — such as preserving image dimensions and avoiding new artifacts — rather than relying on vague quality judgments. Each test run records input file checksums, model labels, and operation metadata to ensure consistency across model or UI changes. The harness also treats asynchronous UI states as first-class test targets, verifying transitions from idle through queued, running, and terminal states like failure or timeout.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Open-Source AI Agent Uses Deterministic Code to Stop SLM Hallucinations in Billing

A developer has open-sourced a local-first AI agent architecture called closed-loop-slm-agent, built around Meta's Llama 3.2 3B model, designed to eliminate hallucinations in high-stakes business workflows like invoicing and ledger billing. The system strictly separates responsibilities: the small language model handles only prose generation, while deterministic Python code owns all arithmetic, balance verification, and transaction matching. To avoid unreliable 'vibe-based' evaluations, the architecture uses Quadratic Weighted Kappa to calibrate AI judges for statistically sound output scoring. The agent also features explicit tri-state routing, allowing it to abstain or escalate to a human expert rather than guessing when data is ambiguous or missing. Human corrections are fed back into the system in-context, creating a continuous learning loop instead of letting operator edits go to waste.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer builds lightweight PHP library to add AI agents without switching to Python

A PHP developer has released NanoAgentPHP, an open-source, MIT-licensed library that lets developers integrate AI agents directly into existing PHP codebases. The project was born out of frustration with Python's environment management and package conflicts, which the developer encountered while trying to add AI features to PHP applications. Existing PHP AI libraries were either too tightly coupled to large frameworks like Laravel or Symfony, poorly maintained, or required significant application restructuring. NanoAgentPHP supports multiple AI providers including OpenAI, Anthropic, Groq, and DeepSeek, and offers features such as tool calling, RAG, streaming, and multi-agent chains with a minimal dependency footprint. The library is available on GitHub and is aimed at PHP developers who want production-ready AI capabilities without adopting an entirely new technology stack.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Long-Running AI Context Servers Silently Degrade — Here Is Why It Matters

A developer maintaining a personal infrastructure context server noticed it repeatedly prompted for re-analysis, which led to discovering that long-lived AI assistant sessions quietly lose their context while continuing to respond confidently. Unlike simple one-shot data lookups, these servers boot once, cache cloud data, and stay alive for hours or days, making them vulnerable to state drift over time. The author documented four distinct bugs from the same codebase — including a missing analyzer in a refresh list, a server stuck serving its boot snapshot, and cache entries expiring on mismatched schedules — each causing silent, hard-to-detect data loss. The core problem is that incremental update paths in long-running processes create opportunities for live state to diverge from what a fresh restart would produce. The author argues this failure pattern is not unique to their tool and urges developers to treat AI infrastructure context as a long-running process, not a reliable lookup.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why AI-Generated Code Passes Linting but Still Breaks Your Android Architecture

AI coding assistants tend to produce code that is locally correct — valid syntax, no null dereferences — but structurally flawed in ways that only become apparent across the broader codebase. Traditional static analysis tools like ktlint, detekt, and Android Lint are designed to catch file-level or function-level mistakes, making them poorly suited to flag these cross-cutting architectural issues. Common AI-generated problems include ViewModels directly importing Retrofit DTOs, hardcoded dispatchers in the wrong layer, and tests that only assert values they themselves mocked. The author argues that catching such issues requires checks operating at a relational or behavioral level, not just local syntax rules. Teams are advised to verify that detekt is actually running with type resolution enabled in CI, since a rule that never fires looks identical to one that consistently passes.

0 comments Read more at DEV Community