Developer Builds Git Worktree Harness to Run Fair Side-by-Side AI Model Comparisons

·1 views

A developer created a shell-based testing harness using git worktrees to compare two AI coding models under identical, isolated conditions on the same codebase. The tool addresses a common flaw in model comparisons where inconsistent prompts or shared working directories skew results in favor of the testing setup rather than the models themselves. Each model receives the same task file and a byte-identical checkout in a separate directory, with results scored on test pass rate, diff size, and out-of-scope file changes. Git worktrees were chosen over fresh clones or branches because they provide clean isolation without duplicating the full repository on disk, and cleanup requires just one command. The author ran the experiment at no cost using MonkeyCode's free tier, and the harness is designed to work with any two models or agents without modification.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Structured Data Models Can Make Survival Game Item Guides Far More Useful

A developer has outlined a data-driven approach to building item guides for survival games like Subnautica 2, arguing that key facts such as unlock conditions, required equipment, and navigation routes should be stored as structured fields rather than buried in prose. The method separates prerequisites, landmarks, hazards, and recovery steps into distinct data categories, making guides easier to follow and errors easier to diagnose. Using the Repair Tool as a case study, the author demonstrated how navigation can be broken into discrete segments, each with its own origin point, bearing, and visual confirmation landmark. The approach also distinguishes between mandatory and recommended equipment, preventing beginners from delaying progress due to unclear requirements. Hazard warnings, the author argues, are most useful when attached directly to the relevant route segment rather than grouped into a generic notice.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer builds browser-style DevTools dashboard for Node.js backend debugging

A developer frustrated with console.log-based backend debugging has released Wevna, an open-source local observability tool for Node.js applications. The SDK requires just two lines of code to set up and opens a localhost dashboard displaying HTTP requests, SQL queries, Redis commands, and logs in a visual, waterfall-style interface. Wevna automatically flags patterns such as N+1 database queries by detecting repeated queries with the same shape within a single request. The tool is privacy-focused by design, with all data staying on the local machine and no account, API key, or external data transfer required. Currently in v1, it supports Postgres and Redis only, and the developer is actively seeking feedback from users testing it on real applications.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds C# .NET App to Help Shoppers Compare Prices and Travel Costs

A developer has created a web application called Digital Fare App using the C# .NET ecosystem, designed to simplify everyday purchase calculations. The tool helps users determine whether bulk deals or longer trips with multiple passengers are genuinely cost-effective. Users can save their calculation records to a database for future reference. Auth0 integration for authorization and user authentication is planned as an upcoming feature. The project is currently seeking community input on branding, including a name and logo.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Build a Lightweight AI Agent in 150 Lines of Python Without Frameworks

A software developer shares a step-by-step guide to building a production-ready AI agent in approximately 150 lines of Python, using no external frameworks such as LangChain or CrewAI. The approach was inspired by a startup founder who struggled with framework abstractions that did not fit his specific workflow. The hand-built agent includes tool use, working memory, budget controls, and escalation logic, with every line remaining fully readable and traceable. Building from scratch gives developers direct visibility into model context, honest budget enforcement, and easier debugging compared to opaque framework internals. The author acknowledges that frameworks can save time for standard workflows, but argues that writing the core loop manually — roughly 50 lines — yields greater control and understanding.

0 comments Read more at DEV Community