How to Build a Free Test Harness for Benchmarking Coding AI Agents

·5 views

Developers can evaluate coding agents more reliably by running a structured audit that tracks five key signals: exit code, elapsed time, modified files, agent output, and test suite results after a task. Rather than relying on model cards or demos, the approach uses a disposable Git directory and a realistic, under-specified task to measure whether an agent makes contained changes without causing collateral damage. A Python script seeds a fixture, injects the task via an environment variable, runs the agent, and returns results as JSON. The same fixture should be run both locally and on a remote server to isolate how much the environment — not the model — influences outcomes. The guide was produced in partnership with MonkeyCode, a platform offering free model access and server-side execution to support this kind of cross-runtime comparison.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developers Use Free Sanctions APIs to Embed EU AI Act Compliance in CI/CD Pipelines

Software teams building high-risk AI systems under the EU AI Act face a compliance gap when vendor screening remains in spreadsheets rather than automated pipelines. A developer has shared a Python-based approach that integrates free sanctions-screening APIs directly into CI/CD workflows, automatically checking vendor and supplier names against major sanctions lists. The script fails a build if a vendor returns a HIGH or MEDIUM risk verdict, surfacing which data field triggered the match. The EU AI Act's Articles 9, 10, and 25 require organisations to manage risks across their entire supply chain, including training data providers, cloud API vendors, and payment processors. Annual vendor reviews are considered insufficient in fast-moving development environments where new dependencies can be introduced and shipped within days.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

form.fscss lets developers build modern forms using pure CSS, no JavaScript

A new open-source CSS module called form.fscss, part of the FSCSS ecosystem, enables developers to build fully featured forms without JavaScript UI logic. The module uses native CSS mechanisms such as the :placeholder-shown pseudo-class to power floating labels and the checkbox-hack pattern for custom checkboxes, radio buttons, and toggle switches. Inline validation states like error and success are triggered simply by adding a class to the field wrapper, keeping the presentation layer entirely in CSS. Micro-interactions such as a gradient button press effect are also handled through CSS pseudo-classes rather than event listeners. Full documentation, design tokens, and markup requirements are available on GitHub at github.com/fscss-ttr/form.fscss.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Docker Networking and Volumes Explained: How Containers Communicate and Store Data

Modern applications typically run across multiple containers — such as web servers, databases, and cache layers — which must communicate reliably with one another. Docker addresses this through built-in networking drivers, primarily bridge, host, and none, each suited to different use cases. A key feature of Docker networking is DNS-based container discovery, which lets containers reference each other by name rather than by dynamic IP addresses that can change on restart. Docker volumes complement networking by providing persistent storage that survives container removal, ensuring data is not lost when containers are stopped or recreated. Together, Docker networking and volumes form the foundation for building and managing multi-container applications in production environments.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Releases ActiveVPN: Open-Source Terminal Tool to Detect VPN and Proxy Use

A developer has released ActiveVPN, an open-source command-line tool designed to verify whether a VPN or proxy is actively protecting a user's connection. The tool checks network interfaces, running processes, external IP addresses, DNS resolvers, and IPv4/IPv6 configurations to detect potential leaks or misconfigurations. It assigns a verdict score from 0 to 100, ranging from 'Clean' to 'VPN Detected,' helping users confirm their actual privacy status. ActiveVPN supports features such as a kill switch, continuous watch mode with alerts, and JSON/CSV export for logging. The project is available on GitHub and is compatible with Python 3.8 through 3.12 across multiple operating systems.

0 comments Read more at DEV Community