New tool 'skilldiff' detects behavioral regressions in AI coding agent skill files

·5 views

A developer has released an open-source tool called skilldiff, designed to catch unintended behavioral changes when editing SKILL.md instruction files used by AI coding agents like Claude Code, Cursor, and Codex. The tool runs a skill file in a real agent environment against a fixture repository, comparing outputs from the old and new versions of the file across a pull request. Unlike standard text diffs, skilldiff tracks what the agent actually does — including files modified, commands executed, and tools invoked — rather than just what text changed. A built-in assertion system flags forbidden actions, such as an agent unexpectedly creating or modifying files it was not supposed to touch. In CI mode, the tool replays recorded traces deterministically without requiring an API key, and can post behavior reports directly to pull requests to block regressions from merging.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Structured AI Model Jev Matches Claude on Bounded Decision Task With Perfect Accuracy

A developer tested TypeSafe's Jev model against Claude Sonnet on a real judging workflow used for the 2026 Arbitrum Open House London Online Buildathon. The test focused on a single bounded decision gate — classifying project submissions as satisfied, not_satisfied, or insufficient_evidence — rather than open-ended writing or reasoning tasks. Both models were given identical JSON evidence packets and a four-step decision procedure, with each configuration run three times across 102 archived submissions, producing 306 decisions per variant. Jev, which TypeSafe positions as a structured decision model with primitives like Choice and Score rather than free-form generation, achieved 100% accuracy with zero false passes and full consistency across all runs. The author notes the test was deliberately narrow, designed not to compare general capabilities but to evaluate whether frontier LLMs are overkill for tightly scoped, policy-bound classification tasks.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Systematically Debug .NET MAUI Build Failures on macOS

A developer documenting their .NET MAUI setup on macOS found that build failures often stem not from missing installations but from misconfigured tool paths and version mismatches. A key issue encountered was Xcode being installed yet inactive, because the developer directory was pointing to Apple's Command Line Tools rather than the full Xcode application. The troubleshooting approach recommended treating the development environment as a dependency chain — from the .NET SDK down to the simulator or physical device — and isolating the failing layer before making any changes. Android builds presented separate challenges, including JDK discovery failures even when the JDK was present on the system. The core lesson is that effective debugging requires verifying what build tools can actually see and use, not just what is installed on the machine.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Django Logging Series Part 2: Python Logging Fundamentals Explained

A DEV Community tutorial series on Django logging has published its second installment, focusing on the fundamentals of Python's built-in logging system. The article explains how Python's logging module uses a hierarchical, object-oriented pipeline anchored by a root logger at the top of the tree. It covers the five standard log levels and their numeric severity values, clarifying how the configured threshold determines which messages get processed and output. Real-world log samples from both a Python service and a Django framework are used to illustrate key components such as timestamps, log levels, logger names, and event descriptions. The guide aims to help developers understand the foundational concepts needed before configuring structured logging in production Django applications.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Backtracking Explained: The Algorithm That Explores and Undoes Wrong Choices

Backtracking is a structured algorithmic technique used to solve problems where multiple choices must be explored, such as Sudoku, maze solving, and generating permutations. The core idea involves making a choice, exploring it recursively, and undoing it if it proves invalid before trying the next option. This make-explore-undo cycle navigates a decision tree, pruning paths that cannot lead to a valid solution rather than restarting from scratch. A classic example is generating all permutations of an array, where elements are added one at a time and removed upon backtracking. The pattern is widely applicable to constraint satisfaction and combinatorial problems in computer science.

0 comments Read more at DEV Community