Developer builds Cross-Examine tool to catch AI-generated Python regressions tests miss

·1 views

A developer discovered that an AI-generated Python code change silently altered a function's return value from an empty list to None, a subtle behavioral regression that passed all existing tests. This prompted the creation of Cross-Examine, an open-source Python verification harness that compares the behavior of a base and revised Git revision by replaying identical inputs against both versions. When a behavioral difference is detected, the tool generates a reproducible receipt containing the input, command, and outputs from both code versions for developer inspection. GPT-5.6 proposes candidate inputs to test via a constrained schema, but all verdicts are determined by deterministic execution rather than AI judgment. The tool was built during OpenAI Build Week and is currently limited to Python repositories the user trusts, as it executes live code outside a sandbox.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How to Properly Test a Salesforce API Connection Beyond Just Authentication

A successful Salesforce login only confirms authentication, but a fully functional API connection requires reaching the correct org, using a valid instance URL, and accessing the necessary objects. Developers should use OAuth tokens returned after authentication to identify the correct instance URL, avoiding errors caused by hardcoded endpoints from other sandboxes or production environments. Salesforce recommends starting with a read-only request, such as the GET Limits endpoint, before attempting any create, update, or delete operations. Testing access to specific objects like Accounts or Contacts is also essential, as authentication alone does not verify that an integration can reach the required resources. Following this structured sequence — authenticate, test basic connectivity, verify object access, then add write operations — makes it easier to isolate failures before broader integration testing begins.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to automate supplier price list imports without breaking your product catalog

A software developer has detailed a practical pipeline for automating price list ingestion at a distribution company, where suppliers send Excel or PDF files every few weeks in inconsistent formats. The solution uses deterministic table extraction tools like pdfplumber and openpyxl, with an LLM applied only once per file to map column headers rather than parsing every row. Unmatched product codes are held in a review queue instead of auto-created, preventing duplicate entries from accumulating in the master catalog, and human resolutions are stored to shrink the queue over time. Hard-coded validations catch issues such as price swings above 40%, zero or non-numeric values, duplicate codes, and unexpectedly low row counts that could otherwise wipe active listings. The author notes that edge cases like decimal separator conflicts, inconsistent VAT inclusion, and scanned PDF images require additional handling, but the overall process cuts what was a half-day manual task down to a few minutes plus shrinking review time.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

JavaScript Variables Explained: var, let, and const Compared

In JavaScript, variables are named containers used to store values of various data types, including numbers, strings, and booleans. They can be declared using three keywords: var, let, or const, each with distinct behavior. The var keyword has function-level scope, meaning it remains accessible outside block statements like if conditions. In contrast, let is block-scoped, so a variable declared inside a block does not affect one with the same name outside it. The const keyword is used to declare constants whose values remain fixed throughout the program.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

nx-safe-suite Offers Five Production-Ready Packages to Standardise Next.js Apps

A developer has published a two-part deep dive into nx-safe-suite, a collection of five production-grade packages designed for Next.js applications. The first package, @nx-safe-suite/env, validates all environment variables at application startup, exiting with a clear report if any are missing or malformed rather than failing silently at runtime. It enforces a strict separation between server-side and client-side variable schemas, catching misconfigurations before the application boots or even before code is deployed in CI. Another package addresses inconsistent API response shapes across large codebases by providing typed helpers that enforce a uniform success envelope and RFC 9457-compliant error responses. The error format includes a machine-readable code field, allowing frontend code to handle errors by switching on stable identifiers rather than parsing status codes or message strings.

0 comments Read more at DEV Community