How to Test an AI Coding Tutor Using a Deliberately Small Bug

·3 views

The article suggests evaluating an AI coding tutor by giving it a short program with one predictable fault whose answer is already known. The user should ask for a hint rather than corrected code, then see whether the tutor directs them to inspect the loop boundary or simply rewrites the whole program. Comparing several tools with the same bug and request helps isolate the quality of help provided. Learners are advised to test explanations against running code and references such as the official Python tutorial, using cases like an empty list and a single-item list. The session should end with the learner writing the fix in their own words and, ideally, recreating the bug in a different small program the next day to confirm the skill is independent of the chat.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Separating Liveness and Readiness Probes for Mock Servers to Prevent Flaky Tests

A mock server in a container should expose distinct liveness and readiness checks rather than one endpoint that reports success as soon as the process starts. Routes are only registered after the spec is read and examples are resolved, so a server can be reachable before it can actually answer the expected endpoints. Premature traffic in that window causes intermittent resets and 404s that look like test flakiness. Liveness should restart a dead process, while readiness should hold traffic back until the contract has loaded. Orchestration can wait on readiness, and a startup probe or generous failure threshold can accommodate slow loads without causing premature restarts.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Catalog Endpoints Do Not Establish a Checkout Contract for AI Agents

A product catalog can show an agent what items exist but grants no ability to purchase them, and this boundary is often overlooked in integrations. A QA run of one gateway found five store actions, covering product search and operations on existing orders, with no order-creation or checkout action discovered and no payment handlers advertised. The author argues this supports only a narrow conclusion about discovered capabilities, not whether purchases are possible or safe to add. Real checkout requires much more than product lookup, including cart construction, price freshness, payment authorization, confirmation, and idempotency. The author recommends that discovery separate read and mutation capabilities, enforce these boundaries server-side, and test both advertised and absent actions.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Arguseye adds Django template previews without starting an app server

Arguseye is adding a native Django workflow that lets developers preview a template with explicit template, engine and input data, without starting the application server. The feature is available on the feat/template-previews development branch, not in the published 0.0.15 npm and VS Code packages. Users define a template mapping and data profile in arguseye.templates.json, then run the CLI to capture a screenshot, accessibility output and errors. After editing, a diff command compares the result against the saved baseline and reports whether it is unchanged, changed or incomplete. The workflow renders templates with supplied context only and does not run views, middleware, context processors or database queries.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Low-bitrate videos can grow when re-encoded, so a quick ffprobe check flags them first

A developer built a small script that uses ffprobe to read header data and compute bits per pixel per frame, then labels each video SKIP or ENCODE before a batch re-encode. The threshold of 0.10 comes from an earlier test of 36 files, where all low-bpp files below 0.08 grew after re-encoding. On five public WebM clips from Wikimedia Commons, the script flagged only one file, a 1080p moose clip, which grew to nearly 2.8 times its original video size. The other four clips shrank by between 17% and 65% when re-encoded with x264 at CRF 23. The script runs entirely locally and only reads metadata, so it can screen a whole folder without modifying any files.

0 comments Read more at DEV Community