Freeze CI-Verified Commands in Tutorials; Let Humans Own Secrets and Failure Steps

·1 views

A proposed workflow for writing quickstart tutorials recommends compiling command blocks directly from a reviewed CI fixture manifest, rather than letting AI models draft them from memory. The approach aims to prevent a common failure mode where tutorials include commands that were never actually tested in continuous integration. Under this model, AI can draft surrounding prose only after command hashes are locked, while humans must review and sign off on secret handling, failure behavior, and teardown instructions. These operational elements are excluded from automated generation because they describe potential harm, not just syntax. The article presents labeled example files to illustrate the workflow and explicitly notes it is a proposal, not a report based on measured production data.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Two Methods to Identify Shopify Stores: Manual Checks and Bulk Detection Tools

Developers and marketers needing to identify Shopify-powered websites can use two main approaches: manual inspection of page source code, response headers, and cookies, or a bulk automated tool for larger lists. Shopify stores leave consistent technical fingerprints, including references to cdn.shopify.com, a powered-by header, and session cookies such as _shopify_y. For bulk detection, a tool called Tech Stack Detector, built on the Apify platform, cross-references sites against over 7,600 open-source technology fingerprints to filter only Shopify stores from a list of domains. The tool also surfaces the broader technology stack of each site, including email, review, and checkout platforms, along with the specific evidence behind each detection. Results can be exported in CSV, Excel, or JSON formats, making it practical for lead generation, agency prospecting, or competitive research.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why LLM Red-Team Reports Need Reproducibility, Not Just Screenshots

A software engineer argues that most AI red-teaming reports are little more than screenshots of bad model outputs, which cannot be independently verified or rerun. Credible reports, the author contends, must include a fixed and versioned probe corpus, raw prompts and model replies, and a SHA-256 checksum so compliance reviewers can verify nothing changed. A meaningful 0–100 score derived from how many probes a model failed should replace vague qualitative assessments, enabling CI pipelines to block deployments that exceed a set risk threshold. The author has built a small toolkit covering 35 probes across 17 attack classes, with a free 8-probe scan available without requiring an account. The core argument is that reproducibility transforms red-teaming from an anecdotal claim into a verifiable, auditable record.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Study finds AI-generated code flags zero true error-swallowing cases across 120 samples

A developer ran a controlled experiment generating 120 code samples using Qwen2.5-Coder 1.5B across 12 tasks in Python and TypeScript to test whether small AI models routinely hide or suppress errors. A Semgrep-based static detector flagged four candidates as potential error-swallowers, but manual review confirmed all four were either false positives or legitimate, documented fallbacks. The experiment found that Python generations mostly let exceptions propagate or raised them explicitly, while TypeScript samples favored try/catch blocks with logging or re-throwing. The researcher's key conclusion is that whether code truly swallows an error cannot be determined from syntax alone, as the same pattern can represent either a sound design contract or a genuine hazard depending on context. The study acknowledges significant limitations, including a single small model family, only 12 tasks, and sole-rater adjudication, and makes no broader claims about AI code versus human-written code.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Build a WebMCP Scoring Tool in Plain HTML With No Dependencies

A developer tutorial published on DEV Community walks through building a WebMCP tool from scratch using only a plain HTML file served via Python's built-in HTTP server. The guide demonstrates how to register a tool called 'score_facts' that evaluates a YAML document against five required fields — name, goal, who, what, and why — and returns a 0–100 score. Chrome supports the document.modelContext API natively behind an experimental flag, while other browsers can use a polyfill as a fallback. The tutorial emphasises defensive coding practices, including input validation and try/catch error handling, so that tool failures return structured error objects rather than raw stack traces. A security section also introduces URL allowlisting to prevent remote-fetch tools from being exploited as open proxies.

0 comments Read more at DEV Community