Team's Security Tool Passed 216 Tests but Failed GitHub Build Over Stale Report Files

·2 views

At Hacktoberfest Hack Day 2026, hosted at Rishihood University in Sonipat, the team Commit-tee built pkgtruth, a tool for comparing published Python packages against their source code to spot security-relevant differences. The project offers a command-line interface and a browser interface, with features such as Release DNA for comparing releases, Proof Capsule for evidence reports, and Release Gate for rule-based evaluation. A local Qwen model via Ollama explains findings in plain language, though the team kept AI out of the actual security determinations. Despite all 216 local tests passing, the GitHub pull request failed CI because committed example reports were outdated relative to recent output format changes. The team described the episode as one of the most instructive parts of the event.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

DraftMark uses a model to write ship survey code, checked against a printed booklet

A developer built DraftMark for the Meta AI Developer Hackathon to time-average a ship's moving waterline from video and have a model generate per-ship draft survey programs. Rather than trusting the model's output, the author tested the generated program against 22 worked loading conditions printed in a Ro-Ro ship's stability booklet, requiring agreement within 0.1 percent. A verbatim implementation of the booklet's procedure matched only 14 of 22 rows, because it omitted a second trim correction and contained unstated conventions. After two regenerations with feedback on failing rows, the model's program reproduced all 22 conditions. The finished survey was also checked on one in-sample row, landing about 0.01 percent from the printed value.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Adversarial test pairs reveal when agent caches serve wrong write actions

Hit-rate benchmarks show how often an agent cache helps but not when it causes harm, and replaying the wrong write step, such as deactivating instead of activating, can outweigh any token savings. The author proposes an adversarial eval built from near-opposite command pairs that share wording and structure but differ in one consequential verb. Embedding similarity tracks topic rather than polarity, so a threshold-only cache scored 1 of 18 false positives on this set, while a gated engine that replays write-containing trajectories only on exact repeats scored 0. The author recommends running such an eval as a CI gate so that threshold or model changes cannot silently reintroduce the failure. The method is presented as portable to other domains, though the 18-pair set itself is only a template and says little about any particular traffic.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Haunted AI: Whispers of Blackthorn Manor, a Local Qwen-Powered Detective Game

Haunted AI: Whispers of Blackthorn Manor is a supernatural detective browser game in which a 1926 death draws the player into a mansion a century later. Three ghosts, Eleanor, Thomas and Clara, each recall different parts of the night, so the player must gather evidence across six rooms, collect eight clues, reconstruct a torn letter and connect findings on a detective board. The game ends with a submitted theory and one of three endings: Truth Revealed, Partial Truth or False Accusation. Its AI gives each ghost a limited, consistent perspective, with dialogue generated by the locally run open-weight qwen2.5:1.5b model through Ollama while the backend enforces game rules. The project was built as a submission to the Hacktoberfest Open-Source AI Challenge and is published as open source under the MIT license.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Handling Email API 429 Rate Limits in Logistics: Who Owns the Template?

The article argues that a 429 response from an email API should be treated as a queuing decision rather than a failed send. It recommends durable verification events, honoring the provider's Retry-After header, and retrying with bounded jitter and a defined deadline. A central point is that retry logic controls timing while template ownership controls content, and mixing the two can cause stale or inconsistent emails. For logistics signup verification, the author favors keeping templates in the application repository, since pinning a versioned render keeps retries reproducible, though provider-managed templates suit teams needing faster copy changes.

0 comments Read more at DEV Community