Tool lets developers benchmark context plugins on their own repos before installing

·1 views

A browser-based tool called Chamnan allows developers to measure how much a context plugin compresses a public GitHub repository before committing to installing it. The tool runs entirely client-side, downloading source code into the browser tab, processing it locally, and deleting it once measurements are complete — no server, upload, or API key required. Thirteen well-known repositories have already been benchmarked, with compression ratios ranging from 24:1 for chalk to 470:1 for the Linux kernel. The developer also documented several bugs discovered during the project, including Unicode zero-width character bypass vulnerabilities, a sort-key injection flaw, and a concurrency defect in file locking. The tool is MIT-licensed, depends only on the standard library, and is available on GitHub.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

AI Answers Are Killing Developer Communities, One Search at a Time

A software developer has raised concerns about the long-term impact of AI-generated answers on online developer communities. Traditionally, searching for coding help involved visiting sites like Stack Overflow or Reddit, where user engagement helped keep knowledge current and communities active. Today, AI tools deliver direct answers by drawing from those same sources, eliminating the need for users to visit, comment, or contribute. This shift is already reducing traffic to community platforms, which may lead to declining engagement and a slowdown in new knowledge creation. The author warns that AI models could eventually stagnate on outdated information if the communities they learn from stop growing.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Open-Source Tool CauterRule Finds Over Half of AI Agent Rule Candidates Inconclusive

CauterRule, an open-source tool released on GitHub and PyPI, learns standing rules from repeated AI agent failures by extracting lessons from interaction trajectories and replay-testing them. A field test covering 394 trajectories across four AI models generated 1,538 rule candidates, of which 52.9% were deemed inconclusive — more than the pass and fail results combined. Among the models tested, the cloud-based GPT-4o-mini had the highest inconclusive rate at 62.9%, exceeding both local models, while meta-llama/llama-3.1-8b-instruct had the lowest at 42.9%. The tool's replay engine uses substring and token-overlap heuristics to evaluate rules, and returns an inconclusive verdict when overlap scores fall into a gray zone between pass and fail thresholds. The findings highlight that ignoring inconclusive results — as most benchmark reports do — means discarding the majority of evaluation data and potentially drawing flawed conclusions.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

7 Key Rules for Safer Email and Phone Verification in User Account Systems

Email and phone verification serve distinct security purposes, protecting different boundaries around identity stability, abuse exposure, and account recovery. Developers are advised to treat verification codes like scheduled jobs — with deadlines, retry limits, and audit trails — rather than simple form inputs. Server-side rate limits per account, destination, IP, and device should be configured independently so they can be adjusted without redeployment during incidents. Recovery paths deserve the same engineering attention as the standard login flow, including documented state transitions for scenarios like lost school mailboxes or recycled phone numbers. A key operational warning: a provider's API success response only confirms the request was accepted, not that the user actually received the code, so delivery outcomes must be monitored separately.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Chat History vs. Agent Memory: How AI Agents Actually Remember Things

AI agents rely on two distinct mechanisms to retain information: chat history and agent memory. Chat history captures the messages exchanged within a single conversation, giving the model context only for that session. Agent memory, by contrast, allows the system to extract and store useful facts — such as a user's tech stack or preferences — that persist across separate conversations. When a new session begins, the agent retrieves relevant stored memories and combines them with the current chat history before generating a response. Understanding this distinction helps developers build smarter, more context-aware AI systems.

0 comments Read more at DEV Community