Developer scraps token-optimization project after benchmark reveals misleading savings data

·12 views

A software developer built a side project called token-optimization-stack to reduce token costs while using Claude Code for engineering work, assembling five tools designed to cut context reads, output verbosity, and model routing expenses. Two of the five tools — Headroom and LiteLLM — were dropped after testing revealed they did not function as documented or did not support the intended per-task model switching. The remaining stack of Graphify, Serena, LeanCTX, and Caveman was prepared for rigorous benchmarking against SWE-bench Verified tasks across sixteen repositories. However, the project was ultimately abandoned when the cost of running a statistically valid benchmark proved prohibitive before any publishable result could be produced. The developer concluded that the token-savings figures observed earlier were actively misleading, and chose to document the failure rather than a clean success.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

LeeX CEO Oscar Awowari on Building a Geospatial Search Layer for City Discovery

Oscar Awowari, Founder and CEO of LeeX, is developing a city discovery platform that goes beyond conventional text search to enable location-based discovery of businesses, events, and infrastructure. The core engineering challenge lies in building a dedicated geospatial search layer that can answer spatial queries such as finding places within a set radius or within a geographic boundary. To handle this efficiently, the system relies on spatial indexing, which narrows search candidates to a relevant geographic region rather than scanning every location in the database. Techniques like radius search and bounding box filtering are used in combination, applying cheaper broad filters first before running more precise distance calculations. Awowari emphasizes that scalability must be considered from the outset, as the platform is designed to grow from thousands to potentially millions of indexed locations across a city.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer releases free Node.js CLI tool to detect hreflang SEO errors

A developer has published an open-source, zero-dependency Node.js command-line tool called hreflang-audit, designed to identify common international SEO errors in hreflang implementations. The tool crawls websites or parses XML sitemaps and runs 11 checks covering issues such as invalid ISO language codes, missing return links, broken target URLs, and canonical conflicts. Each detected issue is accompanied by a fix hint, and output can be generated as a human-readable report or in JSON format for use in automated pipelines. A GitHub Action is also included, allowing teams to schedule weekly audits and trigger build failures if hreflang regressions are detected. The project is MIT licensed and available on GitHub, with sitemap-versus-HTML mismatch detection planned as a future addition.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Zero-Trust Authorization Framework Proposed for Debt Collection Systems

A technical framework applying NIST SP 800-207 Zero Trust Architecture to debt collection operations has been outlined, separating authorization decision logic from execution logic. The design introduces a Universal Action Request Contract that uses cryptographic pseudonymous references instead of raw personal data, ensuring privacy across communication channels. Incoming evidence is classified into four states — missing, stale, conflicting, or invalid — to enforce strict, verifiable authorization checks before any collection action is taken. The architecture mandates deterministic decision engines over AI language models for final authorization rulings, with LLMs limited to fact extraction and record reconciliation. Short-lived, sender-constrained permits based on RFC 9449 are proposed to replace conventional bearer tokens, reducing risk in high-consequence collection operations.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Step-by-Step Guide to Installing Graphify for Use with Claude CLI

Graphify is a command-line tool that converts a codebase, configuration files, or documentation into a queryable knowledge graph, outputting files such as graph.json and GRAPH_REPORT.md. The tool is installed globally on Linux or WSL using uv, a Python package manager, and only project-specific configuration needs to be repeated per repository. Users must install the appropriate backend client extra — such as the Anthropic, Gemini, or OpenAI package — depending on which large language model they intend to use for semantic extraction. API keys for the chosen backend must be set as environment variables, with a noted caveat that quoted values in .env files can cause authentication errors if not stripped correctly. The guide also recommends limiting graph extraction to relevant project paths and using the --max-concurrency 1 flag when hitting rate limits on free-tier LLM APIs.

0 comments Read more at DEV Community