Popular MCP filesystem reference server fails basic AI tool-routing tests, audit finds

·2 views

A developer recently tested routeproof, a tool that checks whether AI models correctly select the intended server function based on tool descriptions alone, against Anthropic's canonical MCP filesystem reference implementation. The audit revealed that the server's list_allowed_directories tool was consistently mis-selected for unrelated tasks such as reading a file and generating a recursive directory tree. Scoring between 3 and 6 out of 6 across two separate runs, the results also highlighted that AI tool routing is non-deterministic, meaning a single test pass can be misleading. The root cause was identified as an overly broad description for list_allowed_directories that failed to clarify what the tool does not handle. Adding explicit boundaries to the description — directing recursive-structure queries to directory_tree — was proposed as a straightforward fix.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Skills and MCP Turn AI Coding Agents into System-Aware Development Partners

AI coding agents often struggle with platform-specific conventions, syntax rules, and runtime details when working on specialized systems. A concept called Skills addresses this by packaging reusable instructions that teach agents how to approach specific tasks, such as structuring workflow YAML or following project conventions. Model Context Protocol (MCP) complements this by giving agents structured access to a live external system, enabling them to inspect, validate, and update workflows beyond just generating local code. Together, Skills and MCP shift the agent's role from a generic code generator to a context-aware development partner that can verify its output against real runtime environments. This pattern is seen as particularly valuable for workflow builders, automation platforms, and internal tools where correctness depends on domain knowledge as much as code validity.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to estimate if your codebase fits an AI model's context window before pasting

Developers feeding entire codebases into AI models often hit context window limits, causing truncation errors or silent data loss where the model answers from incomplete information. A practical workaround involves estimating token count offline using a formula that blends character count and word/symbol runs, achieving roughly 5–10% accuracy compared to real tokenizers. Since context windows differ significantly across models — 200K for Claude, 400K for GPT-5, and 1M for GPT-4.1 and Gemini 2.5 Pro — developers should budget their code bundle against the specific model being used. When a repository is too large, the recommended approach is to omit the largest file bodies first while keeping all filenames listed, so the model retains a full project map. An open-source CLI tool called ctxpack automates this trimming process and is available free under the MIT license on GitHub.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Run a Local LLM on a 4GB RAM PC Using BitNet and Llama.cpp

A developer has shared a lightweight setup guide for running large language models on low-end machines with just 4GB of RAM. The recommended stack combines BitNet 1.58, llama.cpp, and tools such as persistent memory and auto-batching, with Ollama offered as a simpler alternative. BitNet is highlighted for its speed and efficiency, reportedly delivering accuracy comparable to a 7B parameter model at around 25 tokens per second on modest hardware. Users with a dedicated GPU are advised to leverage it for better performance, while a 512-token batch size is suggested as a practical starting point. Optional enhancements like LoRA-based test-time training and tool calling are mentioned for those looking to extend the model's capabilities further.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Web Auth Explained: From Password Hashing to OAuth and Secure Session Management

A structured bootcamp guide by Dr. Angela covers the core levels of web authentication, starting from basic email-and-password registration to advanced OAuth integration with Google. It emphasizes that passwords must never be stored as plain text, advocating instead for hashing combined with salting using tools like bcrypt to defend against rainbow table attacks. Session management is addressed through Express-session and Passport.js middleware, which handle user login state on the server side. The guide also stresses storing sensitive credentials such as API keys and session secrets in environment variables via dotenv, keeping them out of source code repositories. Finally, it introduces OAuth as a delegated authentication method that lets users sign in via third-party providers like Google without exposing their passwords to the application.

0 comments Read more at DEV Community