SShortSingh.
Back to feed

Audit of 36 MCP servers finds a third failing AI agents due to poor documentation

0
·1 views

A developer who integrates MCP connectors into production AI agents built a tool called mcpgrade to audit the quality of Model Context Protocol servers beyond basic spec compliance. Scanning 36 popular servers revealed that 11 — including official offerings from MongoDB, Notion, Airtable, and GitHub — scored D or F grades. The dominant failure was rule D004: parameters lacking descriptions, with firecrawl logging 132 such errors out of 134 total and todoist hitting 110. Most failing servers auto-generate schemas from Zod or OpenAPI definitions without adding human-readable descriptions, leaving AI models unable to correctly select or call tools. The author argues that adding parameter descriptions is the single highest-leverage fix developers can make to improve agent reliability.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Lessons from Building a Location-Aware Contact Management App for Professionals

A development team building a location-aware contact app called Connecti5 found that storing contact details alone is insufficient — the real challenge is helping users retrieve the right connections at the right moment. The team observed that professionals often think in geographic terms, asking questions like who they know in a city they are visiting or which contacts are near a planned route. This insight led them to design a map-based contact experience that organizes professional relationships by location, context, and timing rather than simple name or number searches. However, adding location features introduced significant privacy responsibilities, with the team emphasizing a clear distinction between displaying saved contact location data and real-time people-tracking. The project highlighted that location-aware tools must be designed with a privacy-first mindset from the outset to avoid eroding user trust.

0
ProgrammingDEV Community ·

Enterprise Tech Lead Pivots from Procurement Systems to Browser-Based Developer Tools

A software professional with over 14 years of experience in Microsoft-stack enterprise applications is expanding into browser-based productivity tools built with Next.js, React, and TypeScript. Their enterprise background spans procurement, inventory management, warehouse operations, and EPOS systems. The developer is building tools such as YAML Studio, which generates configurations for Kubernetes, Docker Compose, and GitHub Actions, among other platforms. A key challenge identified in this work is preserving document structure during file conversion, particularly for procurement documents like purchase orders and invoices that each carry distinct layouts and business rules. The transition reflects a broader goal of creating fast, privacy-friendly software that solves real business problems directly in the browser.

0
ProgrammingDEV Community ·

Building a REST API Is More Than Just Endpoints, Developers Find

Many developers underestimate the complexity of building REST APIs, initially finding basic GET and POST endpoints straightforward. However, the scope quickly expands beyond simple requests to include middleware, validation, error handling, and database integration. Authentication, authorization, testing, pagination, and CORS configuration add further layers of complexity. Developers often discover that creating a functional API effectively means building an entire backend ecosystem. The experience highlights a common gap between early expectations and the full technical demands of API development.

0
ProgrammingDEV Community ·

Developer builds opinionated scaffolding tool to cut setup time with fixed stack

A developer created 'create-notils', a project scaffolding CLI tool that uses a fixed, pre-selected tech stack instead of offering extensive configuration options. The tool ships with Next.js, React 19, Bun, Tailwind CSS v4, shadcn/ui, Biome, and TypeScript, reflecting the stack the author uses daily. The rationale is that unlimited configuration choices multiply project complexity exponentially, slowing down the initial setup process. Unlike some opinionated tools, create-notils generates code that developers fully own and can modify freely, with no vendor lock-in. Future versions plan to support modular add-ons for features like authentication and payments, rather than expanding the initial prompt questionnaire.

Audit of 36 MCP servers finds a third failing AI agents due to poor documentation · ShortSingh