Harness Engineering: How AI Agent Reliability Depends on Systems, Not Just Models

·1 views

As AI systems evolve from simple chatbots into autonomous agents capable of writing code, browsing the web, and running commands, engineers are shifting focus from model selection to what is now called Harness Engineering. The concept defines everything surrounding an AI model — including context management, tools, permissions, memory, and error recovery — as the harness that turns raw intelligence into reliable work. While the model supplies reasoning capability, it is the harness that determines how safely and consistently that capability is applied in real-world tasks. Contrary to the assumption that more powerful models will eventually make such scaffolding obsolete, experts argue that better models simply enable harder tasks, which in turn introduce new failure modes requiring more sophisticated harnesses. The relationship between model and harness is therefore seen as continuously co-evolving rather than one replacing the other.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Python MCP SDK 2.0 Breaking Change Forces Emergency Patch for AI Memory Server

The official MCP Python SDK jumped to version 2.0.0 and removed the mcp.server.fastmcp module without a deprecation window, breaking downstream packages including yantrikdb-mcp, an MCP server for AI agent memory. The maintainer issued a temporary version pin within hours and then released v0.12.0 the same morning, introducing a compatibility layer that probes for available imports rather than branching on version strings. The fix covered key API renames — FastMCP to MCPServer, relocated Context and ToolError paths — while identifying that most of the SDK's surface area remained unchanged. A follow-up patch, v0.12.1, was released five hours later after SSE and streamable-HTTP transports were found broken in v0.12.0, a gap missed because the test suite only exercised the stdio transport. The package had been growing rapidly, reaching over 2,100 monthly downloads by late August 2026, making the brief breakage window more consequential than it might otherwise have been.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Microsoft Foundry Toolbox Centralizes AI Agent Tool Management via Single MCP Endpoint

Microsoft has introduced Foundry Toolbox, a governed service layer that consolidates multiple AI agent tools — such as CRM, Work IQ, and Web Search — behind a single Model Context Protocol (MCP)-compatible endpoint. Previously, each agent had to individually configure tools, credentials, and approval policies; Toolbox shifts that responsibility to a centrally managed, versioned layer. An accompanying Tool Search feature uses BM25-based intent matching to hide most tools from the model's initial context by default, exposing only a small candidate set of up to five results per query, which helps reduce context size. Rather than replacing subagents, Toolbox functions as a reusable integration layer accessible not just to Foundry agents but also to any MCP-compatible runtime, including Microsoft Agent Framework and LangGraph. Microsoft cautions that not all direct tools are automatically compatible with Toolbox and advises users to verify the supported-tools matrix, regional availability, and authentication requirements before migrating.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Silent LLM Agent Failures Expose Gaps in Unattended AI Fleet Monitoring

A production LLM agent fleet suffered a series of undetected failures over several days in late August and early September, including process errors and multi-minute timeouts involving Ollama Qwen models. The failures, which lasted up to 1.8 million milliseconds in some cases, were traced to model instability, resource exhaustion, and the absence of real-time monitoring. Because the agents were running unattended, no alerts were triggered and the issues persisted without intervention. In response, the team introduced automated alerts, CPU and memory usage tracking, regular model health checks, and failover mechanisms to backup agents. The incident underscores that standard uptime monitoring is insufficient for AI agent fleets, which require deeper, behavior-aware observability.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Fixed Resize Should Be Default for Thumbnails; Use Smart Crop Only When Needed

A technical guide for e-learning platforms recommends using fixed resize as the default method for course thumbnail pipelines, reserving content-aware cropping only for image classes that fail a framing evaluation. Fixed resize preserves the original composition and scales it to fit a target box, while content-aware cropping selects a focal region before scaling — both can produce identical file sizes if implemented carefully. Storage bloat typically results not from the crop algorithm itself but from retaining multiple variants, regenerating outputs after policy changes, or leaving outdated objects accessible. The guide warns that arbitrary dimension inputs can create redundant cache keys and inconsistent thumbnails across surfaces like lesson search, course outlines, and recommendation rails. A disciplined approach — normalizing requests to named variants, labeling failure types, and only adding smart cropping where it solves a specific problem — is recommended to keep pipelines predictable and maintainable.

0 comments Read more at DEV Community