SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Harness Engineering: Why the Software Around AI Models Matters More Than the Model

0
·1 views

A study published on arXiv in August demonstrated that keeping the same AI model but changing only the surrounding agent system improved bug-fixing performance on SWE-bench Verified tasks from 43 to 72 out of 169. This surrounding software infrastructure is called a 'harness,' a term gaining traction in AI development circles, defined simply as Agent = Model + Harness. The harness controls everything beyond the model itself, including tool access, context management, task loops, and output verification. The field has evolved through prompt engineering and context engineering, with harness engineering now encompassing both while adding the infrastructure needed for a model to take actions. An emerging next layer called loop engineering wraps the harness in automated outer loops that re-run agents based on schedules or events without requiring manual input.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

WebForms Core 2.2 Development Begins With Focus on Stateless Server-Orchestrated UI

Development of WebForms Core 2.2 has officially begun, targeting an advanced stateless server-orchestrated UI model that differs from frameworks like Blazor Server by not maintaining persistent component or DOM state per user. The server uses a WebForms class to generate commands executed by WebFormsJS directly against the browser's HTML document, keeping the server stateless and scalable. Version 2.2 aims to elevate this model by introducing higher-level operations for rendering, state management, data formatting, arithmetic, and extensibility without relying on a separate frontend framework. New features include Isole, a high-level abstraction that simplifies working with the Transient DOM by automatically handling initialization and completion of grouped operations. Additionally, snapshot and rollback capabilities are being introduced to allow developers to preserve and restore an element's HTML state during complex multi-step UI transformations.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Google Expands Gemini in Chrome With Cross-Tab Context and Android Auto Browse

Google has expanded its Gemini AI assistant within Chrome to offer context-aware help across open browser tabs, integrating with services like Gmail, Google Calendar, Google Maps, and YouTube. The desktop and iOS rollout reached India, New Zealand, and Canada on March 11, 2026, with an Android update following on May 12, 2026. The Android version introduces Auto Browse and agentic capabilities, enabling Gemini to assist with multi-step tasks rather than simply answering questions. Key features include chatting about the current tab, extracting highlights from videos, and comparing information spread across multiple open pages. Google describes the broader goal as turning Chrome into an active working layer between information and action, reducing the need to manually switch between tabs or apps.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Why AI Confidence Scores Need Calibration Testing Before Production Use

A technical deep dive published on DEV Community examines why confidence scores returned by AI decision systems like Jev cannot be trusted at face value without calibration testing. Calibration differs from accuracy: a system can be 95% accurate overall yet still report 90% confidence on decisions it gets right only 60% of the time. This gap is especially dangerous in confidence-gated routing systems, where high-confidence decisions are auto-approved and low-confidence ones are escalated to humans. The article explains two standard diagnostic tools — reliability diagrams and Expected Calibration Error (ECE) — that developers can implement in roughly 40 lines of Python. The core takeaway is that calibration is domain-specific and must be measured by each team on their own data, as no vendor or framework can guarantee it in advance.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer builds agent that distinguishes pygame from pygame-ce documentation accurately

A developer has built a documentation agent to address a persistent confusion between pygame and pygame-ce, two incompatible Python game library distributions that share the same namespace but have diverging APIs. The agent was submitted to the Sanity Challenge and uses Sanity's MCP endpoint to query the API reference pages of both libraries — 42 pages each — without blending their content. Every response explicitly names which distribution a feature belongs to, the version it appeared in, and its source entry. The agent appends machine-readable verdict lines to each answer, clearly flagging whether a function exists in pygame, pygame-ce, or both. This approach tackles a common pain point where tutorials and AI-generated code snippets rarely specify which distribution they target, leaving developers with cryptic AttributeError messages.

0 comments Read more at DEV Community