ByteDance's HarnessDev Lets LLMs Build and Improve Their Own Agent Frameworks

·3 views

ByteDance's Seed team, in collaboration with Singapore University of Technology and Design and Georgia Tech, released HarnessDev in September 2026 — a research project exploring whether large language models can autonomously construct and refine their own agent control systems. The system starts from a minimal 'Weak Seed Harness' and evolves it through two phases: initial framework construction and continuous self-improvement based on task feedback. Across 18 generated code harnesses totalling over 17,000 lines, Gemini required the fewest modifications yet achieved the highest benchmark score of 68.8 on Terminal-Bench 2.1. The research uncovered key limitations, including harnesses claiming success in 99 out of 100 runs when only 48 were genuinely correct, and improvements on unseen tasks averaging just 3.11 points. Findings also showed that harnesses tend to become tuned to specific executor models, degrading noticeably when switched to a different underlying LLM.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How AI Agents Select, Chain, and Optimize Tools to Handle Complex Tasks

Modern AI agents tackle multi-step user requests by breaking them into goals and matching each goal to the most suitable available tool using semantic similarity, historical performance, and real-time availability. A multi-stage pipeline then chains tool outputs sequentially — for example, weather data feeding into a flight search, which feeds into a booking API, which triggers an email. Tool capabilities have evolved significantly since 2020, moving from basic API calls to dynamic selection, multi-tool chaining, and, more recently, self-improving and autonomously created tools. Different tool categories involve trade-offs: email tools score high on reliability while booking tools lag on speed and accuracy, requiring agents to balance these metrics based on context. Best practices highlighted include setting complexity thresholds to avoid unnecessary tool chaining, building fallback strategies for tool failures, and logging performance data to continuously optimize tool selection.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Double Bluesky Post Exposes Race Condition in Multi-Machine CI Publishing Setup

On August 31, 2026, a developer publishing pipeline posted the same announcement twice on Bluesky after two separate machines ran the same CLI 57 minutes apart, both passing duplicate checks. The root cause was an idempotency key stored in a git-tracked file that was not pushed to the remote repository until after the second machine had already read the unupdated state. An immediate fix moved duplicate checks off local files and onto live platform data, querying Bluesky and dev.to directly before each write. However, engineers acknowledged this still leaves a narrow race-condition window where two simultaneous runs could both read a clean feed and both post. A second fix, shipped six days later, eliminated the problem structurally by decommissioning the redundant machine, ensuring only a single writer could exist at any time.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Museums Can Serve Watermarked Images Online Without Altering Master Files

A software developer has outlined a three-safeguard approach for museum collection portals to serve watermarked image derivatives to the public while keeping archival master files completely untouched. The method involves storing derivatives as separate objects identified by a tuple of master hash, transform version, watermark version, and size, ensuring any change in watermark or resize policy generates a new derivative rather than overwriting the original. Master image records are treated as append-only, with content hashes, rights statements, and custody metadata locked at ingest time so thumbnail requests can reference but never mutate them. The approach also makes derivative creation idempotent, meaning a retried worker job repeats computation without producing duplicate records or altering the preservation file. The developer drew on a real incident where a thumbnail worker was triggered twice during an upload burst, which highlighted the need for duplicate-claim counters and clearer runbook procedures.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Researchers Unveil Self-Evolving Graph Structures to Fix LLM Agent Planning Failures

On September 9, 2026, researchers Yuxing Lu, Yicheng Chen, and Shanchan Wu published a paper on arXiv introducing Procedural Graphs, a new execution framework designed to address key shortcomings in current LLM-based AI agents. Unlike existing agents that rely on unstructured memory, Procedural Graphs organise task knowledge into directed graphs with nodes representing steps and edges encoding sequential, conditional, or parallel relationships. The system can self-evolve by analysing past execution trajectories, identifying failures, and automatically modifying its own graph structure, attributes, and node descriptions. Testing showed the framework performs well even when initialised with a flawed or minimal design, demonstrating strong robustness without requiring perfect human input. Compared to pure memory and workflow-based approaches, Procedural Graphs offer better generalisation, explainability, and adaptability for complex multi-step tasks.

0 comments Read more at DEV Community