Aivora
AI Daily中文
arXivAI AgentAdvanced

Grow the Harness, Not the Context: Building Low-Cost Specialist Agents via Failure-Guided Code Synthesis

別撐大脈絡，改建構程式框架：Growing Harness 透過「失敗導向學習」自動生成高效能 Agent

2 min read
中文English
Grow the Harness, Not the Context: Building Low-Cost Specialist Agents via Failure-Guided Code Synthesis
The 30-second version

Traditional LLM agents waste context and budget by repeatedly making the same control decisions inside each task's context. Growing Harness solves this by starting from a "strategy-free scaffold" and growing executable harness code from task feedback. When a task fails, function-level traces localize the failure, an optimizer jointly repairs the code, and a success-first rollback gate prevents regressions. Over time, recurring control isCompiled into deterministic code. This approach slashes LLM calls by 76.0-91.8% and costs by up4 to 98.6%, enabling a 4B model to match the 45% success rate of 120B models on WebArena.

Key points

01

Code Over Context

Shifts recurring control decisions from the LLM's context into reusable, executable code, reserving LLM calls strictly for semantic reasoning.

02

Failure-Guided Joint Repair

Localizes failures to specific code surfaces using execution traces, allowing an optimizer to repair multiple failures jointly.

03

Regression-Preventing Gate

Implements a success-first held-out gate that automatically rolls back code edits if they harm prior task capabilities.

04

Empowering Smaller Models

Keeps success rate stable at around 45% on WebArena even with a 4B model, whereas traditional tool-calling drops to 6.7% at that scale.

How it works

Growing Harness Failure-Guided Code Learning Loop
InitializeOn failureSend fail windowPropose editsPassed: AccumulateFailed: RollbackUpdated harnessStrategy-Free ScaffoldFailure TraceLocalizationCode Joint OptimizerHeld-out Rollback GateTask Execution &TracingGrowing Shared Harness

Why it matters

This research breaks the bottleneck of agents relying on long system prompts and giant models. By compiling agent control flow into code, it dramatically frees up context and slashes inference costs by up to 98.6%. It demonstrates that enterprises can deploy highly stable, low-latency specialist agents using cheap, edge-sized models (like 4B parameters), removing key barriers to commercial agent deployment.

Who it affects

  • AI Developer
  • AI Researcher
  • Enterprise Leader

How to use it

  1. 1Cost-effective web automation: Running complex browser tasks using lightweight local models instead of expensive API calls.
  2. 2Domain-specific agent workflow optimization: Automatically generating optimal control and error-handling code structures for specialized software tools.

Limitations & caveats

  • Bootstrap dependency: Requires a pre-defined strategy-free scaffold with fixed tool interfaces to initiate the learning process.
  • Optimizer dependency: The ceiling of the agent's ultimate capability is bounded by the coding capabilities of the generator and optimizer used.

Related