A survey of agent runtimes · 2026

Agent Harness for Large Language Model Agents: A Survey

Qianyu Meng, Yanan Wang, Liyi Chen, Wei Wu, Yihang Li, Wenyuan Jiang, Qimeng Wang, Chengqiang Lu, Yan Gao, Yi Wu, Yao Hu

Minzu University of China · Dalian University of Technology · Xiaohongshu Inc. · University of Science and Technology of China

Agent capability should be analyzed at the model–harness–environment configuration level. The harness is a consequential and frequently under-reported co-determinant of observed behavior.

Read Paper v4 DOI · archived v3 Paper v5 in revision

Agent harness architecture: execution, tools, context, state, lifecycle and evidence — The model, harness, and environment form a feedback loop of observation, inference, and action. Context management selects model inputs; execution and tools translate outputs into actions; persistent state connects successive turns. Lifecycle hooks intervene at execution boundaries, while the evaluation interface collects events and trajectories. Together, these mechanisms explain how runtime design shapes agent behavior.

The paper

A runtime layer made visible

The survey brings together first-party system evidence, controlled harness evaluations, and lifecycle studies to examine how the model, harness, and environment shape observed capability.

Explore the living evidence index

Analytical framework

Six functions. One runtime.

ETCSLV organizes six recurring runtime responsibilities and their interactions: execution, tools, context, state, lifecycle controls, and evaluation.

Illustrative tool-using execution path — A tool-using execution path connects condition checks, authorization, dispatch, execution, and result commitment, with retry and recovery following failure. The lanes relate these stages to ETCSLV responsibilities, showing how a model decision becomes a traceable and recoverable action.

Historical evolution

Where runtime responsibilities come from

System landscape

Evidence-coded system comparison

Thirty-four systems assessed against the same six functions. The matrix cutoff is 2026-10-12; per-source access dates are shown in the evidence details. Ratings do not measure quality, adoption, or production maturity.

Four architectures, four design questions

Claude Code: Claude Code integrates repository exploration, editing, commands, and delegated work into a coding workflow. It shows how context selection and action approval shape the path from a user goal to a tested artifact.
Codex: Codex connects model and tool events through an execution loop, persisted rollouts, and approval and sandbox controls. It makes the relationship between execution, recoverable state, and client interfaces central to architecture analysis.
DeepSeek Harness: DeepSeek Harness exposes runtime components through plugins, profiles, and patches. Typed tools, session events, and replaceable services make it a case for studying component boundaries and controlled runtime modification.
OpenClaw: OpenClaw coordinates messaging channels, persistent sessions, and scheduled work through a gateway. It extends the comparison to agents whose identity, state, and authority remain relevant across interactions and over time.

Five runtime comparison groups and four adjacent analysis units — The landscape distinguishes five groups of runtime systems from four adjacent research units. The former constitute the 34-system comparison; the latter supply capability modules, protocol and sandbox infrastructure, evaluation infrastructure, and task environments. This distinction connects complementary contributions while clarifying the unit of comparison.

View the full component-coverage figure

Documentation-backed component coverage for 34 systems — Documented coverage of execution, tools, context, state, lifecycle, and evaluation interfaces across 34 systems. F, P, A, and ? denote Full, Partial, Absent, and Unverified. Rows show each system’s allocation of responsibilities; columns support comparison of the same function across systems. Supporting sources appear in the table below.

Find a system

FFull · built in and documented PPartial · limited, optional, or delegated AAbsent · explicitly not provided UUnverified · public evidence insufficient

System	E	T	C	S	L	V	Evidence

The comparison set is evidence-driven, not a ranking. Expand a row to inspect first-party sources, version identifiers, confidence, and access date for each component.

Download the matrix data (JSON)

Cross-cutting engineering

Tools and collaboration carry runtime obligations

Tool governance across the execution loop

Tool governance across the execution lifecycle — Tool governance spans registration, selection, dispatch, execution, result handling, and recovery. Interface contracts, authorization, isolation, and result checks appear at their respective execution stages, showing how reliable tool use depends on coordinated controls throughout the action path.

Coordination patterns and responsibility boundaries

Multi-agent coordination patterns and runtime responsibilities — Multi-agent organization determines how tasks are delegated, information is exchanged, and results are combined. The diagram compares identity, authority, shared-state, and recording responsibilities across coordination patterns, connecting topology to runtime design.

Evaluation landscape

Six areas of harness evaluation

Six research questions organize the evaluation literature, connecting benchmark suites, evaluation methods, and the infrastructure needed to run them.

Three sources of evaluation attribution risk — Environment changes, task ambiguity, and harness configuration shape the interpretation of evaluation results. The diagram connects these attribution problems to controls and diagnostic methods, explaining why agent comparisons need to specify model, runtime, and environment conditions together.

Task environments such as OSWorld, WebArena, GAIA, and SWE-bench remain essential. They measure configured-system task performance; a harness-effect study adds controls that isolate runtime changes.

Literature update

Selected research updates

Seven studies added on 18 October 2026 cover benchmarks, methods, implementation testing, and security.

Harness evolution

How harnesses change through experience

Harness evolution operates on different objects: reusable skills, executable runtime code, or the model and runtime together. The survey compares what changes, which feedback guides the change, and how improvements are evaluated.

Work	Object of change	Role in the comparison
SkillOpt	Reusable skill text	How can rollout feedback guide persistent skill edits?
Meta-Harness	Executable harness code	How can traces and task scores guide runtime implementation search?
AutoHarness	Task-specific harness and policy code	How can environment feedback guide action-validating code?
EvoHarnessBench	Changing tools, skills, and agent configurations	How are adaptation and retention measured across stages?
TaoLive	Model weights and runtime components	How does model training interact with a changing harness?

Inspect the Meta-Harness code-optimization loop

Reported Meta-Harness code-evaluation-history feedback loop — Meta-Harness connects candidate code, task evaluation, execution traces, and accumulated history in an optimization loop. It illustrates one route to harness evolution: translating execution experience into persistent implementation changes and assessing those changes in subsequent evaluations.

Research agenda

From component coverage to testable effects

The research agenda moves from documenting components to testing how their interactions shape task performance, state continuity, authority, and adaptation.

Citation

Paper versions and citation

The repository PDF is v4; the DOI points to archived v3; v5 is in revision.

Open DOI record

BibTeX

@article{meng2026agentharness,
  title     = {Agent Harness for Large Language Model Agents: A Survey},
  author    = {Meng, Qianyu and Wang, Yanan and Chen, Liyi and Wu, Wei and
               Li, Yihang and Jiang, Wenyuan and Wang, Qimeng and
               Lu, Chengqiang and Gao, Yan and Wu, Yi and Hu, Yao},
  year      = {2026},
  doi       = {10.20944/preprints202604.0428.v3},
  url       = {https://www.preprints.org/manuscript/202604.0428/v3},
  publisher = {Preprints}
}