ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents
06:00 · July 1, 2026 · arXiv cs.AI RSS
Production large language-model (LLM) agents are increasingly deployed not as lone problem-solvers but as managers: a main model creates specialized subagents, delegates work, and orchestrates their parallel, asynchronous returns through dynamic workflows. Whether one model can actually run such a team is largely unmeasured: existing benchmarks score a policy's own task-solving or a fixed multi-agent system's emergent behavior, but none isolate the management ability of the single LLM acting as leader. We introduce ClawArena-Team, a benchmark of 41 multi-turn, multimodal, multi-directory scenarios spanning 258 evaluation rounds and 72 staged updates that measures this management ability. The main agent is deliberately constrained: it natively perceives only text and directly accesses only part of the workspace. It commands a fixed, locally served subagent pool, so score differences reflect management skill, not raw capability. All scoring is execution-based with no LLM judge: an overall score -- the Subagent-Management Score (SMS) -- multiplies task correctness by a least-privilege and modality-routing factor. Across twelve proprietary, community-hosted, and self-hosted models, experiments show that the management bottleneck is privilege granting rather than perception (no model exceeds 50% workspace-permission precision); that cost and management quality are decoupled (API cost spans over 100 times while the overall score spans under 4 times, with the cheapest open models on the Pareto frontier); and that most leaderboard scores cluster within a 9.9-point band while orchestration behaviors diverge by more than an order of magnitude. Code and data will be released.
Summary
Production LLM agents increasingly function as managers rather than solitary solvers: a primary model must instantiate specialized subagents, delegate tasks, and coordinate their parallel and asynchronous outputs across evolving, multimodal workspaces. ClawArena-Team isolates this management capability by presenting a single text-only main agent with 41 multi-turn scenarios that span 258 evaluation rounds and 72 staged workspace updates. The agent perceives only text and can reach only part of the workspace directly, forcing it to create, empower, and schedule subagents drawn from a fixed, locally served pool whose underlying capabilities remain constant across trials.
All evaluation is execution-based and does not rely on an LLM judge. The composite Subagent-Management Score multiplies task correctness by separate factors that penalize excessive privilege grants and incorrect modality routing. Experiments across twelve proprietary, community-hosted, and self-hosted models reveal that privilege management constitutes the dominant bottleneck: no model exceeds 50 percent workspace-permission precision. In contrast, differences in raw perception contribute comparatively little to score variance.
The benchmark further shows that management quality and inference cost are largely decoupled, with API costs varying by more than two orders of magnitude while overall scores span less than a factor of four; several low-cost open models lie on the Pareto frontier. Although aggregate leaderboard scores cluster within a narrow 9.9-point band, the underlying orchestration behaviors diverge sharply, with forbidden-access rates differing by roughly an order of magnitude among otherwise comparable systems. The authors release the full benchmark code and evaluation data to support controlled comparisons of subagent orchestration.
Why it matters
Provides a reproducible, technically rigorous benchmark directly usable by Dutch AI research teams working on agentic systems. Novel focus on management skill rather than raw capability aligns with Netherlands strengths in ethical and transparent AI. High technical depth and planned open release support advanced readers and practitioners.
