Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale
06:00 · August 4, 2026 · arXiv cs.AI RSS
AI coding agents like GitHub Copilot, Claude Code, and Codex interleave multi-step LLM inference with tool execution, creating a workload different from chatbots. We present the first production-scale characterization of this workload using sampled GitHub Copilot traces from June 2026, comprising 3.2M users, 13M sessions, 761M LLM calls, and 95T tokens. Our analysis reveals distinctive workload properties with important systems implications. For example, agentic coding sessions consist of sparse user-initiated turns, each unfolding into an autonomous agent loop of LLM calls almost always coupled with tool execution. This structure yields KV cache hit rates averaging 90% within a turn, but falling to 55\% across turn boundaries and drastically invalidated after events like model switches or context compaction. Diverse workflows and user behaviors are observed with variable and long-tailed token consumption, time span, and tool calls. We highlight the difference between quick agentic turnaround times and the minutes-long user idle periods at turn boundaries, and design a lightweight idle-time predictor that captures 86-90\% of total idle time, enabling proactive decisions for efficient resource orchestration. These findings challenge assumptions underlying current LLM-serving systems and provide an empirical foundation for agent-native infrastructure.
Summary
The paper delivers the first large-scale empirical characterization of production AI coding agents, based on sampled GitHub Copilot traces collected over one week in June 2026. The dataset covers 3.2 million users, 13 million sessions, 761 million LLM calls, and 95 trillion tokens, together with 775 million tool invocations. Unlike single-turn chat or code-completion workloads, these sessions consist of sparse user prompts that each launch an autonomous agent loop in which LLM calls are coupled nearly one-to-one with tool executions; 87 percent of all LLM invocations are agent-initiated rather than user-driven.
The traces expose a session-structured KV-cache lifecycle. Prefix caching achieves average hit rates of 90 percent within a turn, yet these rates fall to 55 percent at turn boundaries and collapse to 8 percent after model switches. Context compaction, triggered in 7.8 percent of sessions but affecting 44 percent of total tokens, produces similarly severe cache resets. Tool failures occur in 9 percent of turns and trigger retry loops that can increase compute demand by up to four times, while five distinct user archetypes exhibit a fifty-fold range in per-turn token consumption.
Execution alternates between GPU-bound inference and CPU- or IO-bound tool phases, creating bimodal idle periods: seconds within turns versus minutes at turn boundaries. The authors present a lightweight idle-time predictor that captures 86–90 percent of aggregate idle time, offering a practical signal for container reclamation and KV-cache eviction or offloading. Collectively, the measurements challenge assumptions embedded in current LLM-serving systems and supply concrete data for the design of agent-native infrastructure.
Why it matters
High technical depth and novelty on agentic LLM serving directly informs Dutch AI researchers and infrastructure teams building or optimizing coding agents and efficient GPU/ KV-cache systems in an EU context.
