Coding Agents Are Not Chatbots

Microsoft and collaborators traced 13.5 million real GitHub Copilot coding-agent sessions from June 2026 — 3.2 million users, 761 million model calls, 95 trillion tokens — and published the first production-scale measurements of how coding agents actually work. The paper is “Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale” (Liu, Qiu, Goiri, Fonseca, Bianchini, Choukse — arXiv:2608.00101).

The headline is simple. A coding agent does not behave like a chat request. Treating it like one wastes the signals the system already has.

Finding

Coding agents extend a language model with tools — read files, search the repo, edit code, run builds, read the results, try again. One user instruction can trigger a chain of dozens of model calls interleaved with tool calls, running for minutes or hours with little supervision.

Across the sampled traces that chain has a clear shape.

• Most of the work is not user-initiated. The median session has 3 user turns, 15 model calls, and 13 tool calls, lasting 4.2 minutes. The mean is 6.1 turns, 40.6 calls, and 62.6 minutes. A single user prompt fans out before returning. 87% of model calls are started by the agent itself, not the user.

• One thought, one action, tightly coupled. Model calls and tool calls track each other almost 1:1. Most model calls make a tool call. Most tool results immediately trigger another model call. The pattern is observe → decide → act, on repeat.

• Long-tailed and varied. Six recurring workflow types appear: deep exploration (30.5%), pure reasoning with no tools (20.2%), read-edit-build loops (19%), other multi-round exploration (13.2%), terminal-heavy runs (8.1%), and extended retry loops after failures (9.1%). That last group averages 36 model calls per turn, four times the median.

• Input-heavy, output-light. The median model call reads about 68,000 tokens and writes about 247. The input-to-output ratio is roughly 275:1. About 48% of the input is conversation history, 28% is tool outputs, 14% is the system prompt. Context grows because the agent accumulates its own history.

• Memory reuse follows structure, not just policy. Prefix caching — reusing what the model already read — averages about 90% within a session at the median. The trajectory inside a turn is predictable: roughly 45% reused on the first call (cold start), 86% on the second, 92 to 94% from the third onward. At a turn boundary, where the user may pause to think, it falls to about 55% for the same model and to 8% if the model switches. Model switches affect about 6.4% of sessions and are mostly reactive to rate limiting or errors, not user choice.

• Time between work comes in two sizes. Within a turn the median pause while a tool runs is about 1.2 seconds for cached state and 5.8 seconds for the container. Between turns, where the user is idle, it is about 172 seconds and 243 seconds. That gap is large enough to predict. With only turn- and session-level features, a lightweight predictor captures 86 to 90% of total idle time.

• Two additional costs are concentrated. Context compaction — rewriting history when the prompt nears its limit — happens in only 0.5% of calls, but in 7.8% of sessions and 44% of all tokens processed by those sessions. It discards over 70% of the prompt and resets cached state. Tool failures happen in 9% of turns and can trigger retry loops that inflate compute up to four times.

The serving systems these agents run on, like vLLM and SGLang, were designed for chat. Chat assumes short, independent, stateless requests. These traces show the opposite — long-lived, sequential, stateful chains.

Meaning

The useful way to read this is not as a performance tuning report. It is as a statement about what the unit of work should be.

In chat, the request is the right unit. Each message stands alone, carries its own context, and can be scheduled anywhere. Caching helps but is not load-bearing.

In coding agents, the turn and the session are the right units. A turn is one user instruction plus the full autonomous chain that follows it. A session is all turns from one coding task. State accumulates inside that unit and predicts what happens next. Where the agent is in its chain predicts whether cached state will be reused, whether the next call will be a retry, and whether the container will sit idle for seconds or minutes.

Reusing that state saves work that is large relative to the output. When the median call reads 68,000 tokens to write 247, avoiding a re-read is the main optimization. Within a turn the system can keep almost everything. At a turn boundary the cache has the laziness of an LRU — it is kept briefly, then dropped after a few minutes of user think time. The paper estimates that plateau and the cliff between roughly 2 and 10 minutes of idle time. If the system throws that away too early, it pays to re-read tens of thousands of tokens on the next turn. If it holds it too long, it holds memory that may never be used while the user is idle.

Switching models is worse than waiting. A cache built for one set of weights cannot be reused by another. A switch is a full cold start on top of the turn-boundary loss. The fact that most switches in the traces are automatic downgrades after hitting a rate limit means the system is often causing its own worst cache miss by moving the session to make room, then paying to rebuild what it just evicted.

The same structure explains retry cost. In chat a failed tool call is an error to show the user. In an agent loop a failure is an instruction to try again with more context. Each attempt grows the prompt. Four attempts is four times the input billed to the next call. Tool reliability is not just a quality problem. It is a serving cost multiplier.

The predictor result matters because it turns these observations into an operational signal. If turn boundaries predict minutes-long idle periods and within-turn gaps predict seconds-long ones, the system can decide differently for each — keep hot state inside a turn, offload or reclaim between turns, and do it before the idle time is fully elapsed. Capturing 86 to 90% of idle time with a small model means the signal is strong enough to act on without a complex classifier or perfect knowledge of the workflow.

What remains open is the usual constraint. These are sampled traces from one week in June 2026, from GitHub Copilot, across three US time zones. The absolute numbers generalize less than the structure does. Specific cache sizes, rate limits, and compaction thresholds will differ per deployment. The structure — sparse user turns fanning into autonomous 1:1 model-tool chains, input-heavy calls, high within-turn reuse with predictable turn-boundary decay — is likely durable across coding agents, because it follows from how agentic work is organized rather than from any one implementation.

Connection

The lay version has a physical analogue. Chat is like serving short orders at a counter. Each customer states their order, you make it, you forget it, the next customer arrives unrelated to the last. You optimize for making any single order fast.

A coding agent is like a workshop job. A customer asks you to fix a machine, you open the case, look up the manual, fetch a part, test, find it still fails, try another part, re-test, write up what you did. Each step depends on what you just learned and what you have spread on the bench. The cost is not the final test result you hand over. It is all the parts and manuals you laid out to be able to do the next step quickly.

Current serving treats the workshop like the counter and clears the bench between each small action. What the traces quantify is the price of clearing it. Keep the bench for the duration of the job and you save almost all re-reading. Clear it between jobs and you must relay everything. Swap benches mid-job and you must start over.

This is also a familiar maintenance problem. A system that removes one kind of friction creates another. Cheap code browsing and automated edits remove the friction of finding files and typing patches. They create the friction of long, stateful, input-heavy chains that must be scheduled, cached, and reclaimed without breaking the chain that gave them value. Efficient operation is not about making each step faster in isolation. It is about recognizing that the chain is the job, keeping its working set alive while it is in motion, and releasing it promptly when the user steps away.

That shift — from scheduling requests to scheduling workflows — is straightforward to state and consequential to implement. The paper does not argue that agent-native infrastructure will require new model architectures. It argues that workflow state, especially cached input and container lifetime, should be treated as schedulable resources with turn and session lifetimes, because request-level policies discard the strongest predictors the workload already provides.

---

Source: Banruo Liu et al., “Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale,” arXiv:2608.00101, 30 July 2026. arXiv / PDF