The cost of keeping the thinking

By mid-2026, several organizations running agentic AI coding tools at scale reported the same pattern: token spend consumed budgets sized around per-seat software pricing. Microsoft canceled most internal Claude Code licenses effective June 30, 2026. The pilot had launched in December 2025 across roughly 5,000 engineers; within months it consumed the team’s full annual AI budget, and usage was redirected to GitHub Copilot CLI. Microsoft’s $5 billion Foundry investment in Anthropic and Anthropic’s $30 billion Azure compute commitment were unaffected by the change. The cancellation was a cost-control decision inside one team, not a break in the companies’ relationship.

Uber’s engineering organization exhausted its entire annual AI coding-tools budget in four months, according to its CTO (The Information, April 2026). Individual engineers were reported spending $500 to $2,000 a month on tokens, against an estimated 70% of newly committed code being AI-originated.

NVIDIA vice president Bryan Catanzaro described the shift to Axios in similar terms: for his team, the cost of compute now exceeds the cost of the employees using it. The pattern sits inside a larger trend: Gartner projects worldwide AI spending of roughly $2.5 trillion in 2026, up 47% year over year, and Goldman Sachs projects roughly 24 times token-consumption growth by 2030, to an estimated 120 quadrillion tokens a month.

An API-metered model is billed by the token, and the bill is a function of how much the model has to process, not how many people use it. A single chat exchange sends a modest amount of context once. An agentic workflow calls the model repeatedly toward one outcome, and a language model retains nothing between calls: each call either resends the accumulated context or has the model reconstruct it from what is available. Without a cache or an external memory, that reconstruction is billed as fresh input on every turn.

This is the mechanism behind the cost pattern described above. The organizations in that pattern did not necessarily use more tokens per task than expected. They used the model in an agentic, multi-call pattern, and paid the fresh-input rate for context that had already been processed earlier in the same task, or in an earlier one.

Anthropic’s prompt cache is native to the API: a request marks content as cacheable, the first read writes the cache, and later reads within the cache’s lifetime are billed at a fraction of the fresh-input rate. Published pricing sets a cache write at 1.25 times the base input price for a 5-minute cache or 2 times for a 1-hour cache, and a cache read at 0.1 times the base input price (a 90% discount). A cache hit also refreshes the cache’s remaining lifetime.

Independent measurement supports the scale of that saving in practice. PwC evaluated prompt caching across OpenAI, Anthropic, and Google over 500 agent sessions using 10,000-token system prompts (January 2026) and found API cost reductions of 41–80% and time-to-first-token improvements of 13–31% across all three providers.

The saving is bounded by the cache’s lifetime: 5 minutes to 1 hour after the last read. A session that goes quiet past that window, or a new session started the next day, begins again without it. Nothing about the mechanism carries context past that boundary. That is a design property of a session-scoped cache, not a shortcoming particular to any one implementation.

Substrate memorydoes not change how Anthropic’s cache works, replace it, or run alongside it as a second cache. It is a store, external to the model and to any one session, that a new session can read from at its start. Read early enough in a session, that context becomes part of the same stable prefix the cache already discounts, so the session’s cache initializes against a project state that reflects what earlier sessions already established, rather than nothing.

The same architectural gap recurs at more than one scale of the work. The full architectural comparison, developed independently of the cost framing here, is set out in horizontal coordination.

The cache lowers the cost of one session’s repeated reads. A substrate is aimed at the cost of the project’s lifetime: preventing each new session from starting without what earlier sessions already worked out.

The following figures come from Vinculum’s own production telemetry, not from a model or an estimate. They describe one system’s usage; they are not a projection of what any other system or workload would show. Source: a single 20-hour Opus 4.8 session, recorded June 2026.

If the same 3.8 billion cache-read tokens had been billed as fresh input instead of cache reads, the input portion alone would have cost approximately $57,131 (a 97.3% saving versus that counterfactual). Per turn, the session’s actual cost worked out to $0.18, or roughly $78 an hour of Opus 4.8 use across the full 20 hours.

Measured across a longer, cumulative window, the same pattern holds:

An independent measurement outside Vinculum, PICHAY (arXiv 2603.09023), found a comparable pattern at a different scale: 21.8% of tokens across 857 production agent sessions, totaling 4.45 million effective input tokens, were structurally repeated or otherwise unnecessary rework, as confirmed in the paper’s abstract.

Applying that 21.8% waste figure to Anthropic’s own reported inference-compute spend, projected at roughly $32 billion for 2026–2029 (analyst and press figures, not independently re-verified here), produces a range of modeled, not measured, savings scenarios: approximately $5.4 billion over four years under a conservative recovery assumption, approximately $10.2 billion under a moderate one, and $20–25 billion under an aggressive one. These are models built on a published spend estimate and a measured waste fraction from an unrelated system; they are not a forecast of what any one deployment, including Vinculum’s own, would save.

The cost pattern described above is real and independently reported. What is not known, and not claimed here, is what any of the organizations named in it would have spent with a different architecture. Their token totals, workloads, and engineering practices are not public, and a substrate is not a drop-in replacement for the tools they were using. A specific dollar figure attributed to a company that has not measured it would not be a case study; it would be a fabrication in a case study’s shape.

The claim that does hold is narrower and checkable: a session-scoped cache and a persistent substrate address different parts of the same problem, Vinculum’s own measured cache-read share and session receipts are given above as one system’s existence proof, and a reader assessing their own exposure can apply the same measured-versus-modeled distinction to their own workload rather than borrowing a number that describes someone else’s.