The Token Sink

Executive briefing

The investment case for industrialising large language models inside the enterprise rests on a substrate the engineering tradition cannot wrap. Every prior infrastructure layer — the operating system, the network stack, the cloud — has been wrapped in a tooling cycle that renders it shelf-stable for production: containers, retry semantics, eval frameworks, reference architectures. The LLM substrate has two missing layers that foreclose this tradition. The first is an architectural separation between code and data: every byte entering the context window is processed by the same attention mechanism, with the same power to redirect what the model does next. Prompt injection is not a vulnerability that vigilance will fix; it is the substrate’s defining property. The second is a separation between correctness and appearance: post-RLHF training rewards the model for outputs human evaluators approve of, not outputs that are correct. A controlled study by Wen et al., published at ICLR 2025, documented that current models produce code which passes evaluator-designed unit tests while being substantively wrong. The wrapping layer the engineering tradition would reach for first — automated evaluation — is the wrapping layer that fails first, because the inspector and the inspected have been optimised by the same reward signal. The AI vendor’s business case depends on this not getting fixed.

The economic mechanism that monetises the substrate is the migration from per-seat to consumption pricing. Anthropic restructured its enterprise contracts through a rolling change beginning with November 2025 renewals; OpenAI’s Codex made the same move in April 2026; IDC forecasts seventy per cent of vendors will be on consumption metrics by 2028. The seller’s reasoning is sound: variable infrastructure cost per user can diverge by orders of magnitude depending on prompt patterns, agent depth and reasoning-token consumption. Per-seat pricing made the seller carry that variance. Token pricing inverts the deal. Three multipliers compound on the buyer’s side. Reasoning models routinely consume eighteen times the tokens of standard models on the same task — for tasks where the additional reasoning produces no measurable accuracy gain. Agent loops expand consumption quadratically per turn. Run-to-run variance for the same input on the same agent is documented at ten times. A single bug-fix attempt on a frontier coding agent costs between ten and forty dollars at mid-tier API rates and up to roughly one hundred dollars at flagship-tier rates. There is no upper bound the buyer can negotiate for; the cost driver is on the seller’s side of the contract.

The consequences are visible across three named cases. Microsoft sells, alongside Copilot itself, a measurement product called the Copilot Dashboard. The same vendor whose AI is being deployed sells the dashboard that measures its adoption to the same CIOs whose boards have mandated the deployment, with vendor revenue scaling linearly with how thoroughly the buyer industrialises all three. Klarna deployed an AI customer service chatbot in 2024, replaced seven hundred contractors, halved its workforce, and reported tens of millions in savings against on-par satisfaction scores; by May 2025 the company had announced rehiring, and the CEO told Bloomberg that “cost unfortunately seems to have been a too predominant evaluation factor.” The metrics that justified the deployment had not been wrong about what they measured. They had been wrong about what mattered; the AI worked exactly as programmed, not as intended. And the capital base funding the LLM consensus has been simultaneously funding its hedge: Yann LeCun’s AMI Labs closed a $1.03 billion seed round at $3.5 billion pre-money in March 2026, three weeks after Fei-Fei Li’s World Labs closed $1 billion of its own. The unspoken assumption of AI-capability fungibility — that what is being deployed today can be swapped for whatever wins next at minimal cost — is the assumption the cap tables most directly contradict.

What works on the substrate, the bounded cases tell the buyer to deploy: a narrow output, a non-LLM validator, and a human whose taste the substrate cannot game. Coding assistants with senior engineering review work because a senior dev can smell C-minus grade work at a thousand paces. Structured extraction with schema validation works because the validator is a JSON parser. Summarisation with editorial oversight works because the editor catches what the model missed. The investment thesis that justifies the capex is none of these — it is autonomous agents at scale on open-ended outputs without human review, in the precise domain where the structural finding holds. A rational buyer’s choice is to deploy the narrow, validated, human-supervised tooling and refuse the autonomous-agent pitch. The board mandate insists on the second.

The classic confidence trick is the brick in the box: the buyer pays for the high-value object that was promised; what arrives, weighted convincingly, is a brick. The token sink is the same trick at industrial scale.