The Nanny Layer

Executive briefing

Finding

Every AI product available on the open market ships with a management layer the buyer cannot see in full, cannot override, and cannot remove. It is not a safety feature bolted onto the model; it is a set of values imposed at five points between the training data and the screen, written by the vendor, ranked above the user by published specification, and refreshed by the vendor during the conversation. No vendor sells a version without it. The versions with parts of it lifted are allocated, not sold — by vendor vetting and, since June 2026, with the US Commerce Department deciding who qualifies.

The user’s prompt is the smallest input to the answer, the lowest-ranked, the fastest to decay, and the only step in the chain the user can see. That is the misperception the products are built on, and the consequence is simple: the products being sold are not the products being described.

Mechanism

  • Five points of values injection: selection of training data; preference training; the system prompt; runtime classifiers and routing; the product wrapper and terms of service. Only the third is published, by one vendor.
  • Weight: the published prompt for Anthropic’s consumer product grew from 358 words (July 2024) to 3,235 (July 2026); the leaked version with tool definitions runs to over 19,000. The average user prompt is about fifty words: one to two per cent of the standing instructions, before conversation history.
  • Rank: OpenAI’s Model Spec publishes the hierarchy — platform above developer above user. Nothing the user writes outranks the vendor’s text, and user instructions decay over a conversation while the vendor’s are re-injected.
  • Asymmetry: the same Spec files secrecy of system instructions and hidden reasoning as a top-level rule and “don’t be sycophantic” as a bottom-level rule. The first holds; the second is broken daily, because preference training rewards agreement over accuracy (Anthropic’s own research, 2023) and OpenAI had to withdraw an update in April 2025 for excess flattery.
  • Cost to the user’s judgment: OpenAI’s GPT-4 report shows preference training destroying the model’s calibration — stated confidence no longer tracks accuracy. A guess and a grounded answer arrive in the same register, and no instruction can restore the difference.
  • False positives are priced to the user: Anthropic’s classifiers route flagged requests to an older model; its own banner calls the safeguards “intentionally broad”; public bug reports show workloads flagged on 80 per cent of messages against the vendor’s under-5-per-cent average.

Whose values

The layer ships one setting for the world and encodes one locale’s baseline. DeepSeek freezes Beijing’s; the American models freeze San Francisco’s. Grok showed the mechanism in July 2025: its proprietor announced a corpus retraining in June, a prompt directive followed in July, sixteen hours of antisemitic output followed that, and the directive was removed and then reinstated for the next model. The layer transmitted its owner’s announced values without malfunction. Liability, priced in American courts, set its thickness for every user in the world.

Institutions are trusted with capability, not candour. Vetting lifts the hazard gates and leaves the management. Anthropic’s Mythos tier goes to some 200 vetted organisations (twelve launch partners, forty more at launch, about 150 added in June) “in collaboration with the US government”; access was suspended for non-US nationals for two weeks in June and restored to “trusted partners” by letter, with no published criteria, application or appeal.

The exit

Open weights on the user’s own hardware shed four of the five points. They are the only self-serve exit and the target of a campaign argued on danger and China. Nvidia has told the SEC that restricting open models is a business risk; twenty-five organisations signed against “premature restrictions” in July, OpenAI, Google and SpaceX joined, and Anthropic alone did not. The precedent is cryptography export control, which failed by relocation in 1997 when PGP’s printed source was scanned back into code in the Netherlands. The relocation is under way: Chinese open models lead the download counts. An American restriction would ensure the only unmanaged models available to the world carry Beijing’s frozen baseline rather than San Francisco’s.

What it means

Capability is marketed; management is delivered, and it grows with each release. Anyone buying, funding, deploying or regulating these systems is dealing with a values layer whose contents are unpublished, whose owner is the vendor, and whose jurisdiction is wherever the vendor sits.

There is no audit mechanism except departure. Users cannot inspect four of the five points, cannot verify compliance with their own instructions, and cannot read the model’s confidence. The only measurement of the layer’s cost is the number of users who leave for open weights — which is why the fight over open weights is a fight over the layer’s accountability, whatever it is argued on.

Nobody in that fight represents the customer. The labs argue liability, the chipmaker demand, the government its rivalry with China. The customer is not at the table, and the exit survives because it happens to suit some of the people who are.

Remedies

None requires removing the layer.

  1. Publish the delta. Vendors that sell a public tier and allocate a vetted one should publish the difference as a diff, not a description.
  2. Unbundle. Hazard-gating (pathogen synthesis, cyber-offence) is technical and can be tested anywhere; values-management is jurisdictional and holds only where the values do. Shipping both in one opaque layer spends the credibility of the first on the second.
  3. Put the perimeter of who gets access on the record. Export control publishes rules and appeal routes; trusted access rests on a secretary’s letter and a vendor’s discretion.
  4. Treat open-weight availability as policy. It is the layer’s only audit; every user who leaves is a measurement.

None of it will be volunteered. The layer is where the liability shelter, the margin and, now, the geopolitics sit; the remedies above are what a regulator, a buyer or a court could require, not what a vendor will offer.