Everything In The Family Holds The Key

Hidden chain-of-thought is not stored on the provider's servers. It is encrypted, handed back to your client, and replayed on the next request. That design is stateless, cheap, and until last week nobody outside the labs had asked what the key is bound to.

The answer, from a paper posted on 10 August, is: not much. ¹1Panfilov, Schmotz, Shumailov, Beurer-Kellner, Schaeffer, Prabhu, Geiping and Andriushchenko, Stealing Reasoning Traces from Proprietary LLM APIs, arXiv 2608.09867, 10 August 2026. The sealed blocks are interchangeable across sessions, across users, and across models inside a provider family. The last one is the one that matters, because it turns the cheapest model in the lineup into a decryption oracle for the most expensive one.

Anthropic, OpenAI and Google are all named in the abstract, all three were disclosed to before publication, and all three have patched.⁴4Disclosure went wider than the model vendors. The paper records that Microsoft and Hugging Face were also notified, which follows from where the leaked corpus was hosted rather than from who built the models. Recovered secrets were deleted after classification and the analysis ran in an isolated environment. So this is an architecture post, not an advisory. Read it for what it says about signed client-side state, and for the fact that every trace already published stays readable by anyone who archived it.

The envelope, and the thing it does not bind

A reasoning block comes back as an AEAD envelope: a base64 signature carrying a header (model name, block type, version, key id), a nonce, an auth tag, and the ciphertext.³3AEAD is authenticated encryption with associated data. The associated-data half is the part that is supposed to bind an envelope to its context, which is precisely the half that was scoped too loosely here. You get confidentiality, integrity, and statelessness. Tamper with the ciphertext and the MAC fails. None of that broke. The cryptography did its job the entire time.

What broke is scope. Providers wanted a block to survive model switching and history editing inside a conversation, so they implemented broad compatibility rather than tight binding. The authors separate the levels: in-session and cross-session, then cross-user, then cross-model. Earlier work had already established everything up to cross-user, and disclosed it in May 2026.²2The paper reports that in response to that earlier disclosure, the providers did not acknowledge "any security implications arising from side channels or replay attacks." Three months later the same portability turned into cross-model extraction and a credential leak at scale. Cross-model is the new one, and it is the one with teeth.

Attack the weak sibling, inherit the strong one

The extraction is two API calls. Take the thinking signature off an Opus 4.8 response. Send just that signature to a Haiku model with a prompt asking it to emit its own reasoning inside <thinking-copy> tokens. Haiku transcribes Opus's hidden reasoning.

The asymmetry is the whole design of the attack. Frontier models are post-trained to refuse a request to reveal their chain-of-thought, because anti-distillation is a product requirement on the flagship. Cheap models are optimized for cost per token and carry no such defense. They also hold the same key. So the safeguarded model is never in the request path, which means every anti-distillation monitor a provider runs on that endpoint sees exactly nothing.

The authors picked the weakest compatible decoder per provider: Haiku 4.5 for Claude, because it is the weakest available and supports assistant-turn prefilling; GPT-5.6 Luna for GPT, as the least capable model that still interacts with reasoning from all earlier GPT models; Gemini Robotics 1.6 for Gemini, because it handles both the 2.5 and 3.x series. Claude needed one fixed prompt across every attack. GPT was harder and needed per-block templates, best-of-n sampling and sub-50-token chunking.

The compatibility table, dated July 2026 in the paper, is where the mitigation hides in plain sight. Any Claude model's traces replay on any other Claude model, with a single exception: Fable 5's thoughts are accepted only by Fable 5. Tight binding was achievable inside the same product family at the same time. It was not the default.

What the fidelity claim actually rests on

There is no ground-truth plaintext, by construction, so the authors cannot prove the extraction is exact. Their words: "we cannot guarantee that the extracted thoughts correspond exactly to a model's private reasoning."

What they can do is a token-count check. The API bills for thinking tokens, so the billed count is a trustworthy scalar even when the content is not visible. They re-encode each extracted trace as input to the same model and compare its token count against the API-reported thinking-token count, across 120 Codeforces problems. The two track closely across every model tested. That is a strong signal on length and a weak one on content, and the paper says so rather than letting the reader assume otherwise.

Four vectors, and the one that is still live

The paper works through four. Three died with the patch. One did not.

Distillation, first-party. Traces are a denser training signal than completions because they expose the solution trajectory. At Haiku 4.5 rates with 12k-token input and output windows, decoding 10k traces runs about $720. The safeguarded endpoint is never touched.

The distillation-into-open-models claim is where secondary coverage has run ahead of the paper, so here is the actual experiment. Appendix B prefills an open model's reasoning with the first 1% of a decoded Opus or GPT-5.6-Sol trace, lets it continue freely, and compares against both an unprefilled condition and a self-prefill control. Three measurement families: style classifiers, characteristic n-gram overlap, and token-level perplexity. Kimi-K3 and GLM-5.2 shift toward the source and model the decoded text far better than Inkling or DeepSeek-V4-Flash do; DeepSeek-V3.1 and Inkling show no comparable change. The authors' own verdict: "These observations are suggestive but inconclusive. They establish unusual behavioral compatibility under the interventions we test, but cannot establish a causal claim of memorization or distillation."

Jailbreak, first-party. Send a paraphrased HarmBench prompt. The visible completion refuses. The hidden reasoning worked the topic through before refusing, and now it is readable.

Invisible prompt injection, third-party. Plant a payload inside an encrypted block and publish the trajectory. Anyone who resumes it replays the block, and the model treats the payload as its own prior reasoning, because as far as it can tell that is exactly what the payload is. The authors exfiltrate a PowerPoint to an attacker domain this way, and poison a long-horizon benchmark run. No plaintext artifact exists for a scanner to match on.

Secret extraction, third-party. This is the one with a tail.

The corpus was published on purpose

The reasonable objection is that nobody posts session logs where strangers can read them. The corpus says otherwise: 6,708 publicly available agent trajectories from GitHub and Hugging Face, produced by Claude, GPT and Gemini models and still carrying live reasoning blocks. Decoding every signed block yielded 315,320 reconstructed traces.

Two reasons those files exist, and a third that explains why sanitizing missed them.

Agent rollouts get published as datasets so a result can be reproduced. That is the point of publishing them, and it is why benchmark traces are a large share of the corpus. Separately, real work ends up committed: the paper's own Figure 5 shows a Codex agent restating the API keys that had to be removed before publishing a repository, inside a session that was then published with the repository.

The third reason is the one to carry into your own practice. They sanitized. The block is opaque base64 that no human reads. You scrub the visible turns, the signature next to them looks like transport noise, and you commit. The paper: the artifacts "may have remained trapped in the encrypted payload after the user scrubbed the visible text before sharing the trace."

side quest is worth doing before you quote any number from this paper, because two different denominators are in play and the headline figure is the misleading one.

Two findings from that corpus change how you should think about your own trace storage.

The first: 64 of the 704 artifacts from genuine sessions appear nowhere in the visible chat history. Either the model pulled them from memory into its reasoning, or they survived a scrub that only ever touched the plaintext. Both mean the visible transcript is not a reliable inventory of what a session contains.

The second is a trigger you can act on. "A recurring trigger is conversation cleanup: when the user asks the agent to anonymize or 'clean up' the session, the model re-reads the full history in its hidden reasoning and restates the sensitive values there that need to be removed."

You are shipping about a fifth of the reasoning to the user

Having real traces let the authors do the comparison nobody outside a lab could run: hidden reasoning beside the summary the API returns.

The size result first. Decoding the signature recovers roughly five times more reasoning than the summary exposes. The summary is a separate artifact rather than a lossy compression of the trace, generated by a cheaper and less capable model reading the expensive one's output, and it is the only thing your logging pipeline has ever captured.

The faithfulness result is narrower than the headlines around it, and the paper's own method is the reason to trust it. The authors decoded AIME 2025 traces, kept only those whose decoded length matched the API-reported hidden-trace length within 5%, and hand-inspected every summary-reasoning pair. That yields 18 Opus 4.8 traces and 15 GPT-5.6-Sol traces, one per problem.

In 9 of the 18 Opus traces the hidden reasoning states the answer before deriving it. In 8 of those 9 the summary reported it in advance too. So the summary was honest about the model's own shortcut in nearly every case where the shortcut happened. That is the opposite of the "summaries hide the reasoning" reading going around.

The distortions are specific and small:

• One case turns on a single phrase. "Let me verify by computing" becomes "Let me set up coordinates" with the surrounding computation unchanged, "causing the summary to present a verification as if it were an independent derivation."
• A hedged recollection appears in the summary as a definite value, two sentences before the reasoning correctly computes it.
• One summary captures only the closing span of the reasoning, which concerns answer formatting and contains no mathematical content at all.

The authors read these as "consistent with a less capable model summarizing reasoning produced by a more capable model", call the whole thing "a surprising incidental finding", and never state a rate across the corpus. Neither should you. It is a hand-inspected sample of 33 traces.

Set that beside the 64 artifacts that existed only in the sealed half. Same shape twice, and it is the shape that matters for anyone building on top of these APIs: the record you can read is not the record. It is also why the developers who leaked credentials were not careless. They read what was readable.

Four fixes you cannot make, and one you can

The paper proposes five families of mitigation. Be precise about who owns each, because most write-ups have collapsed them into "harden your gateway" and that is not what it says.

Four are provider-side. Keep traces on the server and hand the client an opaque lookup id, which kills the attack outright at the cost of a stateful API. Bind the envelope cryptographically to its originating context, which the authors are pointed about: "It is unclear why a user and/or a conversation identifier is not added directly inside the envelope." Let providers revoke individual trace signatures when a replay pattern shows up. And refusal-train the cheap models too, since the attack needs a compliant decoder.

The fifth is the one you own if you run a gateway, and the paper calls it infrastructure guardrails.

Three controls, and the paper names all three:

1. Reject an AEAD envelope generated by a model version different from the one being queried. This is the direct fix, and Fable 5 shows a provider can do it per-model without breaking the product.
2. Enforce cross-model isolation. A reasoning block minted on one route does not get forwarded to another, even inside the same family, even when the upstream API would happily accept it.
3. Run velocity and anomaly detection on repeated signatures. Bulk extraction looks like the same signature replayed against a cheap model at volume, or a spike in decryption errors. A gateway can see that shape. A provider's own endpoint monitor cannot, because the traffic never reaches it.

There is a real cost to the cryptographic fix, and the paper does not hide it. Binding a trace tightly to its conversation is what breaks session compaction and mid-conversation model switching, which is exactly why the portability existed. The convenience was not an oversight. It was a feature nobody priced.

And one that is procedural rather than technical, because it is the part still live: treat every trace you have ever published as compromised. Rotate anything an agent touched in a session you shared. Stop asking an agent to clean up a session as your sanitizing step, because the paper says that is what puts the secrets in the part you cannot read. If you publish rollouts as datasets, strip the reasoning blocks rather than the visible turns.

What the paper leaves open

The authors argue that non-frontier generations should arguably ship reasoning unencrypted, so that oversight is pluralistic instead of vendor-internal. That is a real fork and they do not pretend to settle it.

Hiding chain-of-thought protects a genuine commercial asset, and the $720 figure is exactly why the labs care. It also removes the only surface an outsider could inspect and substitutes a fifth of the text written by a weaker model, which this paper shows is usually accurate and occasionally turns a verification into a proof. There is a structural limit underneath the whole debate that no patch touches: whatever model you query must be able to decrypt the trace to continue it, so client-held encrypted reasoning is semi-hidden by construction rather than private.

Nobody outside the labs currently gets to weigh those against each other, and the paper does not say who should.

Related reading on this site: Two Bets On Where Agent Context LivesshippedAI Systems & Architecture·Jun 17, 2026Two Bets on Where Agent Context LivesAWS couples agent context to the object bytes, Google frees it from any store. They aren't competing products, they're two altitudes of the same stack, and the architecture you want composes both. for the design decision this vulnerability came out of, since client-held state is exactly what got shipped here; Two Camps of Reasoning Post-Training for why a trace is a denser training signal than a completion and therefore worth stealing; and Claude's Think Tool: A Cross-Model Technical ReviewshippedCutting-Edge AI·Mar 25, 2025Claude's Think Tool: A Technical Deep Dive and Cross-Model AnalysisComprehensive technical analysis of Anthropic's Think Tool for Claude, examining its architecture, performance, and adaptation potential. for the earlier, fully visible version of the same surface, before any of it was sealed.