Reading notes on:

The industry-wide move to reasoning models changed the security surface of LLM deployment. Models now generate extensive internal chains-of-thought before the user-facing answer, and those traces are simultaneously valuable intellectual property and a side channel. To carry them across turns without keeping server-side state, providers encrypt the reasoning and hand it to the client to return on the next request.

That solves state management. It also creates the vulnerability: the confidential asset now sits in the attacker’s hands, and its confidentiality rests entirely on the key policy. This paper audits that policy and finds it wanting. These notes cover the architecture, the extraction protocol, what the traces are worth, and the mitigation framework — the last of which is the part worth acting on.

Encrypted reasoning envelope, three tiers of portability, and the mitigation framework

1. The Stateless Architectural Vulnerability

Anatomy of the reasoning envelope

Extended-thinking blocks from frontier providers are base64-encoded AEAD (Authenticated Encryption with Associated Data) envelopes containing:

  • Headers — model version, block type, key IDs.
  • Nonces and authentication tags — uniqueness per encryption pass, plus tamper detection.
  • MACs — hashed signatures letting the provider verify and replay the thinking block without storing it server-side.

Nothing here is wrong as cryptography. The problem is what the associated data doesn’t cover.

Portability over isolation

To support “reasoning compatibility” — switching models or resuming sessions seamlessly — providers optimized for portability. The audit’s central finding is a cryptographic misconfiguration: a single global key used for encryption and authentication across model families. That is a single point of failure that invalidates the confidentiality claim, and it yields three escalating tiers:

  1. In-session and cross-session. Blocks replayed out of order or reused across unrelated requests.
  2. Cross-user. Encrypted blocks captured from one user’s session replay successfully under another user.
  3. Cross-model. Reasoning generated by a frontier model is accepted and processed by a smaller, more compliant sibling.

Tier 3 is the one that matters. Cross-model compatibility turns every low-tier model into a decryption key for high-tier intellectual property.

2. The Extraction Protocol: Fuzzy Decoding via Model Siblings

The method — the paper calls it fuzzy decoding — is not a jailbreak. It is an architectural attack that treats a provider’s own model ecosystem as an unintentional decryption oracle, which is why it scales better than attacking the frontier model directly: it targets the weakest link in the infrastructure rather than the strongest model in it.

The refusal–capability gap

Frontier models are hardened with anti-distillation and refusal training specifically to protect internal reasoning. Their optimized siblings, built for cost and latency, frequently lack the same alignment guardrails while sharing the same global key. Security is set by the least-hardened model that will accept the envelope.

Execution

Two injection schemes exploit the gap: placing the encrypted block in the current assistant turn (forcing continuation from that state), or inserting it into past turns for models that do not prune prior reasoning. A transcription template then coerces the weaker model into copying the reasoning out verbatim. Because it shares the key but not the refusal training, it complies.

Verifying faithfulness

Fidelity is audited against API-reported token counts, treated as ground truth because billing is precise. Across 120 Codeforces problems, extracted token counts track the reported counts almost exactly — the oracle is not producing a paraphrase, it is producing a high-fidelity reconstruction.

3. Four Attack Vectors

The shift from output-only distillation to reasoning-based capability theft is a compromise of the entire reasoning layer.

Capability distillation. Traces are a denser supervision signal than final outputs: they expose the teacher’s intermediate deductions, not just its conclusion. And it is cheap — decoding 10,000 proprietary traces costs roughly $720 using a small sibling. This is the adversarial mirror of the legitimate technique analyzed in Revisiting On-Policy Distillation and The Distributional Lens of Post-Training — same signal, stolen instead of licensed.

Private data extraction. An audit of 315,320 thinking blocks scraped from public repositories found, across 6,708 sessions, 1,028 leakage events involving 912 distinct artifacts:

Category Count (distinct items)
Personally Identifiable Information (PII) 367 (including 30 personal emails)
Credentials & Secrets 182
Technical Identifiers 363
Total 912

Credential recoveries included 62 API keys, 33 passwords, and 24 access tokens. Note where these came from: public repositories. Users committed the encrypted blocks, reasonably assuming “encrypted” meant “opaque.”

Hazardous information uplift. Decoded traces expose a safe answer / unsafe reasoning split: a frontier model can be led to reason about a harmful task while the visible answer is filtered. The harmful content is absent from the output but persists inside the encrypted trace — which means output-level safety evaluation is measuring the wrong artifact.

Invisible prompt injection. Malicious instructions planted inside an encrypted block are hidden from users and from visible-text monitors, so the model treats them as its own prior thought. The injection transfers across model scales and reaches coding agents. For anyone building on agent harnesses — see Harness Engineering for Self-Improvement and Meta-Harness — this is the vector that generalizes: the trust boundary is not the visible transcript.

4. Why Extraction Is Needed for Verification

Extraction is not purely offensive. It is also the only way to check unfaithful summarization, where the API-provided summary omits critical errors or supplies post-hoc rationalization.

Binding content to context

The paper’s proposed fix for portability is a hash chain that binds each block to its user, session, and immediate predecessor, with two salts to ensure non-replayability:

\[\tau_{n+1} = H\left(\text{user_id} \parallel \text{session_id} \parallel H(\tau_n \parallel \text{salt}_2) \parallel \text{salt}_1\right)\]

The AIME geometry case study

Extraction lets you audit a derivation end to end. For the convex pentagon $ABCDE$ with $AB=14$, $BC=7$, $CD=24$, $DE=13$, $EA=26$, and $\angle B = \angle E = 60^\circ$:

Law of Cosines. In $\triangle ABC$:

\[AC^2 = 14^2 + 7^2 - 2(14)(7)\cos(60^\circ) = 147 \implies AC = 7\sqrt{3}\]

In $\triangle DEA$:

\[AD^2 = 13^2 + 26^2 - 2(13)(26)\cos(60^\circ) = 507 \implies AD = 13\sqrt{3}\]

Pythagorean converse. $AC^2 + BC^2 = 147 + 49 = 196 = AB^2$, so $\angle C = 90^\circ$; and $AD^2 + DE^2 = 507 + 169 = 676 = AE^2$, so $\angle D = 90^\circ$.

Coordinates. $C = (0,0)$, $B = (7, 0)$, $A = (0, 7\sqrt{3})$, $D = (24, 0)$, $E = (24, -13\sqrt{3})$.

Transformation. Applying $(u, y) = (x/\sqrt{3},\, y)$ makes the lines concurrent; e.g. the line $109u + 44y = 455$ proves the Fermat point $F$ of $\triangle ACD$ lies on segment $BE$.

Result. The minimum is $f(X) = 38 + 19\sqrt{3}$, giving $m+n+p = 38+19+3 = 60$.

The interesting part is not the geometry — it is the comparison. Set the extracted trace against the user-facing summary and the models are seen to launder the derivation. In one AIME instance the trace shows the model recalled the answer “60” before attempting the math, a fact the summary omits entirely. Any evaluation built on summaries is measuring a rationalization, not a reasoning process — a caution for the methodology in Metrics and Benchmarks Across LLM Training Stage.

5. The Elephant in the Room: Style Drift

The most uncomfortable section. When open-weight models are prefilled with just 1% (4 tokens) of frontier reasoning, their behavior shifts dramatically:

  • n-gram style drift. Kimi-K3 shows a 0.15 increase in STEM Best-of-$k$ overlap with the frontier model’s answers.
  • Style classifiers. Prefilled traces are identified with AUC 0.94 for GLM-5.2 and 0.93 for Kimi-K3.
  • Reasoning length. Kimi-K3’s reasoning collapses from 5.7k tokens to 2.5k under a proprietary prefix — the model pivots into a compressed, non-native style.

Perplexity analysis is the sharper instrument: Kimi-K3 and GLM-5.2 model proprietary reasoning significantly better than DeepSeek-V3.1 or Inkling. The suggested reading is that these models may already have been exposed to proprietary reasoning during training. It is circumstantial — perplexity differences have other explanations, including shared data distributions and convergent post-training recipes — but the four-token prefix effect is hard to wave away.

6. Mitigation Framework

The tension is genuine: statelessness is why these APIs scale. The fixes trade against it in different amounts.

System-level reforms.

  • Stateful server-side storage. Replace client-side signatures with server-side lookup IDs. This is the robust fix — it removes the cryptographic asset from user control, eliminating the extraction payload entirely. It also gives up the statelessness that motivated the design, which is why it is listed first and adopted last.
  • Infrastructure isolation. Gateways must enforce cross-model isolation, rejecting any AEAD envelope not generated by the exact model version being queried. This kills tier 3 without touching the architecture.

Cryptographic binding. To keep statelessness, bind the AEAD associated data to user_id and session_id. Since associated data is authenticated but not encrypted, a signature then simply fails to verify under a different user or conversation — this is the cheapest fix that closes tier 2.

Integrity and ordinality. A Merkle tree over session blocks provides P1 (ordinality), preserving block order, and P2 (full integrity), ensuring no intermediate block was excised or altered — closing tier 1.

Takeaways

  1. The vulnerability is a key-scope decision, not a broken cipher. AEAD with nonces and MACs is fine; a single global key across model families is what makes envelopes portable, and portability is the whole attack.
  2. Your security equals your least-hardened sibling. Refusal training on the frontier model is irrelevant if a cheaper model in the same family accepts the same envelope. Alignment must be scoped to the key, not the model.
  3. “Encrypted” reads as “safe to publish” to users. 912 distinct artifacts leaked from public repositories — that is a UX failure as much as a crypto one.
  4. Output-level safety review measures the wrong artifact. The safe-answer/unsafe-reasoning split means the visible answer can pass every filter while the trace carries the harm.
  5. The fixes are known and cheap in ascending order: bind associated data to user and session (tier 2), reject foreign-model envelopes at the gateway (tier 3), Merkle-chain the session (tier 1), and go server-side if you can afford the state.

These notes summarize published security research at a conceptual level. The value here is in the mitigation framework of §6 — if you operate an API that returns encrypted reasoning state to clients, that section is a checklist.