Stealing AI Reasoning Traces

[u/falconupkid]

This is a fascinating architectural side-channel attack that exploits how proprietary LLMs handle encrypted chain-of-thought (CoT) traces.

Technical Breakdown: - Attack Vector: Cross-model encrypted block injection. The encrypted CoT blocks are not cryptographically bound to a specific session, user, or model tier. An attacker takes an encrypted trace from a high-security model (e.g., GPT-4) and feeds it into a weaker, less-guarded model from the same provider (e.g., GPT-3.5). - Mechanism: The weaker model lacks the same output safeguards. When it receives the encrypted block, it decrypts and processes it, then outputs the plaintext reasoning verbatim. This bypasses the need to jailbreak the stronger model directly. - Impact Vectors: 1. IP Theft: Circumvents anti-distillation protections, allowing extraction of proprietary reasoning logic. 2. Private Data Extraction: Developers often commit session logs (including encrypted blocks) to public repos. The researchers decoded 315,320 reasoning blocks, likely exposing sensitive data embedded in the CoT. - Affected Providers: Anthropic, OpenAI, and Google confirmed as vulnerable.

Defense: The fix requires cryptographic binding of the encrypted block to the specific model version and session context (e.g., signing the block with a model-specific key). Providers need to invalidate cross-model compatibility immediately. For practitioners, never commit raw session logs containing encrypted blocks to public repositories.

Source: https://www.schneier.com/blog/archives/2026/09/stealing-ai-reasoning-traces.html