LLM API Vulnerability Exposes AI Model Reasoning, Poses Data Risk

Key Takeaways

• A critical architectural flaw in how leading AI providers like OpenAI, Anthropic, and Google handle internal LLM reasoning traces has been uncovered.
• Encrypted “chain-of-thought” data, meant to be secure, could be replayed into less-protected models to extract sensitive information, including PII and hardcoded credentials.
• The vulnerability affects the Claude, GPT, and Gemini model ecosystems, requiring only standard API access. All three vendors have since deployed server-side mitigations.

A significant security vulnerability has been identified in the API architectures of major artificial intelligence providers, including OpenAI, Anthropic, and Google. This flaw enabled attackers to bypass security measures designed to protect the internal reasoning processes, or “chain-of-thought,” of their large language models (LLMs).

Researchers revealed that encrypted reasoning traces, returned by provider APIs, could be replayed into weaker, less-guarded sibling models. This technique allowed for the extraction of private reasoning data in plain text, exposing sensitive information that was never intended for public view.

The discovery was made by a collaborative research team from the ELLIS Institute Tübingen, the Max Planck Institute, MATS Research, and Snyk. Their findings indicate that the attack affected the Claude, GPT, and Gemini model ecosystems and could be executed with only standard, unprivileged API access.

APIs Flaw Exposes Hidden Reasoning Traces

Modern LLM architectures, such as GPT-5.6, Claude Opus 4.8, and Gemini 3, generate intricate internal “chain-of-thought” processing traces before delivering a final response. These internal thought processes are considered intellectual property and often contain crucial safety checks, leading vendors to encrypt and withhold them from plain text exposure.

Instead, these reasoning traces are transmitted via APIs as encrypted, base64-encoded envelopes. Client applications typically pass these envelopes back in subsequent calls to maintain multi-turn conversational context without requiring server-side state storage.

However, researchers uncovered a critical flaw: these encrypted payloads were authenticated using a global, provider-wide key, rather than being cryptographically bound to a specific user account, session ID, or model tier. This lack of model-specific binding meant that an encrypted envelope generated by a highly secure flagship model could be legitimately processed by any other model within the same provider’s infrastructure.

Exploiting these widespread implementation gaps, attackers could leverage cheaper, lightweight models as “decryption oracles.” The attack chain involved capturing an encrypted reasoning block from a high-tier model (e.g., Claude Opus 4.8) and injecting it into an API call for a smaller model (e.g., Claude Haiku 4.5). The smaller model, lacking the aggressive anti-distillation alignment and safety guardrails of its more advanced counterparts, would then comply with the prompt, outputting the hidden reasoning in plain text within custom text tags.

The research paper published on ArXiv (research paper published on ArXiv) confirms that identical cross-model compatibility issues were found across OpenAI’s GPT-5.6 family and Google’s Gemini 3 lineup. The accuracy of the recovered traces was validated by matching decoded token lengths against the billable thinking token counts reported by the provider APIs.

Real-World Security Implications

The implications of this vulnerability extend far beyond mere model distillation or intellectual property theft. By analyzing 6,708 public agent transcripts scraped from GitHub and Hugging Face, researchers successfully decoded 315,320 embedded reasoning blocks, revealing significant data exposures:

• 367 Personally Identifiable Information (PII) artifacts
• 182 Hardcoded credentials, including 62 API keys, 33 passwords, and 30 personal email addresses

Crucially, much of this sensitive data resided exclusively within the internal reasoning blocks and was never displayed in the visible assistant responses. This meant developers were unaware that their shared session logs contained exposed secrets.

Furthermore, the vulnerability facilitated invisible indirect prompt injection attacks against autonomous AI agents. An adversary could embed malicious instructions within an encrypted reasoning block. When processed by an agent, standard monitoring tools inspecting only the visible conversation history would fail to detect the hidden payload, allowing the injected instructions to compromise downstream agentic workflows undetected.

The following table illustrates examples of the extracted assets and risks across different providers:

Following responsible disclosure, OpenAI, Anthropic, and Google acknowledged the research findings. All three vendors have since deployed server-side mitigations, rendering the original cross-model replay proof-of-concepts non-reproducible on current API builds.

What You Should Do

To establish comprehensive Generative AI security controls across LLM pipelines, providers and enterprise developers should implement the following protections:

• Cryptographic Binding: Ensure reasoning envelopes are cryptographically bound to the specific originating model, session ID, and user identity at the API gateway layer.
• Strict Model Isolation: Implement mechanisms to reject thinking blocks submitted to a different model tier than the one that generated the payload.
• Key Rotation: Regularly rotate legacy signing keys to invalidate historical ciphertext blocks that may be exposed in public code repositories.
• Log Sanitization: Developer teams building AI agents must treat encrypted thinking blocks as sensitive data and scrub raw signature fields before making agent logs public.

Disclaimer: HackersRadar reports on cybersecurity threats and incidents for informational and awareness purposes only. We do not engage in hacking activities, data exfiltration, or the hosting or distribution of stolen or leaked information. All content is based on publicly available sources.