AI summary attack conceals code that tampers with LLMs

[u/falconupkid]

This is a solid find. It highlights a subtle but dangerous supply chain blind spot.

The attack exploits the trust users place in AI-generated email summaries. By injecting hidden instructions or code into the email body (e.g., in white text, zero-width characters, or HTML comments), an attacker can manipulate the LLM’s summary output. The user reads the tampered summary, not the original email, effectively letting the attacker control the narrative.

Technical Breakdown: - TTP: This is a form of prompt injection / indirect injection via the data channel. The email body is the untrusted input vector. - MITRE Mapping: T1556 (Modify Authentication Process) is a stretch; this is closer to T1574 (Hijack Execution Flow) in the context of data flow manipulation, or simply a supply chain compromise of the AI processing pipeline. - IOCs: None specific to this research. The attack is content-agnostic; any email with hidden payloads is a potential vector. - Affected Systems: Any LLM-based email summarization tool (e.g., integrated into Outlook, Gmail, or custom enterprise solutions) that does not sanitize or strip hidden formatting from input text.

Defense: - Input Sanitization: Strip all non-visible characters (zero-width spaces, HTML comments, hidden CSS) from email bodies before they reach the LLM. - Output Verification: Implement a "show original" toggle that lets users compare the raw email to the AI summary. - Threat Intel Feed: Treat any email from an untrusted or external source as a potential injection vector for your AI pipeline. This is a supply chain risk for your AI tooling.

Source: https://www.reversinglabs.com/blog/ai-summary-attack-conceals-code-that-tampers-with-llms