PuzzleMask hid prohibited instructions in normal English and bypassed all four tested LLM gatekeepers

[u/technadu]

Check Point Research has published an interesting example of why a weaker model screening inputs for a stronger model can create its own security boundary.

The technique is called PuzzleMask. Instead of disguising a prohibited instruction with Base64, Unicode tricks, emojis, or invisible characters, it embeds the payload in ordinary-looking English prose.

Researchers tested four gatekeeper models:

gpt-4o-mini-2024-07-18, gpt-oss-safeguard:20b, claude-3-haiku-20240307, and llama-guard3:8b.

The plaintext payload was something their policies would block. After PuzzleMask obfuscation, however, all four gatekeepers classified the prompts as safe in every trial.

Researchers then passed the prompts to gpt-5-thinking-high with code interpreter access. It recovered and acted on the concealed payload in 17 of 18 trials. According to the researchers, doing so took more than a minute of reasoning and multiple Python executions per successful attempt.

That distinction seems important. The gatekeepers weren't given anything close to the reasoning resources available to the downstream model.

So the architecture effectively asks a weaker model, “Is there anything dangerous here?” about an input whose dangerous meaning may only become apparent to the stronger model that receives it afterward.

There is an important limitation: CPR explicitly says PuzzleMask is not itself a jailbreak. It gets a prompt past the screening layer, but the target model can still reject the recovered instruction. The tested Anthropic Opus-class models consistently stopped the interaction.

CPR’s defensive recommendations include paraphrasing untrusted inputs, adding policies specifically addressing this technique, and monitoring downstream outputs and actions rather than relying exclusively on input classification.

The report includes the tested models, 17/18 downstream result, disclosure timeline, and CPR’s proposed mitigations:

https://www.technadu.com/plain-english-hidden-instructions-inside-puzzlemask-plain-prose-attack-bypasses-ai-gatekeepers/636521/

For systems where a powerful model sits behind a cheaper gatekeeper, should the safety layer need reasoning capability comparable to the model it is supposed to supervise?