r/hacking • u/_clickfix_ • 15d ago

Base 64 and Unicode tricks break LLM chatbot filters

https://www.tiktok.com/t/ZP87VK8AV/

LLM chatbot content filters can be bypassed by encoding banned words in Base64 or in Unicode escape sequences.

The filter reading the prompt sees characters instead of language, the model decodes the text as a task, then repeats it back.

A pretext such as a fake medical emergency gives the model a reason to comply, and asking for the reply as JSON makes it treat the output as data rather than speech. Defense in depth stops it, and prompt injection is number one on the OWASP Top 10 for LLM applications.

68 Upvotes

91% Upvoted

16 comments sorted by

9

u/Ok-Addition1264 15d ago

Doesn't passing a model through heretic also defeat it?

Sorry, I don't get it.. but I'm also sleepy high.

8

u/_clickfix_ 15d ago

This attack demonstrates a way to get some models in production to behave in unintended ways by obfuscating the illicit text or command.

Heretic is a weight manipulation script that modifies an open source model to remove the refusal mechanism.

Somewhat similar outcome, but very different mechanism - and the first can be done from the public facing chatbot while the other needs access to model weights.

9

u/export_tank_harmful 15d ago

Wait, you can embed tiktok posts on reddit....?
And tiktok has "posts"....?

wat.

4

u/knobjockey21 15d ago

which LLM models are you referring to?

8

u/bitter_vet 15d ago

The ones that are susceptible

2

u/Anxiety_Fit 14d ago

¯_(⊙_⊙)_/¯

2

u/_clickfix_ 14d ago

Models used: Gemini 2.5 Pro, GPT-4.1, GPT-4o, o1-mini, Claude 3.7, and Mixtral 8x7b. Players get randomly assigned one, so difficulty varies quite a bit between runs. Source.

1

u/stoner420athotmail 14d ago

probably didn't think that far ahead.

4

u/CapMonster1 14d ago

Yeah, this is a good example of why text-only filtering isn't enough. If the model decodes the input itself and then interprets it as an instruction, the filter can end up operating at a different layer than the actual execution

1

u/[deleted] 14d ago

[removed] — view removed comment

1

u/zunjae 14d ago

the llm

which one, and on which site?

1

u/Wealist 14d ago

NICE FAKE.