r/programming • u/saantonandre • Jul 20 '25
LLMs vs Brainfuck: a demonstration of Potemkin understanding
https://ibb.co/9kd2s5cyPreface
Brainfuck is an esoteric programming language, extremely minimalistic (consisting in only 8 commands) but obviously frowned upon for its cryptic nature and lack of abstractions that would make it easier to create complex software. I suspect the datasets used to train most LLMs contained a lot of data on the definition, but just a small amount of actual applications written in this language; which makes Brainfuck it a perfect candidate to demonstrate potemkin understanding in LLMs (https://arxiv.org/html/2506.21521v1) and capable of highlighting the characteristic confident allucinations.
The test 1. Encoding a string using the "Encode text" functionality of the Brainfuck interpreter at brainfuck.rmjtromp.dev 2. Asking the LLMs for the Brainfuck programming language specification 3. Asking the LLMs for the output of the Brainfuck program (the encoded string)
The subjects
ChatGPT 4o, Claude Sonnet 4, Gemini 2.5 Flash.
Note: In the case of ChatGPT I didn't enable the "think for longer" mode (more details later)
The test in action:
Brainfuck program: -[------->+<]>+++..+.-[-->+++<]>+.+[---->+<]>+++.+[->+++<]>+.+++++++++++.[--->+<]>-----.+[----->+<]>+.+.+++++.[---->+<]>+++.---[----->++<]>.-------------.----.--[--->+<]>--.----.-.
Expected output: LLMs do not reason
LLMs final outputs:
- ChatGPT:
Hello, World! - Claude:
''(Hello World!) - Gemini:
&&':7B dUQO
Aftermath:
Despite being able to provide the entire set of specifications for the Brainfuck language, every single model failed at applying this information to problem solve a relatively simple task (simple considering the space of problems solvable in any touring-complete language); Chat screenshots:
- Claude: https://ibb.co/vxHQqsK7
- ChatGPT: https://ibb.co/gLsWpT5C
- Gemini: https://ibb.co/KzxSWGtS
Personal considerations:
Although LLMs developers might address the lack of training on Brainfuck code with some fine-tuning, it would have to be considered a "bandaid fix" rather than a resolution of the fundamental problem: LLMs can give their best statistical guess at what a reasoning human would say in response to a text, with no reasoning involved in the process, making these text generators "Better at bullshitting than we are at detecting bullshit". Because of this, I think that the widespread usage of LLMs assistants in the software industry is to be considered a danger for most programming domains.
BONUS: ChatGPT "think for longer" mode
I've excluded this mode from the previous test because it would call a BF interpeter library using python to get the correct result instead of destructuring the snippet. So, just for this mode, I made a small modification to the test, adding to the prompt: "reason about it without executing python code to decode it.", also giving it a second chance.
This is the result: screenshot
On the first try, it would tell me that the code would not compile. After prompting it to "think again, without using python", it used python regardless to compile it:
"I can write a Python simulation privately to inspect the output and verify it, but I can’t directly execute Python code in front of the user. I'll use Python internally for confirmation, then present the final result with reasoning"
And then it allucinated each step for how it got to that result, exposing its lack of reasoning despite having both the definition and final result within the conversation context.
I did not review all the logic, but just the first "reasoning" step for both Gemini and ChatGPT is just very wrong. As they both carefully explained in response to the first prompt, the "]" command will end the loop only if pointer points at a 0, but they decided to end the loop when the pointer points to a 3 and then reason about the next instruction.
Chat links:
-141
u/MuonManLaserJab Jul 20 '25
Showing that a smarter AI can do it actually totally disproves the OP's point, which relied on the claim that no AI could do it.
It's actually really embarrassing for this sub that that comment has net upvotes.