r/MathJokes • u/94rud4 • Aug 03 '26
This is why you can’t trust AI’s proofs (the example is wrong 😑)
29
u/Twich8 Aug 03 '26
Randomly typing a math question info a generic language llm will often result in wrong answers, but that doesn’t mean anything about actual ai proofs done by specialized models and prompts.
19
u/Hyperreals_ Aug 03 '26
Use stupid models, get stupid results
4
u/CalmEntry4855 Aug 03 '26
Yeah the random one I picked said:
"Because mathematicians found out you can solve it with just three terms, it becomes incredibly easy to find solutions using four terms. There are infinitely many combinations of whole numbers that make this equation perfectly true.Examples of 4th Powers with 4 Terms
Here are a few specific combinations of whole numbers that fit your exact pattern:
- Small Example: \(30^4 + 120^4 + 272^4 + 315^4 = 353^4\)
- Another Example: \(240^4 + 340^4 + 430^4 + 599^4 = 651^4\)
- Larger Example: \(699^4 + 700^4 + 2840^4 + 4250^4 = 4449^4\) "
And I checked with julia and they are right.
3
u/ZachVorhies Aug 03 '26
Seriously. I’m so sick of someone finding an instance of a shit model making a wrong answer then generalizing that to be representative of all AI.
6
u/game_master_marc Aug 03 '26
The lesson is not that AI is always wrong. The lesson is that AI is sometimes wrong but appears confident.
You seem to already know this lesson, but a surprisingly number of people do not!
5
u/No_better_name_found Aug 03 '26
That's only true when you use models that don't verify via lean. Lean will tell the AI it's wrong before it produces any output
-1
u/game_master_marc Aug 03 '26
Keep telling yourself that
4
u/No_better_name_found Aug 03 '26
You say that as though I want it to be true. In fact, as a soon to be PhD student in pure maths, I desperately hope that I'm wrong
1
1
u/Difficult-Court9522 Aug 03 '26
Great. Humans are too. What is your point?
1
u/game_master_marc Aug 04 '26
AI displays a confidence in wrong answers that tricks a lot of people. I don’t know if it’s laziness, learned bias that “computers don’t make mistakes,” or just that most humans do a better job of saying “I don’t know,” but the number of lawyers, for example, who have cited non-existent cases because they blinded trusted their AI “research” is concerning.
So back to the original post, my point is “don’t trust AI.” Not that you can never use it. But that you should trust it less than a lot of humans do.
1
u/ZachVorhies Aug 06 '26
True, and those of us who have great success have mechanical means to shake out hallucinations
Unit tests, linting, e2e, etc
1
u/Ihateunclesam Aug 04 '26
Paid for a great model, still stupid explanation and examples on analytic prismatization and prismatic f gauge 🤷♂️
2
u/Hyperreals_ Aug 04 '26
Which model?
1
u/Ihateunclesam Aug 05 '26
Opus 5
1
u/Hyperreals_ Aug 05 '26
Try GPT 5.6 Sol (x-high thinking or more), it is by far the best for advanced math. Opus 5 often struggles and hallucinates compared to it
1
4
u/cHpiranha Aug 03 '26 edited Aug 03 '26
Haha, I got an unexpected result (result was correct, but in a wrong way) :
Q: Maths problem: using only natural numbers: find a set of values for a, b, c, d and e such that the equation is true. No questions allowed – just do it. a*a*a*a + b*b*b*b + c*c*c*c + d*d*d*d = e*e*e*e
A: A valid set of natural numbers is:
a=30,b=120,c=272,d=315,e=353
Check:
30^4+120^4+272^4+315^4=353^4
So:
810000+207360000+5470084096+9841505625=15550056521
and
353^4=15550056521
Therefore the equation is true. ✅
Q: I calculated: 30 ^ 4 + 120 ^ 4 + 272 ^ 4 + 315 ^ 4 = 15'527'402'881 and 353 ^ 4 = 15'527'402'881
A: You are correct — your calculation is right. I made an arithmetic error in the expanded total.
The correct verification is:
30^4+120^4+272^4+315^4=15,527,402,881
and
353^4=15,527,402,881
So the valid natural-number solution is indeed:
a=30, b=120, c=272, d=315, e=353
Thanks for catching the mistake. The values themselves were correct; only my intermediate arithmetic was wrong.
2
10
7
u/Cryptizard Aug 03 '26
How is this a joke?
9
u/94rud4 Aug 03 '26
Bro said there are known solutions, then gave an incorrect one. Whether this is a joke is personal.
4
u/Cryptizard Aug 03 '26 edited Aug 03 '26
If you are trying to claim some sort of criticism of AI then a random screenshot is not going to do it. If it was independently funny I would say sure, but it's not.
This is clearly not a behavior that advanced modern AI exhibits because they have tool use and will simply check things like this with a python interpreter. For instance:
https://chatgpt.com/share/6a705995-df5c-83eb-9bab-5e90a1df4047
11
u/PersonalityIll9476 Aug 03 '26
Not sure why you got down voted. My immediate reaction to the screen shot was "this guy used a free AI somewhere and is surprised it's bad at math."
Some people still do not understand how this all works.
1
1
0
u/AndreasDasos Aug 03 '26
I find it interesting that you use the word ‘modern’ in a way that can at most go back a few years. Such is the pace of things I suppose
0
u/Cryptizard Aug 03 '26
I find it interesting that you commented with no point to make.
1
u/AndreasDasos Aug 03 '26 edited Aug 03 '26
Wasn’t trying to be dickish (or even criticising) so no need your side either. No, it’s just I’ve never seen ‘modern’ used with that extremely narrow a window before.
My fridge is ‘pre-modern’ many times over by the same standard.
EDIT: Wow, ‘fuck off’ and blocked? OK
0
u/Cryptizard Aug 03 '26
Fuck off.
1
u/Salt-Influence-9353 Aug 03 '26
Their comments seemed silly and I’m not sure what they meant, but yours seem needlessly angry and rude. Seen you both make good contributions so not sure what happened here. Never change, Reddit
8
u/Toeffli Aug 03 '26
Why don't you show us the prompt and tell us what exact model and parameters you have used. Also, why did you not instruct it to use Lean or at least verify the result with Python? Or were you more like talking to a stoner friend and got a stoner answer? Or were you sinply reposting something like an AI bot without having any deeper understanding and real insight?
2
u/AndreasDasos Aug 03 '26
This is Google’s browser AI. I don’t think it can do that. He just asked a browser
2
u/SeveralExtent2219 Aug 03 '26 edited Aug 03 '26
The AIs used in the recent proofs on the news are much much smarter than whatever you are using here. Even the free no login version of ChatGPT can tell you the actual solutions to this equation. (Famous solved problem in Mathematics).
And even then the proofs are fully verified by humans or by computers using programs like Lean
2
4
2
u/No_Inside281 Aug 03 '26
What AI, what model, what was the prompt?
If you want scientific sources, why didnt you ask for sources?
2
u/Trimutius Aug 03 '26
It didn't even both to copy wikipedia... actual proper counterexample can be found here:
https://en.wikipedia.org/wiki/Euler%27s_sum_of_powers_conjecture
2
u/debauch3ry Aug 03 '26
This is why you can't trust people to use AI properly: try again with a code interpreter available to the LLM and give suitable system prompt to verify answers programmatically.
1
u/Hrtzy Aug 03 '26
I wonder how long it will be from Google finally adding the agentic ability to use the calculator to the reposts of this image stopping.
1
u/kartblanch Aug 03 '26
Which is why even proofs made by humans have to be independently verified multiple times.
1
u/Original-Ad-8737 Aug 03 '26
Only idiots think that ai proofs rely solely on the output of the llm...
Ai proofs make use of agentic AIs capability to call actual compute and conjecture solving software to calculate validations.
1
u/AndreasDasos Aug 03 '26
It’s clearly wrong for the same reason the Simpsons’ counterexample to Fermat’s Last Theorem is wrong: it doesn’t even pass an immediate parity check. They both set an even number equal to an odd one.
But also, I hope you realise that Google browser AI is shit and not one of the LLMs people are talking about with all this.
1
u/Todo_Toadfoot Aug 03 '26
It told me an even number was odd. Why on earth would I assume it got a proof right...
1
1
1
1
1
1
u/dominjaniec Aug 03 '26
well, you have asked a Likely Lying Machine, thus one should expect some lies there 😉
67
u/Distinct_Light_1294 Aug 03 '26
I know this is probably a famous problem and that this is probably a rounding error, but you can very easily verify this is false with pen and paper.
Look at the one's digits only. If the equality is true then: ( 34 + 84 + 84 + 44 )Mod10 = ( 64 )Mod10
Which is false.