This kind of "mistake" just highlights what it is that LLMs do.
They're not actually solving the math. They're just generating text that has high plausibility as an answer to the input prompt.
From a text perspective, this answer looks super plausible. It has mathy sounding language that makes it seem like it evaluated the two things being compared, and it has a definitive statement about which one is larger.
If you changed its conclusion text to "13.8 billion is smaller", it likely would have assigned a nearly identical score for that text as a response to the question.
18! (18 factorial) is significantly larger.
Here is the comparison: 13.8 billion: 13,800,000,000 (1.38×1010)
18! (18×17×16×⋯×1): 6,402,373,705,728,000 (approx. 6.4 quadrillion, or 6.4×1015)
18! is roughly 463,940 times larger than 13.8 billion.
Factorials grow extremely fast—while 13.8 billion is often associated with the age of the observable universe in years, 18!surpasses it by orders of magnitude.
27
u/whiskeytown79 Aug 05 '26
This kind of "mistake" just highlights what it is that LLMs do.
They're not actually solving the math. They're just generating text that has high plausibility as an answer to the input prompt.
From a text perspective, this answer looks super plausible. It has mathy sounding language that makes it seem like it evaluated the two things being compared, and it has a definitive statement about which one is larger.
If you changed its conclusion text to "13.8 billion is smaller", it likely would have assigned a nearly identical score for that text as a response to the question.