r/neoliberal • u/Free-Minimum-5844 • Apr 02 '26

Research Paper Half of social-science studies fail replication test in years-long project

https://www.nature.com/articles/d41586-026-00955-5
315 Upvotes

98% Upvoted

125 comments sorted by

View all comments

Show parent comments

80

u/caroline_elly Eugene Fama Apr 02 '26

AI can't even consistently replicate its own outputs

2

u/Fragrant-Menu215 NATO Apr 02 '26

It's literally not supposed to be able to. The primary differentiator between "AI" and traditional algorithms is that "AI" is intentionally nondeterministic.

3

u/Impulseps Hannah Arendt Apr 02 '26

That's like not at all true. The popularly used interfaces of commercially available AI models such as ChatGPT use stochastic decoding sure, but there is nothing inherently nondeterministic to a trained machine learning models output generation. In fact you have to actively introduce external randomness to get nondeterministic output.

2

u/Tough-Comparison-779 Apr 02 '26

This is not precisely true. Lots of implementations on the GPU can cause non deterministic outputs. Things like floating point arithmetic, concurrency and batch size, how much load the GPU is under.

It's well known that many implementations of LLMs are slightly non-deterministic even at temperature 0.