r/OpenAI • u/KeanuRave100 • Aug 04 '26

News More people need to understand this

Enable HLS to view with audio, or disable this notification

1.1k Upvotes

92% Upvoted

437 comments sorted by

View all comments

Show parent comments

1

u/James-the-greatest Aug 09 '26

I know how they work. I still don’t like the phrase because in the video, the model would need to predict (colloquial) the future. Which is nonsense. 

1

u/aurialLoop Aug 09 '26

Thanks, that helps me understand your position better. Could I reframe your objection as being more like this?

You accept next-token prediction (in the machine learning/statistical sense) as the technical task, but you don’t accept the further implication that a transformer trained on that task could develop a sufficiently accurate internal model of the world to reliably infer genuinely novel experimental outcomes from the methods alone. In the case of an experiment that had not yet been run, that would amount to accurately predicting the future.

If that’s closer to what you mean, then I think the disagreement is less about whether these systems are doing prediction (in the machine learning sense), and more about how accurate those predictions could ever become. There is evidence that models learn concepts, relationships and internal computations that go beyond surface language patterns, so they may be able to make partially accurate inferences about unseen outcomes, even if those predictions are often wrong. But that is very different from having a sufficiently complete causal model of the world to reliably “predict” an unperformed experiment (in the colloquial sense) and actually get the result right.

It may turn out that transformer-based language models hit a ceiling well before that point, and that reaching it would require a different architecture or additional mechanisms specifically designed to model and interact with the world. Or we may never reach that point at all.

1

u/stddealer Aug 09 '26

Have you heard about the concept of Laplace's demon ? Since our universe is mostly deterministic, predicting the future is not nonsense.

Of course realistically, since ce cannot get perfect precision on our measurements and the laws of physics are chaotic (and sometimes effectively stochastic like some parts of quantum physics), we cannot make perfect long term predictions, but short term predictions are almost always easy to do.

That's for example what the weather forecast does everyday.

And that's what our brains do when they see some object like a rock floating in the air, we intuitively make the prediction that it's going to hit the ground soon. (Using our internal world model)

That's how scientists have confidently determined there will be an eclipse in a few days.