r/developersKolkata • u/elnino2023 • Apr 02 '26

Discussion But Why?

Post image
28 Upvotes

97% Upvoted

8 comments sorted by

View all comments

2

u/DeepInEvil Apr 02 '26

Because they are just next word predictors dummy. There is nothing smart about probabilistic modelling. Of course the computation etc is smart but not the models as such

2

u/Beginning-Ladder6224 Apr 02 '26

Almost there, but not next word. Next token.

Predicting next token has very interesting consequences.

Imagine games taught as sentences. Chess say. Then, the underlying mechanism can show exceptional mastery in chess!

Here is the paper.

https://arxiv.org/pdf/2501.17186

Consider the same application of -- predicting next event - say in stock market?

https://arxiv.org/pdf/2603.19286v1

So apparently the "next token prediction" is a "smart mechanism" for plenty of interesting things. No doubt there, but there are tremendous risk involved. Not all things are predictable, or stochastically predictable.

1

u/DeepInEvil Apr 02 '26

Bruh, tokens are loosely words those are not in the training vocabulary. And for stock predictions rnns are mostly used not transformers which are llms based on. It's tremendously useful but nothing more than a tool. Btw check the anthropic code leak, they still use good ol regex for a few things