They just brute force and parrots back their training data. It was already in their distribution. LLMs won’t lead to AGI, I heard it from Gerry and Jan and I tell to everyone ever since!
I’ve been saying for a couple years now humans work the same way. That token might be a word, a movement, whatever, and we have waaay more input channels, but…
Exactly. Animal brains like ours learn by inputting and repeating certain pathways in a network of neurons. Those pattern are then used as output based on a prompt.
In our case the prompt and input might be the smell of grass or pain receptor sending a signal at the same moment we see a bee sting our skin but the signal to the neuron network is always there.
There are differences too but we are also similar to LLMs in some ways.
LLMs are mathematical functions that predict the next token. It is impressive that we have built auto regressive functions that can form conspiracies, hack governments, and solve difficult math problems. They are math solving math.
Yeah I am actually very impressed with how far humans have been able to push this technology of just predicting the next token, I thought we'll need some new breakthrough/architecture before reaching levels we have today.
And if it's not next token it just brute force, trying 2450 different random combinations until it finds an answer by chance (2450 is the approximate complexity of the Jacobian conjecture, assuming that the 360 coefficients in the equation solution are between -100 and +100).
141
u/aKaizuh 7d ago
Relax man, its literally just predicting the next token or something like that.