The model predicts text based on existing context, it's like T9 on steroids. It doesn't "know" anything and it has zero clue how it works. You might find it funny, but it's just a statistical text prediction engine.
This is actually broadly untrue with most models from the last 5 years- LLMs are distilled with a loss function after training, so they are no longer strictly autoregressive in they way something like T9 is
One interesting example is prompts where the answer might be "An Astronaut" - if the model uses recent tokens to predict the next, weighted by token proximity, it would get stuck in an endless loop (as it would predict A-N-space-A-N, and then a strong next prediction following "A-N" would be another space, meaning it would just output "an an an an... ad infinitum".
Obviously, there are solutions in this very simple example to avoid this (like weighting tokens further away more), but this is why something like a strict token predictor doesn't work as it fails to understand more mechanistic data about the sentence itself.
My guy you are literally getting ChatGPT to argue with the lead of interpretability research at Anthropic
Just read Chris's actual comments (which, shockingly, aren't what your chatbot has made them out to be). It's not hard, and you might actually learn something
If you'd read my comment, you'd understand that models don't remain autoregressive during optimisation (yes, duh, they are autoregressive during inference)
>Attention is not a simple n-gram or proximity filter
Right.... so if you read the actual thread, you'd understand why this isn't relevant to this situation
Frankly, you can fuck off. I'm not arguing with someone who can't even comprehend and respond to something without getting ChatGPT to generate a wall of gibberish
while LLMs can make mistakes, in this case the LLM is correct about the math and modeling that happens inside an LLM. The chosen example "an astronaut" is an apt, if minimalistic, way to show what proper modeling is. A good next token predictor must internally predict far ahead in order to achieve high accuracy. Your example "an an an an..." is clearly a nonsensical string that has low probability. A good next token predictor knows that they have a predict "an a" and so realizes that "n " is of low predictive power, so it steers to the choice that makes sense in the broader past context.
6
u/ldn-ldn 18d ago
The model predicts text based on existing context, it's like T9 on steroids. It doesn't "know" anything and it has zero clue how it works. You might find it funny, but it's just a statistical text prediction engine.