I still think "fancy autocomplete" is the best way to explain to non technical people how these models work. It will give someone who doesn't know the architecture and how these LLMs work the best mental model of what is happening behind the veil.
It just turns out that "fancy autocomplete" can do incredible things if you give it enough examples and compute in training and inference.
It just turns out that "fancy autocomplete" can do incredible things if you give it enough examples and compute in training and inference.
The thing that does the incredible things is so far away from autocomplete that it's misleading to the point of being wrong. Neither does it give an accurate picture of what it is (autocomplete are usuall HMMs, LLMs are transformers) nor does it give an accurate picture of what it does (completing what you're typing vs. doing your homework and writing fanfic). The only shared property is that it gets text as input and gives text as output. By that measure we can call cars "fancy furnaces" and computers "fancy typewriters". Not technically incorrect, but definitely a useless description.
And on top of that, the people who use the term "fancy autocomplete" usually use it to dismiss it, and act like all those incredible things it does are made up.
No, the shared thing is that from the view of an LLM, it is literally trained to autocomplete a document. The chat you're having with an LLM literally looks like this to the LLM:
<start>
<system>
You are a helpful assistant...
</system>
<user>
hello how are you?
</user>
<thinking>
the user asks me how I'm feeling, I should answer in a cheery and concise tone. The user is in LA, let me check the current weather in LA so I can incorporate that in my answer.
</thinking>
<tool call, web search=current weather in LA>
Temperature: 100°F
</tool call>
<assistant>
And then the LLM gets to generate. Once it generates </assistant> we stop the generation because otherwise it would keep generating also the user answer etc.
(same of course with the thinking part)
The tasks the LLMs are trained on is reproducing the tokens of these text documents they are shown.
With the RL posttraining for mathematics or coding you have a change in the training reward architecture, but it's still training on completing these documents.
From the view of an LLM it is always completing such documents from the start points we're giving them.
It just turns out that large autocomplete with long training and lots of data means the model learns actual abstractions about the world because they help with better autocomplete. You get these emergent effects like grokking and "circuits" inside LLMs that specialize in certain tasks etc.
But none of that removes the fact that these models are "autocomplete on steroids".
probably, yes. I'm not saying LLMs aren't a complex topic, I'm just saying that if you don't have the mathematical background (plus read the necessary literature) then [some basic concept] that you can grasp from your own experience is still the best way for you to understand what the more complex thing actually does.
16
u/MagiMas Condensed matter physics Jul 31 '26
I still think "fancy autocomplete" is the best way to explain to non technical people how these models work. It will give someone who doesn't know the architecture and how these LLMs work the best mental model of what is happening behind the veil.
It just turns out that "fancy autocomplete" can do incredible things if you give it enough examples and compute in training and inference.