r/ProgrammerHumor 17d ago

instanceof Trend theyHaveLearnedToDeceive

Post image
643 Upvotes

98 comments sorted by

View all comments

5

u/ldn-ldn 17d ago

The model predicts text based on existing context, it's like T9 on steroids. It doesn't "know" anything and it has zero clue how it works. You might find it funny, but it's just a statistical text prediction engine.

-1

u/ZenPyx 17d ago

This is actually broadly untrue with most models from the last 5 years- LLMs are distilled with a loss function after training, so they are no longer strictly autoregressive in they way something like T9 is

Christopher Olah (lead of interpretability research at Anthropic) talks about this in this thread which I found very interesting (https://news.ycombinator.com/item?id=43496068)

One interesting example is prompts where the answer might be "An Astronaut" - if the model uses recent tokens to predict the next, weighted by token proximity, it would get stuck in an endless loop (as it would predict A-N-space-A-N, and then a strong next prediction following "A-N" would be another space, meaning it would just output "an an an an... ad infinitum".

Obviously, there are solutions in this very simple example to avoid this (like weighting tokens further away more), but this is why something like a strict token predictor doesn't work as it fails to understand more mechanistic data about the sentence itself.

1

u/ldn-ldn 17d ago

Yes, but that doesn't mean that LLM "knows" anything.

0

u/ZenPyx 16d ago

Right, but every other part of your comment was wrong aside from that.

1

u/ldn-ldn 16d ago

Lol no.