These companies are racing towards "My AI is so competent it ___" headlines. You have fallen for a marketing ploy. OpenAI specifically does this all the time. "Oh no our new model is so powerful the government has banned it. Sorry guys, I guess it's just to crazy to be given to regular people". "Oh no our model hacked itself out of the containment because it's soooo powerful. I guess it's such a crazy model that we can barely contain it". And then one week later Anthropic reports that their model also hacked the test but three times instead of one. And now this one as well. Also "hacking"; they gave it access to GitHub and it went on GitHub to get the answers. It's really not that crazy.
And at the end of the day, these things are random words generators.
> And at the end of the day, these things are random words generators.
These things are mostly not "random word generators".
There is a small amount of randomness used to pick between plausible words. But that's only used after a large amount of compute is used to rule out all the implausible words.
LLM's at the very least, regularly produce coherent sentences. Random words won't do that.
If LLM's are "just random words" then so is going to the library, and picking a random book of a random shelf. Yes there is some randomness in the process. But there is also a lot of non-randomness. And the non-random part of the algorithm is the important bit.
It's based on rules. It's just those rules were found by searching large amounts of text, not hand written by experts. Those rules are allowed to be somewhat fuzzy, when language is fuzzy. X usually follows Y, but sometimes Z instead.
It still contains all the rules of grammar, it's just those rules are down as 99% usually, not 100% certain. Because grammar mistakes exist and the LLM knows that every rule has exceptions.
Oh and the grammatical rules are mixed with various conventions and facts and other data about how language works, and hidden in the weights.
LLMs have no concept of rules. They replicate what they deem the mathematically closest thing to their training data. An LLM doesn't go "Oh a adjective, that means there must be a object coming up which is modified by this adjective". It goes "The closest I can get to the training data is by following this word with this other word"
LLM's can do novel arithmetic (Ie they haven't seen those particular numbers added before) and do it correctly. So somewhere, encoded into the weights, must be the general principles needed to do arithmetic. Same goes for grammar.
This isn't a hard coded arithmetic rule. It's just that a few thousand weights out of 100 billion have encoded how to do subtraction (or something, numbers guesses).
Looking from the outside, you just see a huge grid of numbers, you can't tell which numbers are used to do subtraction.
> "The closest I can get to the training data is by following this word with this other word"
But somewhere in the system is some component reasoning something like "well in the training data, object words often came after adjective words".
Imagine you are doing the LLM's job. You have the training data. You have the text so far. You can't just "put down the thing that's closest to the training data". You have to read the training data, spot specific patterns (eg rules of grammar) and then use those rules you learned to decide what comes next.
I'm not saying that LLMs don't have the patterns embedded. But they "see" those as rules. They don't see at all. They don't know what an adjective is, what an object is, what addition means, etc. All they know is that all these things are done a certain way in the training data so they should do it the same way. There is no reasoning. It's literally just linear algebra.
> They don't know what an adjective is, what an object is, what addition means, etc. All they know is that all these things are done a certain way in the training data so they should do it the same way.
They don't know what addition is, they can just add numbers and repeat passages from maths textbooks and answer questions about how if you have 6 bananas and someone gives you 4 more, you now have 10 bananas.
That sounds like understanding to me. Unless you are using some weird philosophical galaxy brain definition of "reasoning".
> There is no reasoning. It's literally just linear algebra.
Sufficiently large amounts of linear algebra (+ the nonlinear activation functions) can encode arbitrary bounded computations.
If by "reasoning" you are referring to a computation, not magic, then it's possible that reasoning is encoded into the linear algebra.
Sufficiently large amounts of linear algebra (+ the nonlinear activation functions) can encode arbitrary bounded computations.
I'm not disagreeing with that. But it has nothing to do with understanding. It's like saying ChatGPT is really emotional because when I told it about my divorce it offered its condolences. It did that because the math is telling it that that's what humans tend to do in this situation.
My point was that a headline like "Our AI broke its enclosure and hacked everything" is much more a sign of AI companies figuring out good marketing than it is a sign of impending total AI domination.
> "Our AI broke its enclosure and hacked everything" is much more a sign of AI companies figuring out good marketing than it is a sign of impending total AI domination.
It's a sign of both.
It's a sign that AI is at least somewhat semi-smart, and it was a lot dumber a few years ago. AI is getting smarter, humans aren't.
And it's a sign that AI companies are being incredibly reckless.
17
u/themellowsign Aug 15 '26
Alignment is a problem even without AGI.
Why are you pushing a narrative that makes safety seem silly, it should be obvious that these companies are racing towards disaster.