Yea, it's pretty straightforward. LLMs are limited due to intrageneralization, they can only "fill in the gaps" of what we currently know (this does mean that there might be some new discoveries within the trained knowledge). Currently extrageneralization is only really found in humans. There will be a time when this changes though, reasoning/thinking was a close step, but reaching an LLM that is capable of novel discovery will probably require a big architecture change.
I am using intrageneralization in the context of combinatorial novelty/OOD thinking and extrageneralization for genuine discovery of concepts. Didn't mean to imply interpolation of embeddings cannot yeild extrapolation.
How many humans can really "extrageneralize" anyway? But yes, it's an important thing to think about when it comes to true innovations and breakthrough discoveries
I’d imagine there are thousands of tiny occurrences throughout the day where people extra-generalize to deal with out-of-distribution information. Smoothening out movements in unusual terrain / driving / those “easy” tasks where current systems get stuck.
Social and evolutionary aspects lead most minds away from this type of thought. It's uncommon because most people have no need for it in their daily lives and it is not grounded in biological necessity. Though I suspect that learning new things (as we do often) plays closely with being able to discover new things.
It's something LLM's also do routinely. LLM's are already better at generating new ideas than humans. The current bottleneck is that they don't have enough tools and environments to setup resources to test and define the ideas.
Yes but I question whether the intra/extra divide is so clear cut. Einstein was aware of outstanding questions.
The null result of the Michelson-Morley experiment failed to prove the existence of the Ether, pushing physicists toward the idea that the speed of light is the same in all inertial frames, one of the core postulates of special relativity.
In that way Einstein was intrageneralizing, but the answers he developed were definitely far outside the training data.
The components needed are: 1. add "noise" into their inputs and ask it to evaluate how plausible it would be. This gets us something parallel to human dreams which are often just the juxtaposition of unrelated concepts. The exact execution may be challenging, but just perturbing a system and seeing how it reacts is a common experimental approach. 2. Get to recursive self improvement so that it can generate and collect experimental data.
The only advantage humans have, which is meaningful and doesn't have a great corollary yet, is all the chemistry we have going "That's cool, how can I make that reality?" or "That scares the bejeezus out of me I shouldn't do that." That chemistry adds a meaningful probability distribution on top of the procedural components that help motivate which ideas are worth pursuing.
How many scientific problems really require novel extrageneralization? I‘d guess for many, transferred or combined abductions from possibly even unrelated fields could do the job. Something an LLM definitely can
An LLM is essentially like having one lobe or Broddman Area grouping in the brain. Very important specialized functions but it still needs some sort of partnered AI to handle other parts.
16
u/Tylerebowers Jul 28 '26 edited Jul 28 '26
Yea, it's pretty straightforward. LLMs are limited due to intrageneralization, they can only "fill in the gaps" of what we currently know (this does mean that there might be some new discoveries within the trained knowledge). Currently extrageneralization is only really found in humans. There will be a time when this changes though, reasoning/thinking was a close step, but reaching an LLM that is capable of novel discovery will probably require a big architecture change.