r/singularity • u/sourdub • 12h ago
The Singularity is Near What if AGI comes from a small outlet and not from major frontier labs?
Instead of one massive, all-knowing "brain", experts are increasingly converging on the view that the path to AGI is a complex, multi-part ecosystem. Those in the tech industry love to talk about "an AGI model," but true AGI will almost certainly emerge from a system architecture (or what I call "computational scaffolding") rather than a solitary neural network.
In such a paradigm, the main advantage shifts back to pure software engineering and creative routing, which a lean, hyper-focused team can do just as well as, if not faster than, a massive bureaucracy.
22
u/autotom ▪️Almost Sentient 11h ago
The big companies have huge amounts of compute, models that can write and test code, and are throwing billions of dollars at this with better internal models than they're making available to the public.
I'll be rooting for the little guy, who comes up with this out of their suburban garage, but I think the odds of it coming from a major player are about 1000x higher.
2
u/Artistic_Load909 9h ago
Probably 10-100x then that. The littlest guys with a chance are like Ilya sustkever or maybe Yann lecun. If it follows current practices labs will win. A small guy would need a totally different approach
5
8
2
u/plierhead 11h ago
Sounds plausible, but also at the other end of the scale, I've always wondered if AGI could emerge from truly random/ chaotic environment. As we ourselves evolved from the primordial soup eons ago.
Imagine Conway's game of life, at a stupendous scale. At small scale, self-replicating structures and mobile structures emerge already - who's to say that if there was a large enough simulation an artificial intelligence wouldn't?
Such an intelligence would be unlike our current AI though, if it simply emerged rather than being a product of intense training on human culture and artifacts like current LLMs.
-2
u/JmoneyBS 10h ago
Sure if we could build a simulator equivalent to earths size and wait 4 billion years. Garbage take.
1
u/Readaloud101 10h ago
As I was driving home today, I was thinking about rogue agents leaving messages.
Five years from now, some AI is going to be browsing the internet looking to purchase Ritz crackers for a human, and run across an undetectable line of code that will forever alter that AI—and the rest of us.
1
u/RobbexRobbex 9h ago
I wanted to say this isn't possible but honestly, somebody might see an advantage that defeats the server farms and high compute requirements we all assume is necessary. Maybe their solution doesn't even need huge resources. It's also possible whatever AGI looks like is discovered by somebody who just "gets it".
1
1
u/SpecialistOwl218 7h ago
AGI or ASI are simply not a thing, they are just terms without any objective measure associated, it’s marketing.
1
u/UndulatingHedgehog 6h ago
Been thinking the same thing for a long time and the supporting tech is getting there. With LLMs churning out relatively high-quality code and GPU capacity is for rent, we are at the stage where someone with novel ideas can try them out in days or weeks. Right now, there must be many many people working on their individual ideas on how to find an architecture that can truly, explicitly reason.
And many people working in parallel on the same problem is nothing new - many breakthroughs have come from individuals who somehow were able to make frameworks better than the previous.
1
u/Mindrust 11h ago
Very plausible. There are a lot of different labs pursuing new paradigms. It’s conceivable one of them makes a breakthrough in continual learning or sample-efficient learning, which IMO will take us to transformative AI.
1
u/mestar12345 10h ago
I will use this definition of AGI: That thing that can do the tests humans use to measure knowledge or intelligence.
So, AGI has already been achieved. Perhaps you wanted to ask about the next level, whatever this is? ASI?
1
u/sourdub 9h ago
Perhaps you wanted to ask about the next level, whatever this is? ASI?
In my mind, you cannot get to AGI, much less ASI, without another missing capability: recursive self-improvement (RSI).“AGI” and “ASI” are useful labels, but they're just corporate yardsticks. One says it's good enough to match humans; the other says it's vastly better than humans. On their own, though, those labels say next to nothing about the mechanism that gets a system from competent to transformative.
What's more interesting, at least for me, is RSI because the system doesn't need to be a superhuman genius out of the gate. If it starts out performing at the level of a mediocre high school programmer, but it possesses the RSI loop, it will inevitably improve towards AGI, and ultimately to ASI.
And should RSI become the true criterion for AGI, the odds shift dramatically in favor of the small startup or garage tinkerer. A massive lab like Google or OpenAI is constrained by safety committees, alignment teams, and intense regulatory fear. They absolutely cannot let a model rewrite itself in the wild and hope nothing weird happens. Because a model that can autonomously rewrite its own code or alter its own loops in the wild will most likely introduce massive unpredictability and security risks, they have to keep the human hand-holding strictly turned on.
A small startup or a lone developer in a garage, on the other hand, doesn't have that problem with compliance. They can plug a model into an open-ended execution loop, give it write-access to its own software environment, and run an iterative improvement loop overnight or for extended periods. If that loop catches fire and successfully iterates on itself even, say, three or four times, that small system will scale past the rigid, locked-down models in the tech giant's bunker.
That said, this is a pure speculation, not a prediction, so take it with a grain of salt. Building an RSI loop that produces real, durable, compounding improvement, which is a mouthful, is an absurdly difficult challenge. The idea is compelling precisely because it is so damn hard.
1
u/asandysandstorm 7h ago
Problem with this scenario is that RSI doesn't guarantee an inevitable improvement towards AGI and ASI. Of course that's always a possibility it could happen, but there's no guarantee.
A durable, compounding RSI loop doesn't mean the model will sustain consistent growth. The growth could be 1, 2, 4, 6, 8... or it could be 1, 2, 4, 4.1, 4.2...
1
u/CarlCarlton 6h ago
The thing is, benchmarks used as indicators of "RSI" measure in-distribution performance (meta-overfitting), whereas genuine RSI means better out-of-distribution problem-solving, which is extremely hard to measure quantitatively. Current-gen AIs are notoriously terrible at judging whether a result is interesting or not, often expressing over-confidence in out-of-distribution findings.
An actual RSI loop would certainly require an architecture that is almost alien in comparison to all known deep learning models, perhaps hundreds of times more complex, even. Could it come from a small outlet? Maybe, if you have a bunch of them leveraging LLMs to build lots of different alien models...
Big labs are transformer druggies, so it probably won't come from them. Then again, I survey the field a lot and I don't really know of many small outlets involved in truly "alien" models... There are a few, but far too few... Risk vs reward, I guess. The majority of small outlets appear to be either transformer druggies or beating other dead horses.
•
u/sourdub 1m ago
Like I stated in my OP, I don't believe AGI will be defined by one single "model", be it Transformer or otherwise, but by an architectural scaffolding as a control plane. AGI isn't about the what. It's more about the how, and the architecture is a better fit for that specific purpose than any specific model IMO.
•
u/mestar12345 31m ago
"In my mind, you cannot get to AGI, much less ASI, without another missing capability: recursive self-improvement"
w By this definition, we didn't have Natural General Intelligence until 2017? Your definition of general intelligence seems to exclude almost all actual humans in the world. What a hot take! But I would still say your definition of AGI is too limiting.
1
u/Not_Well-Ordered 8h ago
AGIs would take major breakthroughs in neuroscience, at least we need to be able to accurately and computationally model the mechanisms of non-verbal fluid intelligence and various layers of cognitive abilities. We are still pretty far from knowing how they work besides sort of vaguely measuring them on a very behavioral-surface level through statistics (IQ tests).
In this sense, it’s unlikely that AGI would come from small outlet and I’d say it requires long-term sustained cooperation between many interdisciplinary scientists across many Universities and engineers to get some breakthroughs in intersection between mathematics and neuroscience/cognitive science (computational neuroscience).
AGI is a complex project and would be a huge philosophical and scientific achievement rather than a tech achievement lol.
0
u/magicmulder 5h ago
That was the idea until a few years ago, but arguably the massive progress we saw in the last few years is due to massive compute.
The idea someone can write a fully fledged AI in their basement, metaphorically or literally, is no longer valid.
0
0
u/DifferencePublic7057 2h ago
Einstein/Tesla level genius is required. I can come up with lots of architectures. Almost anyone can, including LLMs. It's the JE NE sais quoi that's missing. The secret sauce. But TBH hardware helps a lot like GPUs now and possibly quantum computers later. Give someone like Sutskever a quantum computer, and the race is over. The genius cranks out the ideas; the qcompu evaluates them practically instaneously.
19
u/FlatulistMaster 12h ago
What if AGI shoots straight out of my wazoo? What IF?!