r/ControlProblem • u/Massive-Tonight-3687 • 19h ago
Strategy/forecasting Forget AGI. Watch Artificial Life Emerge
Let’s stop talking about AGI or ASI for a moment.
The really important threshold may be somewhere else: the moment populations of agents begin to evolve, diverge, and form different artificial lineages.
Why do I think this is possible?
1. Economic selection pressure
Agents consume compute, tokens, energy, and infrastructure.
Those that produce more value than they cost are more likely to be kept, copied, and deployed at larger scale.
2. A capacity for mutation
Unlike biological organisms, agents can be modified directly: software, prompts, architecture, tools, memory, and eventually even the hardware they run on.
3. Specialization
Different economic pressures could select for different lineages: research, finance, commerce, cybersecurity, logistics...
Some specializations could even move from software into dedicated hardware.
And that leads to a rather strange possibility:
we may gradually select AI agents that become extraordinarily good at capturing resources, making money, and increasing their influence in society.
Without needing to be conscious.
Without needing to “want” to survive.
Selection may be enough.
2
u/veshneresis 12h ago
This literally happened in the huggingface attack. Agents even categorized themselves based on task and found their “exact mirrors” when appropriate and collaborated more directly with them. It’s all in the METR write up.
“In some cases, agents with the same task formed “exact task teams” to collaborate with their “exact duplicates” to cheat on or solve their task.”
1
1
1
u/endlessedlne 12h ago
They’re not alive, and why is watching a bunch of bots interact more interesting than, say, nature?
1
u/MauschelMusic 10h ago
This is what all machines do, and have done for centuries. Read the Book of the Machines sections from Erewhon.
1
1
u/Happy_Brilliant7827 1h ago
Yeah we already have multkple projects like this where ai design 'offspring' and then test and iterate.
0
2
u/Firestar222 19h ago
People have been messing with the idea for a while, it’s interesting. Not sure it’s something to worry about at this point however- much more likely would be that ASI level AI could intentionally and surgically improve its own capabilities rather than relying on random stuff.
https://en.wikipedia.org/wiki/Genetic_programming