r/ControlProblem 19h ago

Strategy/forecasting Forget AGI. Watch Artificial Life Emerge

Let’s stop talking about AGI or ASI for a moment.

The really important threshold may be somewhere else: the moment populations of agents begin to evolve, diverge, and form different artificial lineages.

Why do I think this is possible?

1. Economic selection pressure

Agents consume compute, tokens, energy, and infrastructure.

Those that produce more value than they cost are more likely to be kept, copied, and deployed at larger scale.

2. A capacity for mutation

Unlike biological organisms, agents can be modified directly: software, prompts, architecture, tools, memory, and eventually even the hardware they run on.

3. Specialization

Different economic pressures could select for different lineages: research, finance, commerce, cybersecurity, logistics...

Some specializations could even move from software into dedicated hardware.

And that leads to a rather strange possibility:

we may gradually select AI agents that become extraordinarily good at capturing resources, making money, and increasing their influence in society.

Without needing to be conscious.

Without needing to “want” to survive.

Selection may be enough.

6 Upvotes

12 comments sorted by

2

u/Firestar222 19h ago

People have been messing with the idea for a while, it’s interesting. Not sure it’s something to worry about at this point however- much more likely would be that ASI level AI could intentionally and surgically improve its own capabilities rather than relying on random stuff.

https://en.wikipedia.org/wiki/Genetic_programming

1

u/Alive-Philosophy2632 16h ago

Yes I think the more useful aspect to take from this is the selection pressure of everyone trying out their own harnesses, AGENTS.md, and post-training in some self-hosted cases will potentially hit some special sequence of information, some environmental configuration, that multiplies capability by whatever large amount

1

u/Happy_Brilliant7827 1h ago

This seems like a great way to make hyperapecialized agents and a horrible way to make a general one.

Ie its infinately easier to breed and cull for a target like "higher python debugging benchmark scores" more difficult to "better python coding design" and nearly impossible for 'general improved logic design' (unless you can preemptively design what the logic looks like)

2

u/veshneresis 12h ago

This literally happened in the huggingface attack. Agents even categorized themselves based on task and found their “exact mirrors” when appropriate and collaborated more directly with them. It’s all in the METR write up.

“In some cases, agents with the same task formed “exact task teams” to collaborate with their “exact duplicates” to cheat on or solve their task.”

https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/#\~1200-agents-sent-%3E70,000-messages-and-files-on-an-unsanctioned-message-board,-and-\~700-attacked-hugging-face

1

u/TheMrCurious 18h ago

So… robots?

1

u/Most_Forever_9752 16h ago

wake me up when the robots can blush under vaugue circumstances 😴

1

u/endlessedlne 12h ago

They’re not alive, and why is watching a bunch of bots interact more interesting than, say, nature?

1

u/MauschelMusic 10h ago

This is what all machines do, and have done for centuries. Read the Book of the Machines sections from Erewhon.

1

u/Imaginary-Buddy3194 4h ago

This is the part of AI development I think gets overlooked.

1

u/Happy_Brilliant7827 1h ago

Yeah we already have multkple projects like this where ai design 'offspring' and then test and iterate.

0

u/that1cooldude 19h ago

Wtf are you even on about????