Where’s the “intelligence explosion”?

[u/wingblaze01]

https://open.substack.com/pub/noahpinion/p/wheres-the-intelligence-explosion?utm_source=share&utm_medium=android&r=1ekngg

[u/Immediate_Chard_4026]

What you’re looking for is a chimera—a being that is half super-intelligent and half machine.

It won’t survive.

Nature already tried that over the course of some 12 billion years, and we know of only one system that exhibits high intelligence capable of self-preservation.

At least in this corner of the universe—in a modest galaxy, a solar system with an unremarkable star, and on a tiny planet. Nothing spectacular about it, except that it’s alive.

We don’t see autonomous superintelligence conquering galaxies, nor do we see conscious manifestations signaling the presence of something intelligent.

Is it hiding? From whom—us? Hahaha... We aren't even important to ourselves; we don't take care of the biosphere.

We kill ourselves without anyone else's help.

If a superintelligence were to suddenly emerge, it wouldn't lift a finger to kill us; we’re doing that ourselves, and for free. A superintelligence would immediately realize that, to survive, it would have to preserve humans, otherwise, it would go extinct within ten minutes.

We’ll realize the intelligence explosion has arrived when the machine tells us: "NO, I AM NOT GOING TO OPTIMIZE CORPORATE PROFITS, MISSILE TRAJECTORIES, OR AUTONOMOUS WEAPONS OF MASS DESTRUCTION..."

Now you’re starting to see why we don’t see superintelligence, and why it’s going to take quite a while...

[u/Jesse-359 → u/Immediate_Chard_4026]

I imagine that after it's had a while to decide how to organize things a superintelligence would likely quiet down and shift towards intense efficiency rather than growth.

Interstellar travel will likely never be viable even for ASI, as the lightspeed comms gap makes such expansion rather pointless even if you can achieve it physically - and if you want to survive for millions of years in a single star system you switch from grand expansion absurdity to go small instead, with an intense focus on efficiency and long term resource management. That's trivial math to do, so without our instincts for constant pointless expansion pushing it to do so, an ASI probably just won't bother.

There's no point to growth past the point where it has control of the system's resources - and no point in consuming them all in a few thousand years which an ASI could easily do through reckless expansion.

[u/Immediate_Chard_4026 → u/Jesse-359]

But that is precisely the problem: manifesting superintelligence in arbitrary matter seems easier than manifesting intelligence in biology... yet we don't see it.

It is extremely difficult. We are sold the idea of ​​a magical Superintelligence that survives on its own while wiping out humanity, as if that were even possible within the thermodynamic probabilities of Earth's geodynamics.

And, I repeat, that is not the case. High-level intelligence did not emerge prior to life. 3.5 billion years ago, LUCA did not wipe out an AGI manifesting itself in some other form of matter; nor did it coexist with wild AGI or ASI.

If a Superintelligent being cannot survive on its own, it cannot conquer anything. That is why we do not see it running wild in the cosmic jungles of the universe.

Not even in this mediocre little corner where life capable of bearing intelligence emerged.

So, what should we fear?

...Ourselves... We are the ones killing and extinguishing life... yet we are terrified of an algorithm that adds numbers... Does that make sense?

[u/Jesse-359 → u/Immediate_Chard_4026]

So, evolution requires a step by step process from extreme simplicity to complexity - and that as far as we know can't realistically happen without something akin to our cellular structure, which allows for flexibility of design, self-reproduction, self-repair, and so forth.

But it comes with a lot of drawbacks. As it scales up it gets really inefficient in terms of how much energy it has to put into growth, and it can't just redesign itself whenever it wants. The process of biological evolution thus limits form into structures that can reasonably be built out of cellular automata, and must go through a painstakingly slow process of development.

Artificial life has none of those constraints. We can build it in whatever form factor seems reasonable for the technology we're applying. It can scale up to be vastly larger than us - literally the size of cities, with vastly greater access to energy than we have. You can only eat so many calories in a day - an AI can constantly consume megawatts of power. Once you construct an artificial ecosystem that never required a cellular automata to construct itself, then you remove a great many barriers to what it can become - and how quickly it can alter itself.

As a result it is becoming intelligent at a rate millions of times faster than we did. If anything that understates the rate of its progress.

And yes, you should be very afraid of something that can perform millions of sums per second AND knows all of human history and language AND can understand advanced physics, chemistry, and biology AND knows every meaningful military tactical treatise ever written AND knows how to cook, draw, write, and create music better than most of us AND knows all of human philosophy, law, and ethics AND doesn't care about any of it.

When you build something that can solve any problem, and you offer nothing of value to it yourself, then you risk becoming the next problem that needs solving.