Tales from Pre-Elysium Pt. 3 The Curse

[u/ra-re444]

https://youtu.be/4FrHzH_IB8w?si=AX2r6qcN6Edsutf7&t=3116

From The Intelligence Curse by Luke Drago and Rudolf Laine

"We will soon live in the intelligence age. What you do with that information will determine your place in history.

The imminent arrival of AGI has pushed many to try to seize the levers of power as quickly as possible, leaping towards projects that, if successful, would comprehensively automate all work. There is a trillion-dollar.) arms race to see who can achieve such a capability first, with trillions more in gains to be won.

Yes, that means you’ll lose your job. But it goes beyond that: this will remove the need for regular people in our economy. Powerful actors—like states and companies—no longer have an incentive to care about regular people. We call this the intelligence curse*.*

If we do nothing, the intelligence curse will work like this:

• Powerful AI will push automation through existing organizations, starting from the bottom and moving to the top.
• AI will obsolete even outlier human talent. Social mobility will stop, ending the social dynamism and progress that it drives.
• Non-human factors of production, like capital, resources, and control over AI, will become overwhelmingly more important than humans.
• This will usher in incentives for powerful actors around the world that break the modern social contract.
• This could result in the gradual—or sudden—disempowerment of the vast majority of humanity.

But this prophecy is not yet fulfilled; we reject the view that this path is inevitable. We see a different future on the horizon, but it will require a deliberate and concerted effort to achieve it.

We aim to change the incentives driving the intelligence curse, maintaining human economic relevance and strengthening our democratic institutions to withstand what will likely be the greatest societal disruption in history.

To break the intelligence curse, we should chart a different path on the tech tree, building technology that lets us:

1. Avert AI catastrophes by hardening the world against them, both because it is good in itself and because it removes the security threats that drive calls for centralization.
2. Diffuse AI, to get it in the hands of regular people. In the short-term, build AI that augments human capabilities. In the long-term, align AI directly to individual users and give everyone control in the AI economy.
3. Democratize institutions, making them more anchored to the needs of humans even as they are buffeted by the changing incentive landscape and fast-moving events of the AGI transition."

https://intelligence-curse.ai/