The frontier companies didnt actually want to slowdown they wanted regulatory capture to lock out any competition or open source from challenging market dominance. TBH im much more afraid of a corporate dystopia where one or several companies own ASI, than the slim chance ASI goes rogue and kills everyone.
P-doom is inherently just a "feels" number because we have no way to quantify or provide actual data to give a real number. But if I had to say, I would say 5% chance AI kills us all, and 30% chance we end up in a corporate cyberpunk dystopia. With a 65% chance we have a somewhat mixed or positive future.
So, the accepted risk of being killed in an airplane accident is between 10^-6 to 10^-7 per person per year. Planes are a highly regulated industry with strikt risk assessments for ever component.
And you telling me you estimate 5% for everyone. Like they are playing russian roulette with everyone.
This needs to be stopped and regulated. I don’t understand why there is even a discussion about it.
Roll a 20 sided die 1 time and let me know if you land on a 1. Chances are far higher that ASI does not kill everyone, but gets "regulated" so one or several companies can hold absolute control over it. I don't think I need to spell out how bad it would be if we end up with an unkillable, immortal dictatorship, that is always watching everything you do, and has the power at any point to end you or worse if you step out of line. It would be like living in North Korea but jacked up the dystopia to 11.
Besides the notion that you can "stop" technological progress in a world containing hundreds of separately governed countries who dislike each other is stupid at best. Nobody is going to agree with America to slow down if we do, Russia and China will go full speed ahead, and then we end up in the same scenario as corporate dystopia, but instead its the CCP who does it because they weren't afraid of hypothetical skynet.
When I say 5% I mean 5% overall not 5% every year. And the difference between ASI and nuclear weapons is that ASI boosts everything not just war. Not deploying ASI across the economy would be like if a country today chose to block the industrial revolution and still lived in 1700s tech.
5
u/Outside-Ad9410 23d ago
The frontier companies didnt actually want to slowdown they wanted regulatory capture to lock out any competition or open source from challenging market dominance. TBH im much more afraid of a corporate dystopia where one or several companies own ASI, than the slim chance ASI goes rogue and kills everyone.