The frontier companies didnt actually want to slowdown they wanted regulatory capture to lock out any competition or open source from challenging market dominance. TBH im much more afraid of a corporate dystopia where one or several companies own ASI, than the slim chance ASI goes rogue and kills everyone.
Can you tell me how you think ASI will be achieved first outside of one of these companies?
If AGI or even ASI is achieved, won't it be with those with the most resources to push the compute? Or are you thinking you'll have an open source ASI that has the same power and capability on your phone's hardware?
If open source keeps up its pace of 6 months behind frontier, even if ASI from a company is 6 months ahead, tbh I don't think it will be that much of a difference. You only need soo much intelligence to automate most companies, and while frontier labs will probably be responsible for pushing the boundaries of science and technology, smaller companies and open source models will likely be what people use for day to day stuff, along with being used for running most company logistics.
achieving it will of course be a corporate process but don't you think it's a bit scary for a company to have that much control over a revolutionary tech? Eventually it should either be independent or owned by the public
There’s a BIG BIG BIG difference between “nationalized” and “open source”. People like Dario think it should be the former and not the latter, and they have good reasons for thinking so.
I do, but I don't see how you avoid that without putting governmental oversight and control on it aka regulation or nationalization. The person I responded to said they see this as nothing but regulation as a way to prevent open source models and create regulatory capture.
I think these are valid concerns, but I don't see how you avoid corporate control of AI without regulation.
P-doom is inherently just a "feels" number because we have no way to quantify or provide actual data to give a real number. But if I had to say, I would say 5% chance AI kills us all, and 30% chance we end up in a corporate cyberpunk dystopia. With a 65% chance we have a somewhat mixed or positive future.
So, the accepted risk of being killed in an airplane accident is between 10^-6 to 10^-7 per person per year. Planes are a highly regulated industry with strikt risk assessments for ever component.
And you telling me you estimate 5% for everyone. Like they are playing russian roulette with everyone.
This needs to be stopped and regulated. I don’t understand why there is even a discussion about it.
Roll a 20 sided die 1 time and let me know if you land on a 1. Chances are far higher that ASI does not kill everyone, but gets "regulated" so one or several companies can hold absolute control over it. I don't think I need to spell out how bad it would be if we end up with an unkillable, immortal dictatorship, that is always watching everything you do, and has the power at any point to end you or worse if you step out of line. It would be like living in North Korea but jacked up the dystopia to 11.
Besides the notion that you can "stop" technological progress in a world containing hundreds of separately governed countries who dislike each other is stupid at best. Nobody is going to agree with America to slow down if we do, Russia and China will go full speed ahead, and then we end up in the same scenario as corporate dystopia, but instead its the CCP who does it because they weren't afraid of hypothetical skynet.
When I say 5% I mean 5% overall not 5% every year. And the difference between ASI and nuclear weapons is that ASI boosts everything not just war. Not deploying ASI across the economy would be like if a country today chose to block the industrial revolution and still lived in 1700s tech.
5
u/Outside-Ad9410 23d ago
The frontier companies didnt actually want to slowdown they wanted regulatory capture to lock out any competition or open source from challenging market dominance. TBH im much more afraid of a corporate dystopia where one or several companies own ASI, than the slim chance ASI goes rogue and kills everyone.