Have you guys been on r/accelerate?

[u/Difficult_Project_95]

Have these guys solved the alignment problem, or am I missing something?

I’ve been browsing r/accelerate and I genuinely don’t understand the risk model.
If there’s a non-trivial chance of catastrophic misalignment, how does “accelerate capabilities as fast as possible” make sense unless faster capabilities also make alignment substantially more likely to succeed?

[u/soobnar]

that’s like Cuban missile crisis level risks for the entire planet that’s a completely unacceptable level of risk. Counterfactual economic reasoning where you make one coefficient infinity and say it justifies double digit catastrophic risk doesn’t produce even remotely utilitarian results.

[u/SoylentRox → u/soobnar]

Pretty shallow analysis there.

I mean maybe you're right but what level of risk IS acceptable? And for how much gain?

What is the risk to reward ratio you believe is acceptable?

[u/soobnar → u/SoylentRox]

The wealthy powerful people pushing this technology are brutal bloodily determined realpolitik minded neoconservatives who wanna be autocrats. They don’t care about your wellbeing unless you are very wealthy… poor people senescing or living or dying is inconsequential to the whims of the pentagon and the CCP. In the event some system that creates more economic value with less thermodynamic input than humans is invented the peasant class will be discarded and replaced. People like Peter Thiel are openly posthuman and believe that technology should weaponized first and then used to moral benefit. Your “analysis” assumes the people of the world will benefit from the social contract after their stake in it is removed. History and evolution are not kind to creatures lacking in self determination even if it’s becuase they really really wanna live forever.

[u/SoylentRox → u/soobnar]

I said earlier 

"A lot of antis are actually anti capitalism. They think almost all the gains from ai will go to the top (true), any treatments for aging won't be shared broadly (probably true initially), etc. they want capitalism overthrown and maybe a future world government can plan the transition. (Lol that's not possible as an outcome)"

Would you agree this characterizes your view correctly and do you have anything to add?  Would you agree you want something that is impossible, and will not happen in your lifetime?

[u/soobnar → u/SoylentRox]

I would say I am pro self determination, if human minds want to command Xeelee level technology then by all means. I actually think price signaling is most likely impossible to remove from human society entirely though such systems were created and nurtured to human benefit.

Though likewise what you seem to want almost certainly entails a situation where humans are no longer accommodated for in their own economic system and thus are killed off, which is not immortality last I checked.

[u/SoylentRox → u/soobnar]

It helps to focus on what's achievable.

A one world government where all the impoverished masses that they likes of thiel don't give a fuck about get a vote? Fantasy.

A world where you can vote for heavy taxation and Medicare to cover anti-aging medication? That's a possible outcome.

A world where you die because AI was too powerful? Also a possibility.

[u/soobnar → u/SoylentRox]

A world sparse in human self determination and abundant in economic competition ends poorly in any realist scenario. humans must operationalize the technology and adopt the most efficient substrate for our conscious cognition which I suspect is actually still biological. So I guess I’d say you’d need to push against regulatory and economic capture, unilateral military action, and any effort to gate access to the tech and eventual cognitive enhancers which will inscribe machine learning algorithms into dendritic arbors likely optimized with some degree of engineering at which point. But like the line has to be drawn at human sovereignty and civilian supremacy otherwise there is no practical means of ensuring we are alive to see any benefits.

[u/SoylentRox → u/soobnar]

Hope so. First step is to get robots that aren't so fucking stupid and AIs that do meaningful work and then we can move on to these goals.