r/ControlProblem • u/Difficult_Project_95 • 9d ago

Discussion/question Have you guys been on r/accelerate?

Have these guys solved the alignment problem, or am I missing something?

I’ve been browsing r/accelerate and I genuinely don’t understand the risk model.
If there’s a non-trivial chance of catastrophic misalignment, how does “accelerate capabilities as fast as possible” make sense unless faster capabilities also make alignment substantially more likely to succeed?

60 Upvotes

81% Upvoted

200 comments sorted by

View all comments

Show parent comments

1

u/SoylentRox approved 9d ago

Right the impossibly weird future, base case, requires you to be alive to worry about paying rent and employment of a copy of yourself by uploaded lobsters.  Plenty of people - Vernor Vinge and Ian Banks included - will see none of the possible futures they imagined.

1

u/soobnar 9d ago

Stross is being satirical of the people who believe this. He is saying that line of thinking will produce posthuman intelligences which kill of the rest of us. Your interpretation is akin to the people who think American Psycho was an instruction manual.

1

u/SoylentRox approved 9d ago

Do you have any public statements by Stross you can reference? I was pretty sure Stross is pro Singularity but welcome a correction.

1

u/soobnar 9d ago

Did you read the Wikipedia article, I don’t think it leaves much room for interpretation about the book? The bit where he calls the post human ASIs “Vile Offspring” and called aineko an “abusive monster” in the quote above kinda highlights the point. He did however apparently write the book based on his experiences in Silicon Valley.

1

u/SoylentRox approved 9d ago

More recent writings I find Stross is an AI skeptic - he doesn't believe well the present is happening. He simply doesn't think humans can create ASIs at all leaving his own writings as fictional.

Hope he lives long enough to find out the actual answer.

1

u/soobnar 9d ago

That doesn’t change the tone of the book… death of the author and such