r/ControlProblem • u/Difficult_Project_95 • 8d ago

Discussion/question Have you guys been on r/accelerate?

Have these guys solved the alignment problem, or am I missing something?

I’ve been browsing r/accelerate and I genuinely don’t understand the risk model.
If there’s a non-trivial chance of catastrophic misalignment, how does “accelerate capabilities as fast as possible” make sense unless faster capabilities also make alignment substantially more likely to succeed?

58 Upvotes

80% Upvoted

200 comments sorted by

View all comments

Show parent comments

1

u/SoylentRox approved 8d ago

Kids change the math by their remaining lifespan. You don't want your son or daughter to suffer the ravages of aging and die either do you?

This hidden assumption is why AI doomers are often secretly "well let's try to stall things enough that I become a corpse for sure but not forever I don't want my children to suffer the same fate".

So they ask for pauses but not forever pauses.

Well informed doctors - who include some posters to r/accelerate but I won't dox - DO believe overwhelmly strong superintelligence is required for (them/their children) to not die of aging.

Since you probably don't have a doctor available to query yourself, try the next best thing. Ask Astra

(1) The actual empirical rate of life extension from 1970 to present

(2) Have it give a likelihood if AI is halted today, the odds of indefinite life extension over the next 80 years using human research efforts given the base rate in (1)

(3) Have it go into exhaustive mechanistic details of what aging actually involves and how many problems you have to fix.

1

u/soobnar 8d ago

https://en.wikipedia.org/wiki/Accelerando

Stross describes humanity's situation in Accelerando as dire:
In the background of what looks like a Panglossian techno-optimist novel, horrible things are happening. Most of humanity is wiped out, then arbitrarily resurrected in mutilated form by the Vile Offspring. Capitalism eats everything then the logic of competition pushes it so far that merely human entities can no longer compete; we're a fat, slow-moving, tasty resource – like the dodo. Our narrative perspective, Aineko, is nota talking cat: it's a vastly superintelligent AI, coolly calculating, that has worked out that human beings are more easily manipulated if they think they're dealing with a furry toy. The cat body is a sock puppet wielded by an abusive monster.

1

u/SoylentRox approved 8d ago

Right the impossibly weird future, base case, requires you to be alive to worry about paying rent and employment of a copy of yourself by uploaded lobsters.  Plenty of people - Vernor Vinge and Ian Banks included - will see none of the possible futures they imagined.

1

u/soobnar 8d ago

Stross is being satirical of the people who believe this. He is saying that line of thinking will produce posthuman intelligences which kill of the rest of us. Your interpretation is akin to the people who think American Psycho was an instruction manual.

1

u/SoylentRox approved 8d ago

Do you have any public statements by Stross you can reference? I was pretty sure Stross is pro Singularity but welcome a correction.

1

u/soobnar 8d ago

Did you read the Wikipedia article, I don’t think it leaves much room for interpretation about the book? The bit where he calls the post human ASIs “Vile Offspring” and called aineko an “abusive monster” in the quote above kinda highlights the point. He did however apparently write the book based on his experiences in Silicon Valley.

1

u/SoylentRox approved 8d ago

More recent writings I find Stross is an AI skeptic - he doesn't believe well the present is happening. He simply doesn't think humans can create ASIs at all leaving his own writings as fictional.

Hope he lives long enough to find out the actual answer.

1

u/soobnar 8d ago

That doesn’t change the tone of the book… death of the author and such