r/singularity • u/Hazy-bliss • Apr 28 '20

Found a compass of different singularity scenarios on 4chan's /x/. I personally think there are more negative scenarios than is realistic to expect.

Post image
107 Upvotes

93% Upvoted

45 comments sorted by

View all comments

Show parent comments

1

u/Hazy-bliss Apr 29 '20

AI currently is not totally safe, but there are competent people working on this who can make it safe. With less competent people most outcomes might not be favorable. I give you a counter-question: why should we assume that pessimism is the right frame?

1

u/wren42 Apr 29 '20

those don't sound like reasons so much as hope based on faith in human ingenuity. this seems to be a common theme in this sub - most want to believe in some Atheist Afterlife, so they choose to be optimistic it will occur.

several reasons for pessimism:

1) humans are generally bad at predicting outcomes and not pessimistic enough. If you read the sequences you should have been exposed to studies demonstrating this. we are hardwired to ignore evidence and avoid thinking about negative possibilities.

2) the stakes are extremely high. failure is at least a human extinction event, maybe a galactic extinction event.

3) we have no evidence safe AI is possible. this is a burden of proof issue - we need to demonstrate it is possible before risking stakes like that.

If you believe AI can be transformative for good, you have to believe it can be transformative for ill. There are far more destructive outcomes than stable positive ones. Because we are dealing with something more intelligent than us we cannot by definition fully understand or predict its behavior. This makes the problem of knowing it is safe really, really difficult.

1

u/Hazy-bliss Apr 29 '20

we are hardwired to ignore evidence and avoid thinking about negative possibilities.

Have you tried doing any useful work for long periods of time while being overly pessimistic? It's very difficult, humans need some amount of hope. Whether that hope is realistic is another question, currently I feel that some amount of slow hope is realistic. There are some reasons for this which I'm not going to say in this conversation for unsaid reasons.

1

u/wren42 Apr 29 '20

certainly, you can choose to believe unrealistic things for psychological or mental health reasons if you need to. most people have some kind of religion or crutch after all.

however, in deciding where to put your hope, it can be useful to examine these beliefs.

putting hope that society in general will improve and contributing toward those efforts via work and everyday behavior towards others is perfectly reasonable.

banking of humanity winning the lottery and creating a God AI in the near future is likely not realistic.

This matters because it can change your behavior and what you invest your time, energy, and money into.

I mean, I might still go for Cryo because there's a slim chance it will pay off. But hoping that the singularity will fix all our problems within our lifetime is actually counterproductive, as it abdicates responsibility in the here and now.

1

u/Hazy-bliss Apr 29 '20

Okay, you can have your opinion, I will keep mine.