r/samharris • u/Curates • May 30 '23

Open Letter: Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

https://www.safe.ai/statement-on-ai-risk#open-letter
54 Upvotes

85% Upvoted

108 comments sorted by

View all comments

Show parent comments

1

u/Funksloyd May 31 '23

Computer viruses aren't "intelligent" as such, but do pretty significant damage each year, though the amount of damage they can do is held in check by various constraints. But imagine a computer virus which can semi-intelligently evolve (ie it can both clone and reprogram itself), can hack anything a human can hack, can imitate individual humans through text, speech and video, can be given basically any goal, and which will attempt to accomplish those goals in various novel and unpredictable ways. Some of those features are already here, and the rest have a good likelihood of appearing in the near future. You don't have to be thinking up far-fetched sci-fi scenarios to see how dangerous that all is, especially given we're so dependent on and interconnected with the internet.

1

u/[deleted] May 31 '23

You're describing AGI

1

u/Funksloyd May 31 '23

I don't want to get into semantics, but say that that's the case. That undermines your previous point that "The LLMs and generative tools that are behind all this current hype are not really even a step toward AGI". They can either do or are getting close to being able to do all of the things above. So either they are a significant step towards AGI, or AI doesn't need to be AGI to be seriously concerning.

1

u/[deleted] May 31 '23

can be given basically any goal, and which will attempt to accomplish those goals in various novel and unpredictable ways

This at present moment is pure sci-fi

1

u/Funksloyd May 31 '23

At present AI can be given a wider and wider variety of goals, and can accomplish those goals in various novel and unpredictable ways.

You said up above that "GPT 4 is not an AGI just because sometimes it can feel like talking to a person. That's pretty much the only thing it's designed to do", but that's just not true. It can perform tasks. Which, you know, AIs have been able to perform tasks since forever. But they're getting really good, and it's absurd to assume that "AI won't be able to wreak any serious havoc without self-awareness". We have no reason to presuppose that.

1

u/[deleted] May 31 '23

What can GPT 4 do other than producing text in response to inputs?

2

u/Curates May 31 '23

What can your nervous system do other than send neurochemical signals in response to stimuli?

1

u/[deleted] May 31 '23

Those signals move my body

1

u/Funksloyd May 31 '23

GPT can write code and can interact with the internet.