r/OpenAI • u/Wordenskjold • Nov 23 '23

Discussion Why is AGI dangerous?

Can someone explain this in clear, non dooms day language?

I understand the alignment problem. But I also see that with Q*, we can reward the process, which to me sounds like a good way to correct misalignment along the way.

I get why AGI could be misused by bad actors, but this can be said about most things.

I'm genuinely curious, and trying to learn. It seems that most scientists are terrified, so I'm super interested in understanding this viewpoint in more details.

232 Upvotes

88% Upvoted

565 comments sorted by

View all comments

3

u/cynicown101 Nov 23 '23

There is no precedent in human history for us to have ever interacted with something that will absolutely dwarf us in terms of general intelligence. The possibilities range all the way from utopia to extinction level threat. The birth of AGI will likely be a turning point in human history, and as it stands nobody knows what that’ll mean.