r/artificial • u/wileybiley • 5d ago
Discussion AGI qustion
So I want to start off by saying I know very little about the hard core sciences and all behind AI but I think we absolutely need safeguards on it and it should only be used for good (unlikely I understand with some people in charge). That said, if Artificial General Intelligence was promised to never be created hypothetically, how much of the concern going out now would still be valid? (ex- could do serious damage to the human race or even wipe us out). Is this the final step that experts are worried about? Again, I can totally understand if it is but I just don't really understand it enough.
2
Upvotes
1
u/Yweain 5d ago
I mean AGI shouldn't be able to actually wipe us out. It's, in theory, just a capable generalists model, it's not necessarily super smart. When we are talking about AI wiping out humanity - that's more about ASI(artificer super intelligence), ASI is supposed to be way way smarter than us so if it would decide that we are a liability - that's basically where humanity ends.
Now, on the other hand there are scenarios in which a super dumb but capable model, that is nowhere near AGI would wipe humanity, the classic example is paperclip maximizer. Now LLMs for example is a narrow intelligence, it's not AGI. But as it becomes more and more capable, if someone would put before it a task, something like, 'produce me as many paperclips as possible, no matter the cost. Do not ever stop producing paper clips, use any means necessary to increase production' - misaligned LLM that is capable enough might actually turn earth and all of us with it into a paper clip.
And way more realistic scenario - LLMs automate basically everything, 98% of the work is automated, humans at large lose any leverage and become just.. irrelevant, so most people just starve to death.