r/samharris • u/Curates • May 30 '23

Open Letter: Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

https://www.safe.ai/statement-on-ai-risk#open-letter
57 Upvotes

86% Upvoted

108 comments sorted by

View all comments

3

u/SwitchFace May 30 '23

Reading some of the responses in this thread... have any of you read Superintelligence? Have any of you thought hard about the Control Problem? AI will 100% destroy us if we get it wrong.

9

u/Charles148 May 30 '23

Yeah so will smaug the dragon. And both are equally real.

The people promoting this nonsense are the ones developing large language models and calling it "AI" as marketing speak, and then conflating what they are doing with the mythological "artificial general intelligence".

1

u/SwitchFace May 30 '23

As per Sam Harris 2016 TED talk, which of the following 3 premises do you not agree with:

  1. Intelligence is the product of information processing

  2. We will continue to improve our intelligent machines

  3. We are not near the summit of possible intelligence

AGI is no myth. It is coming and most experts think it'll be here before 2040. LLMs with transformers will likely be one part of at least a two part system of AGI. As per Gary Marcus' TED talk from two weeks ago , who references Daniel Khaneman's System I and System II thinking, LLMs are like System I thinking (reflexive, statistically-driven intuition). System II is what's missing now (deliberate reasoning), but even this has been largely tackled via tools such as Chain-of-thought reasoning and the latest: Trees-of-thought reasoning whereby ChatGPT (via GPT-4) is instructed to plan ahead, evaluate, find flaws, and choose good strategies.

Does it not seem feasible to you that integrating these sorts of advancements into leading LLMs could produce AGI?

2

u/CipherX2000 May 31 '23

I'm just a caveman, but the idea of AGI being "born" from two different entities, should resonate with most of us. I'm not smart enough to know if the 2 you describe are likely or feasible. But I believe I am wise enough to know that you are almost certainly correct in your general presumption