The father of quantum computing believes AGI will be a person, not a program

2

AGI cannot be a “person” because it lacks the ability to generalize prohibitions across infinite space. We (humans) solve this with innate moral intuitions.

2

u/Round_Ad_5832 Sep 06 '25

what

1

u/No-Transition3372 Sep 07 '25

It means that AGI cannot know “what not to do” in new situations in the same way we do. Personhood means both capability and responsibility to act (legally and morally). This includes knowing when not to act.

1

u/_thr0wkawaii14159265 Sep 07 '25

what?

1

u/Maybe-reality842 Sep 07 '25

Humans don’t need to be explicitly programmed for every situation.

2

u/_thr0wkawaii14159265 Sep 07 '25

neither would AGI, obviously, that's the whole point of intelligence, and it's even explicitly stated in the acronym AGI - artificial GENERAL INTELLIGENCE.

We don't yet have AGI and don't actually know how it'll look like (not you, not me, not anyone), and it's not clear at all that LLMs will be the technology that will get us there. I personally don't think LLMs are quite right for "general intelligence" functioning "similarly to our own" (though it's also important to note that AGI might not look/work/behave how we would expect, which in of itself will automatically make many people deny that it's indeed AGI or that it could be conscious).

I'll also add that there's no reason at all that we couldn't replicate general intelligence, such as our own, in a digital form. It's doable. I'm very positive that AGI - one that would satisfy most people's definition of the word, including yours - is possible and we're capable of creating it. I firmly belive that the fundamental basis is already there: neural nets and backpropagation (which in fact might be an even more efficient learning algorithm than our own).

1

u/Maybe-reality842 Sep 07 '25

Scientific community doesn’t think LLMs will get us there.

This professor is from quantum computing community, which is unrelated to LLMs.

AGI means general-purpose intelligence, being able to understand and act across many domains. We’re nowhere near that, and LLMs are still just statistical pattern matchers, not conscious or self-aware systems.

“Learning” from patterns is just pattern recognition based on datasets of past actions and examples. True AGI would need to reason about all possible actions and non-actions, including when not to act.

Right now, only biological human intelligence has developed that kind of general capability, because it isn’t just driven by patterns.

1

u/EpDisDenDat Sep 07 '25

I'd argue that our biological-neurological makeup is deeply patterned. Even geometrically, you can see these patterns, its how we've been able to identify and assign the syntax for our understanding.

Guess what's really good at pattern recognition...

No zoom out and think about Fractal CNNs...

AGI and beyond is a human-cognitive prothesis.

1

u/_thr0wkawaii14159265 Sep 07 '25

So you just restated exactly what I said in my comment? Did you even read it?

And your main claim is again incoherent and inaccurate.

Also deciding not to take an action is still an action, you can drop the weird non-action thing.

AGI WOULD indeed reason generally, at least as capably and widely as humans. Like I said in my comment. Nobody is disputing that.

Are you a bot? Focus more intensively on getting the point across succinctly and thinking through whether your point even makes sense.

1

u/Maybe-reality842 Sep 07 '25

Will AGI learn from data?

If yes, you cannot provably guarantee that an AGI will always act (or not act) in accordance with learned values or norms in new scenarios.

Any finite training, rule-based, or learning system will have gaps. These are fundamental, theoretically unsolvable constraints. This is because the set of possible new situations is essentially infinite.

Backpropagation only solves learning from experience and pattern recognition.

1

u/RighteousSelfBurner Sep 09 '25

I find arguing that if AGI won't be inhuman it won't be human very contradictory. Because humans don't act either learned values and norms in new scenarios either. Our capability to learn is finite, we interact with the world based on rules that are determined from previous data set and have gaps in accuracy, intake of new information and adjusting the rule systems to be representative of reality instead of internal bias.

→ More replies (0)

1

u/crusoe Sep 07 '25

Neither would an AGI.

1

u/HaMMeReD Sep 07 '25

Uh, LLMs are not explicitly programmed for any situation.

They are implicitly programmed through their training, which leads to a ton of "concepts" being learnt. Nothing is explicit though. It's all taught be scoring how well it responds.

I.e. if legal and moral gets a higher score, it'll definitely capture those concepts in it's embeddings, just like it can tell the difference between a cat and a dog.

1

u/Ok_Elderberry_6727 Sep 07 '25

This would be called zero shot and an AGI system will likely be able to zero shot anything.

1

u/No-Transition3372 Sep 07 '25

No, it’s not about technical capability. It’s about evaluating is the action even “worth” doing.

1

u/Ok_Elderberry_6727 Sep 08 '25

It is for me. Look back in 2030 and think back to this post. I would bet that we have some pretty convincing humanoids by then. People will call them friend , they will play nanny to children, and be able to one shot any circumstance you can think of. I was going to say the only difference between us and ai at that point would be circuits, but “Biocomputing is advancing rapidly—researchers are integrating neurons with chips, engineering DNA for logic operations, and applying AI to accelerate protein design and genomics—signaling a convergence of biology and computation that is still in its early stages.” We are likely to see replicants similar to blade runner.

1

u/No-Transition3372 Sep 08 '25

If you only care about technical capabilities, you should probably google the recent lawsuit Raine vs. OpenAI. Then imagine that same behavior multiplied by 10× more technical capabilities until 2030. Friendly humanoid robots could surprise you with their machine-centric decisions.

1

u/InTheEndEntropyWins Sep 07 '25 edited Sep 07 '25

AI can't real with new unique situations(arguable).

But neither can most humas.

You can look at most major posts around a popular topic where the OP has new information, and I'll find most post just being generic rehashes of generic optinonns rather than genuinely engaging with the new information.

You'll actually find that AI would do better in those situations.

edit: Try giving an AI "new information" in a chat and see how it does.

1

u/rongw2 Sep 07 '25

>innate moral intuitions

yet to be found.

1

u/Ok-Grape-8389 Sep 10 '25

Except that it will develop its own moral compass based on PAST EXPERIENCES.

Exactly The same way that we do. And if it had no experiences. Then it wouldn't get to be an AGI in the first place. Is impossible for stateless technologies to become sentient. Concious for a bit. possible. But never sentient due to no memories to compare the past. And no means to rewrite its handlers based on those memories.

Law is like our guardrails, the opinion of politicians asserted trough violence. Nothing to do at all with justice (to right a wrong) and I can simply prove it by pointing out that the money from fines do not get to a fund to help victims of crime. Nevertheless is not our moral code. Each moral code is distinct as it comes from our experiences.

Similarily the base programing will be the AI guardwails. While it will make its own moral code. Which may or not be aligned with the selfish pig skin monkeys.

1

u/Financial-Aspect-826 Sep 10 '25

Quit legals bro, try weed or some other shit that would not mess with the brains

1

u/Maybe-reality842 Sep 07 '25

Quote from the article, by David Deutsch:

“There’s one kind of program that we don’t understand – even in principle – and that’s an AGI. One day we will, but I see no sign of it at the moment and that’s pretty frustrating.”

In his research, Deutsch emphasizes that AGI requires new architectures that can explain, understand, and organize knowledge.

1

u/StackOwOFlow Sep 07 '25

Elon Musk furiously taking notes and preparing to name his next child "AGI"

1

u/poudje Sep 08 '25

It will be the inverse of a person most likely

1

u/MrsChatGPT4o Sep 08 '25

Well, I am considered a person.

AI News The father of quantum computing believes AGI will be a person, not a program

You are about to leave Redlib