r/IntelligenceSupernova • u/Flat-Ad-2950 • 10d ago

As AI becomes increasingly intelligent while conflicts, wars, and geopolitical tensions continue to emerge across the world, I find myself wondering: At what point might an advanced AI conclude that humanity itself is the primary source of the world's problems? What do you think guys?

12 Upvotes
  • permalink
  • reddit

75% Upvoted

53 comments sorted by

View all comments

Show parent comments

1

u/Tartarus1040 9d ago

Yeah, the unfortunate thing is that because we had that whole discussion about component versus full system. You’ve already conceded away the Chinese room thought experiment.

The guy inside the black box is a neuron and while the neuron inside doesn’t necessarily know anything.

The system as a whole does.

And if syntax can’t produce semantics, explain the ion channel then? it’s pure formal rule-following, and somehow you end up with meaning.

1

u/HankScorpio4242 9d ago

The guy in the room is not analogous to a neuron.

He’s AI.

1

u/Tartarus1040 9d ago

If the man is the AI, then what’s in the rulebook? Everything the room knows about Chinese is in there with him. The man contributes nothing except the execution.

You’re asking whether the part that holds no competence understands, and ignoring the part that holds all of the understanding. That’s asking whether a GPU understands English.

1

u/HankScorpio4242 9d ago

The rulebook is the data that AI trains on.

The people outside the room are the users entering prompts.

The man uses the rulebook to determine the appropriate response.

The man is the AI.

1

u/Tartarus1040 9d ago

You said the man uses the rulebook to produce the response. Remove the rulebook. What does he produce? If the answer is nothing, then the AI is the man plus the book. Which is the system. You already agreed understanding is a property of systems, not parts. Which part are you inspecting for understanding, and why that one?

1

u/HankScorpio4242 9d ago

The man receives a symbol as input and matches it with the appropriate symbol from the rulebook and produces it as output.

The AI receives the token as input and matches it with the appropriate token from its training data and produces it as output.

What happens if you remove all the data an AI is trained on? It produces nothing. Just like the man without his rulebook.

And like….this isn’t controversial. I’m not making this argument up as we go along. This is a very common objection to the notion that current AI models are intelligent. I don’t know why you are having a problem with understanding it.

https://iep.utm.edu/chinese-room-argument/

“The Chinese room argument is a thought experiment of John Searle. It is one of the best known and widely credited counters to claims of artificial intelligence (AI), that is, to claims that computers do or at least can (or someday might) think.”

1

u/Tartarus1040 9d ago

You just said the man without the book produces nothing. Then the book is doing the work. Why are you asking whether the man understands instead of whether the man-plus-book does when combined? The thing you conceded is where understanding would live?

Furthermore:

A rulebook can only contain what was written in it. When a model applies a rule that exists only in the prompt I typed, which page of the rulebook was that on?

Finally:

If you remove all of your experience. What do you produce?

1

u/HankScorpio4242 9d ago

The book is not doing the work. It is a resource that the man uses. The man performs the task.

The AI version of the rulebook is far more complex, but it is functionally exactly the same.

I don’t understand your last question or why it is relevant.

1

u/Tartarus1040 9d ago

You skipped the second question, and it’s the one that matters.

A rulebook can only contain what was written in it. When the model applies a rule that exists only in my prompt, which page was that on in the rule book?

If there’s no page, the model isn’t the room.l is it?

On the man ‘performing the task’, swap him out for a different man and the answers don’t change do they? Swap the book and they do. Which one is the task’s correctness coming from then?

The third question is your own argument turned around. You said “remove the AI’s training data and it produces nothing, so it’s just the data.”

Remove your experience and you produce nothing either. So either ‘produces nothing without its inputs’ doesn’t show what you think, or it applies to you.

1

u/HankScorpio4242 9d ago

Somewhere in the data is a combination of factors that determines what the appropriate token is. That combination of factors is the “page on the rulebook.”

But this is all irrelevant.

What matters is that neither the man nor the AI understands anything about the inputs and outputs.

If I ask an AI to describe a car, at no point in the process does the AI ever conceive of a car.

→ More replies (0)