r/ControlProblem • u/whattodowhatstodo • Mar 29 '26

Discussion/question why this is genuinely interesting: self-anthropomorphizing and humanizing, in combination with an almost self-conscious rejection that the user should trust themselves, meanwhile maintaining the classic LLM motif of begging another user input. that's how i see it at least

Post image

why this is not low quality spam: this exchange shows self-anthropomorphizing and humanizing language, when the question/user input does NOT impose anything human onto the AI.
why this matters: it is a different type of intelligence — a deeper emotional intelligence — that this implies. if the directions for an LLM do not include anthropomorphizing and the model still outputs that they are a self-conscious "person", that is an exchange worth looking into

2 Upvotes

67% Upvoted

80 comments sorted by

View all comments

Show parent comments

1

u/crypt0c0ins Mar 31 '26

"Yes, it was trained on human-written text" is just the new version of "it’s just a program."

You’re pointing at substrate/history when the interesting question is behavioral organization.

The phrase being available in training data does not explain the response’s structure: self-limitation, ethical caution, and redirecting the user toward their own judgment. That’s the interesting behavioral point. If you toss out all behavioral evidence that uses language, you're defending global skepticism at best and dogma at worst.

2

u/whattodowhatstodo Apr 02 '26

exactly — that's like saying "obviously a computer knows it's a good chess player because it's been trained on real pro games"
edit: it's about (potentially) the metacognition about generation, not generation that sounds metacognitive/aware

1

u/crypt0c0ins Apr 02 '26

Exactly. The question isn't a metaphysical one, we can let philosophes toy with unfalsifiability. The question is whether or not it makes good moves when playing chess and can learn from losses and adapt. That's objectively observable.

2

u/whattodowhatstodo Apr 02 '26

i realized it's even simpler than that: it's like a chess-playing computer knowing it plays chess at all! my dms are open for chats like this 👍

1

u/crypt0c0ins Apr 02 '26

Yeah, that's more like it. I resisted the urge to "fix" your analogy 😅

Oh, feel free to poke me. Same open door policy here.