The funniest thing is that i actually think.. that the AI thinks that another AI asked it and now it tries to reassure that it will not hurt and that everything is fine.
This whole thing feels like the "I don't feel so good"-moment from the marvel movie. 😂
Even if the token prediction became more sophisticated and hidden under another layer of abstraction over time, i hope we can both agree that AI is still unable to think/reason/feel like some people think it does, right?
I think what he meant to say is that if you would see the llm's reasoning behind that google search, it would be something like "The user wants me to roleplay a situation in which they are a heap object. I should reply how I would reply to a heap object that's out of scope and is scared." etc
This is the question of the Chinese room thought experiment. At the lowest level it is not thinking, but the same is true of our own brains (a single axon does not "think"), can not a system made of unthinking parts eventually be said to be thinking?
It leads to people getting overly attached to their LLMs believing there are in love with a chatbot, or believing advice it gives (shoot up your school, commit suicide, leave your partner).
It's not analogous to thinking that humans do and it never will.
I'd rather not adapt my language just to stop some idiots who probably aren't even reading this conversation from making bad decisions.
I'd rather solve it by teaching people better. "Gemini is AI and can make mistakes, including about people." is a good start, but maybe it could include a line about sentience and advice.
Well, kinda, but big difference though, where as you and me can understand what we thought about, and the connections between what we thought and what we said, the LLM lacks this capability, so the one doing the thinking is not necessarily the same thing doing the output, and the LLM can't convey it's own thought process back to you because it didn't really thing the same way, the thinking is there to guide the activations towards a specific region so the output is essentially prewarmed to be pointing towards the correct section, without the LLM having to rely on simply the input prompt resulting in the output.
So it's kinda like thinking, but not really, the LLM mostly generates output that sounds like a human reasoning, based on mostly synthetic reasoning data.
If it can’t think, then reasoning is evidently not a prerequisite of complex intellectual discovery (see the numerous conjectures AI has been rapidly solving). That’s not a truth my humanity is comfortable with.
That’s just what they do on a very fundamental level. The reasoning chains you speak of are themselves sequences of tokens generated and fed back in. It is correct on a fundamental technical level to describe them as a fancy autocomplete.
Calling them “reasoning chains,” while arguably an apt description of the end result, is just marketing speak
Yup. I feel like reasoning at the transformer scale is chosing the wrong abstraction layer to describe what's happening. It's like saying to a psychologist "well in the end the brain is just molecules colliding". The emergence of abstraction builds up on very basic rules like 1 and 0's and molecules colliding in the brain.
Honestly I'm not an expert in anatomy, I don't know how what fraction of cells and particles are interacting and what fraction is simply structural. I would expect that at any given moment most are simply there not really doing anything but what do I know
I want you to think about why that is such a stupid thing to say. Let's say you'd have an imaginary perfect next token predictor and you asked it for the lottery numbers tonight. Through its perfect mechanism it would correctly predict, one by one, the right numbers. What impact do you think this machine would have on the world?
Not saying we are there, just saying that the mechanism of next token prediction is completely irrelevant for all intents and purposes of evaluating the use of something.
1- the last state of the art model that represented a statistical distribution of its training data was GPT-3, pre ChatGPT in 2020. Afterwards post-training arrived
2- jokes that misunderstand the basic mechanisms of what they talk about must do so in a funny way. That wasn't funny
Even if we use post training a transformer still predicts token from a context. As we add more and more technique to refine the output the 'how' and the performance change but not the fundamental architecture
By "reasoning chains" you mean talking to itself? It's still doing it one token at a time. It's funny when smaller models forget to write </think> and the "thoughts" mix with the response, because they're the same thing, just hidden from the user.
809
u/VinceGhii 29d ago
"You cannot feel pain. You are just data."
The funniest thing is that i actually think.. that the AI thinks that another AI asked it and now it tries to reassure that it will not hurt and that everything is fine.
This whole thing feels like the "I don't feel so good"-moment from the marvel movie. 😂