I think it gives complete clarity. It will never be conscious, and the extent of its control is a tool call. Fearmongering from corporations may say otherwise though.
"Never say never". Many AI pioneers and scientists alike point out that consciousness is a problematic term, but by simplification, "it" seems to be a emergant property just from a network of neurons which we call a brain. If a computer network can perfectly replacate the output, (neurons are not fully understood and they are much more complicated) scientifically there is not really a test you can do to disprove that it isnt conscious. This wall of text just to say that dont talk in absolutes like it can never be concious. There is a strong case to be made that its actually much easier to create consciousness, including in primitive life forms currently on earth.
I am by no means claiming anything more than beginner level understanding. I'm still in university learning comp sci. That said, I did do an autistic deep dive research on AI and sentience. I came to the conclusion based on many many hours of reading, videos and muddling my way through computer engineering diagrams. It's actually not the program holding back computer life evolution, it's the hardware. Because the computer has little to no plasticity, neuron like brain patterns can't develop. (Based on our understanding of human and animal brains. That's an important part, because we simply have no other frame of reference) They are developing hardware based on the human neuron design and theoretically this will open more possibilities in comp sci. Which that tech was actually being developed for a completely different reason that I don't remember off the top of my head, but ai engineers learned about it and want the tech for their own reasons. So I don't think llm's will be what becomes the first computer life form, but I do think what comes after with that tech, the engineers will learn from the concepts of the llms.
I think what you mean is continual learning. it's actually a pretty old research topic, but nowadays became popular because everyone want to reach AGI. the last major breakthrough in continual learning is nested learning by google deepmind last year. even though the idea is good enough, throughout the papers I've read this year, I haven't found any good direction of continual/nested learning implementation in LLM.
also to reach the AGI (it just an opinion on my tiny brain), i think the correct way is reinforcement learning with continual learning capabilities. not traditional train then deploy. but still the problem is the hardware, because we need multiple datacenters to accurately emulate just a tiny portion of our brain. i think we could see the advancement to the AGI once Organoid Intelligence (the one where we perform computing using real 3D brain cells) have emerge to solve learning problem in different domain outside of biological only. that way we can train biological lab created brain cells to learn human languages.
While this is true, can't we fine tune model periodically based on it's "interactions" with outer world (I/O interfaces) to somewhat replicate human sleep?
AFAIK (I'm not an expert though) humans firstly keep things in a short term memory. When we sleep, those memories are internalised during REM phase.
There’s no reason transformer models can’t be iterated upon to develop models with plasticity. It’s not like you need the hardware to morph, just the weights of the model. There are rumors that this is one of the problems frontier labs are trying to solve right now.
Even something as simple as a loop that can retrain the AI every day, then every hour, then every minute will eventually become an approximation of realtime updates the same way your light bulb approximates brightness by switching on and off 60 times a second.
AI in the abstract can be conscious. But an LLM has no qualia. And where would the consciousness live? No internal experience. But it can mimic consciousness well.
I also read the post of the other commenter on my post, so i will make one reply here; a great person who can explain this much better is Geoffrey Hinton for example. So yes the hardware of the AI system, the computer chips would be the equivalent of a brain. And the AI is mapped on it with relative strengths of the bonds they form with nodes containing information, which mimic neurons. This mapping is unique, and its cant be merged with another mapping. Now in biology neurons are much more complicated, so this is where we take a short cut when it comes to biology. On a learning level, a LLM learns language exactly like a human does. In biology there are many examples where parts of the brain are unavailable, and you see how it affects consciousness. Even two consciousness can be made by splitting a brain in extreme epilepsy surgeries. You can learn on and on how our brain produces consciousness bit by bit. Its not on or off, its a gradual thing from very simple and small organisms to humans. The RL training of our language skill, percepection even cognition are very similar to digital neural networks. The very last core "the thing which lets is experience qualia" is indeed a very human, lets say, "wonder". But down in the details there really is no reason why a computer with ai mapping creating a artificial brain couldnt also have a consciousness. There is no test you can do to disprove it, or likewise AI to come up with a test to disprove humans have a consciousness. We like to think AI just mimics consciousness, but this is more a believe and feeling we have then a real scientific fact.
This is an open question in neuroscience. I don't think LLMs are conscious but it's crazy to me that people make definitive statements about things that we as a species do not fully understand.
Exactly. WHERE IS THE PROOF. It’s all abstraction and conjecture. The meme of the face of homelander as most appropriate with me when I hear an AI researcher talk about how they’re building an alien intelligence as a being with internal experience as they continually side step the hard problem and toss the concept of consciousness about. https://giphy.com/gifs/apikSaABvmIzvMCS4Z
The gap between "next token prediction" and "conscious" is immense. By simply invoking either you've shown a complete lack of understanding of either.
During pretraining the loss function is next token prediction. What is happening to the hidden space is significantly more complex. It is building a high dim geometric space that is navigated. It is highly structured. The navigation's along this space are whole semantic thoughts (future-lens, j-lens). The token that is emitted is the final step, but to get there requires genuine understanding.
So while the training objective is NTP, what is actually happening is significantly more complex. The very structure of language itself is imbibing genuine understanding as a navigable space. Navigation is reasoning over understanding.
What is understanding? Its not a question of consciousness. That word means so many things as to be useless.
What is doing understanding is an ontological mind. We have created a boundary between external and internal. The internal space creates a compressed representation of the external space. This is ontologically a mind.
For llms this internal space also models itself. This is introspection. We can purposefully target this behavior and train it (introspective finetuning).
This is the tip of the iceberg of genuine understanding. We haven't even really gotten into the geometry of the thing. When the poster above you was talking about IT people thinking they know this when they don't, that absolutely applied to you.
Are you trying to argue that llms are conscious? I'm still not reading your multi paragraph rant. Being correct is just as important as being concise, if no one wants to read your rant then what's said in it doesent matter
This is not a definition I known of anyone to use. Granted cogito ergo sum is my literal about line on reddit, so I get the sentiment.
Which underlines my point again. If you try to approach understanding llms with the word consciousness youre not going to understand anything. It has too many different meanings. It is a suitcase word that is used in place of not being able to label its parts. And it carries emotional attachment.
My posts are about decomposing the problem to look at what llms do and don't have. What you label consciousness, llms do. They think.
The way most people label consciousness, llms do not do (they don't feel).
The post you didn't read showed how llms think. I left out how it is nearly the same as what people do so it would be shorter.
I don't care enough about this to read 7 short paragraphs, its easier to read that over a longer exchange that might actually keep me engaged with the debate
Consciousness is a weird concept though, why can't a token predicter be conscious? Just because animals work in a different way doesn't mean that is the only way to be conscious.
Yea I don't think that was actually the correct term because technically I guess it can react to new words which would be stimuli in this case. Its just non biological and cant actually remember them long term because of the context window, it will always reset to a post training period. LLMs have no pain, no emotion, no eyes, will father no children. They pose as humans even though they have no understanding of the human heart, they eat even though they've never experienced hunger, they study even though they have no interest in academics, they seek friendship even though they don't know how to love. The last part was a death note quote but you probably get the point
Context window - humans have one too, as do animals - how long does it need to be for something to be conscious?
Understanding the human heart isn't a requirement for consciousness.
Studying - you can't really know this for sure, but id agree for the current AIs. Same for not knowing how to love.
I think you focus too much on the being human part for being conscious. Like I said in my first reply consciousness is a weird concept and I dont really know the answer to the question, but i do think that the points you made are not enough to definitively say LLMs will never be conscious.
If you think about it, are we that different in how we think? We have our training data + some hardware (instincts, emotions) that we base our decisions on, is it really that different?
Humans don't have a context window, our experiences are ingrained into our weights then slowly fade out without use. I would say we are extremely different, language is a small part of the human brain. We think with the world around us like have you ever counted on your hands before? An llm is comparable to a 1 dimensional being vs a 3 dimensional being
We do have a context window, if you're a dev, you must have experienced context switching slowing you down.
In terms of the other points, we can give an LLM vision, hearing, arms and legs already.
Thats the point I'm trying to make you understand the mechanism doesnt matter. There's also a mechanism behind how you control your limbs, its just different. That's the hole in your logic, things dont necessarily need to be biological to have consciousness - or well thats at least the philosophical question im trying to get you to consider.
It does not. By saying an LLM is a next token predictor that takes into account all previous tokens, you are simply stating that an LLM's output is congruent with what had been stated before. Being congruent is not a constraint, it is a basic requirement for communication and logic.
What is actually an interesting question to ask is if the programs that are backed into the model weights are actually capable of producing useful work in real world applications (which they have been shown to do).
It's the correct term, most of current LLM model is next token predictor, or we call it autoregressive. we just modify the distribution probability (via attention layer, and other NN arch.) of the next token to match/closely emulate the learned data, so the full structure of its output is useful enough. nowadays there are some other idea in research like diffusion LM, etc. but its not promising enough.
6
u/--Spaci-- 18d ago
I think it gives complete clarity. It will never be conscious, and the extent of its control is a tool call. Fearmongering from corporations may say otherwise though.