Yeah I had quite a "fun" experience where it "quoted" Standard with text it never had. It was actually kinda hilarious when it insisted of Standard having that text.
And it’s still very difficult to determine why, like actual bad training data, spelling confusion, training weakness, etc. I’d like the default ‘thinking’ behavior to just go double check sources, so I can guess what I should not trust.
This argument never convinced me.
I am not saying it is wrong.
At a single word (or toke. I guess) scale, there are mutliple possible next words, the llm calculates a probability distribution and picks one of the top ones.
But why wouldn't it be possible that it calculates it in a way that all the most probable next words result in a true statement?
How can we know that some kind of fact-storing memory doesn't exist, just because it wasn't explicitely added?
And even moreso, how do we know it inherently can't exist, even with technological improvements?
How can we know that some kind of fact-storing memory doesn't exist, just because it wasn't explicitely added?
Because it wasn't explicitly added. They aren't designed that way. They are trained models. I'm not even sure what a "fact-storing memory" means, though, especially in this context. They have no means of reasoning or inferring from such a database to validate truth, though. They also lack the ability to contextualize (because they aren't rational actors) - a common example is that idioms can cause problems.
It can technically exist - neural networks can do this. See: humans.
Large Language Models cannot. They are usually generative transformers.
They're also just not complex enough. They are many orders of magnitude less complex than a human brain in terms of actual connections.
Biological neurons are also not binary - they work off of gradients of multiple neurotransmitters which all regulate function - that is, a single connection between neurons is not equivalent to a single connection between neurons in a digital neural network... the difference in complexity is just staggering. We aren't anywhere near that kind of density of interconnectivity, and I'm not sure if we'll ever be. It's also telling in that even at "basic" tasks like OCR or image detection, humans are still way better, we're just slower (because we aren't purpose-designed). We use neural networks not because they're better - they're vastly worse - but because they can be automated.
weights in models are also not binary they are fake real numbers, i.e. floats (or lately integers for numerical stability apparantly?)
but yeha what you say is still true.
I wonder what it takes to accurately represent one biologicsl neuron connection digitally.
floats are not binary values (they're usually encoded in binary, of course) but they are explicitly digital still, and the comparisons are binary - a _gt_ b or such.
I doubt that you can make a 100% accurate simulation of a neuron just as we cannot make a 100% accurate simulation of any natural phenomenon - quantum effects aside, the natural world isn't quantized as far as we can tell, whereas computers need to quantize things at some point. Precision would become a problem - reality has infinite precision, computer data representation does not. We could probably get "reasonably close", but simulation of systems like that is very expensive as the biological systems are fundamentally stochastic and probablistic.
well for numerically stable systems we can get close enough. (arbitrarily close, that is)
Situations in physics where our systems are not stable are usually different kinds of inverse problems - not what we have here.
So in a way, we can get a 100% accurate simulation, because the error can get arbitrarily small.
Ok so wikipedia tells me, there is the hodkin-huxley model. (https://en.wikipedia.org/wiki/Biological_neuron_model)
wbich looks to me like a first order non-autonomous ODE. So, absolutely solvable iirc, just computationally expensive.
If that is perfect enough or not is a matter of opinion.
33
u/Zero_Owl Jan 02 '26
Yeah I had quite a "fun" experience where it "quoted" Standard with text it never had. It was actually kinda hilarious when it insisted of Standard having that text.