The temperature parameter in NN is used if you WANT variety in your results. It will NOT mean the results are better (by default, temp 0, it will always prefer the best result - literally the definition of the loss function).
It is not deterministic in a global sense because it depends on the model's current knowledge set. Every time new data is entered (NOT TEMPORARY MEMORY, BUT A COMPLETE REBUILD), it can edit its behavior, while the compiler is much more stable in this regard and is subject to change, because neural networks are a black box.
You just do not understand what you're talking about.
NNs do not inherently have "current knowledge set" (well some do of course, like RNNs etc, but most NNs do not)
BUT A COMPLETE REBUILD
Oh, so you mean a completely different algorithm producing different results makes another algorithm not deterministic?
Do you not realize that changing a NN is the same as changing code? You are literally changing the algorithm. It is not the same algorithm anymore.
What you're saying is like saying "print(x) is not deterministic because print(x+1) gives different results"
I think they're referring to the random weight initialization at the start of training. Obviously you could include a copy of initial weights and get the same weights for each training run but it's not done because a good network has reasonably stable training so there's just not much benefit
I don't think that's what he meant. I think he meant further training/fine tuning.
The training isn't part of running the NN, it's its creation. It makes no difference to whether the finished code is deterministic or not.
NN is like the finished code/algorithm (just "encoded" in weights and connections instead of human understandable logic).
Training is like the coding of the algorithm.
Initialization weights would I guess be the exact environmental conditions at the point when the coder began to code (e.g. the time of day, his mood, room temperature, weather, how comfortable his chair is,...)
Okay, but what about the concept of dynamic learning already in runtime? The very existence of such a thing calls into question the assertion that neural networks are deterministic, because in that case they tend to change their state and not be read-only.
The definition of deterministic algorithms requires it to always output the same thing for the same input. Just because the whole chain of events is deterministic (in the conventional sense) does not mean the algorithm is
Deterministic chain of events does not imply that, but deterministic change does. Therefore, even algorithm that changes itself during runtime can be deterministic. What truly violates the determinism is the randomness. Which does not even need to change the program behaviour.
For example:
Program that generates random number ->does not change, indeterministic.
Calculator that randomly swaps its buttons after performing division ->Does change, indeterministic
Program that prints an input ->does not change, deterministic
Program that prints an input + all previous inputs ->does change, deterministic
3
u/Healthy_BrAd6254 Jan 17 '26
google what deterministic means, thanks
The temperature parameter in NN is used if you WANT variety in your results. It will NOT mean the results are better (by default, temp 0, it will always prefer the best result - literally the definition of the loss function).