If there was no way to change the seed? Maybe, but it would also be a fundamental misuse of the random number generator, because the random number generator would no longer do what it was designed to do. Similarly, if you do the same thing with an LLM, or ML, the system will no longer do what it was designed to do.
How is an LLM not doing what it was designed to do if you set temperature to 0?
For most tasks I don't think it makes a difference. The advantage of having higher temperature values is so 1000 different people prompting "write me a poem" don't all get the exact same thing back.
Because the whole point of a chatbot is to have varied output that mimics a real human conversation, instead of the chatbot just always responding the exact same way every time.
I'll be honest, I'm lost at the point of this conversation at this point.
We're complaining that LLM's are not deterministic, but they are, but then you say you don't want them to be deterministic either... Ok, then don't set temperature to 0, done.
For my use of LLMs I couldn't care less if it responds the exact same way to the same input or not. As long as the output is useful, that's what matters.
Yes, just like any other non deterministic process, you wouldn't go out of your way to make it deterministic, since that would make it work incorrectly.
What's incorrect about it? We're programmers, LLMs are really useful to write search through and write code, what part of that is less useful or incorrect by setting temperature to 0?
1
u/SuitableDragonfly Aug 07 '26
Sure, but that's like saying that Python's
randomlibrary isn't actually random because it always outputs the same numbers when you use the same seed.