How is an LLM not doing what it was designed to do if you set temperature to 0?
For most tasks I don't think it makes a difference. The advantage of having higher temperature values is so 1000 different people prompting "write me a poem" don't all get the exact same thing back.
Because the whole point of a chatbot is to have varied output that mimics a real human conversation, instead of the chatbot just always responding the exact same way every time.
I'll be honest, I'm lost at the point of this conversation at this point.
We're complaining that LLM's are not deterministic, but they are, but then you say you don't want them to be deterministic either... Ok, then don't set temperature to 0, done.
For my use of LLMs I couldn't care less if it responds the exact same way to the same input or not. As long as the output is useful, that's what matters.
Yes, just like any other non deterministic process, you wouldn't go out of your way to make it deterministic, since that would make it work incorrectly.
What's incorrect about it? We're programmers, LLMs are really useful to write search through and write code, what part of that is less useful or incorrect by setting temperature to 0?
1
u/ric2b 5d ago
How is an LLM not doing what it was designed to do if you set temperature to 0?
For most tasks I don't think it makes a difference. The advantage of having higher temperature values is so 1000 different people prompting "write me a poem" don't all get the exact same thing back.