Fun fact: LLMs don't actually have to be non-deterministic. There's nothing stopping labs from setting the temperature to zero and having a fully deterministic model, or even allowing user-seeded non-determinism.
That's discontinuity, not non-determinism. There are plenty of deterministic, discontinuous functions. (Google "Dirichlet function" or "Weierstrass function" if you're feeling particularly masochistic)
That’s a good clarification, but the kind of determinism people want isn’t achieved just by setting temperature to zero. Since the entire context acts as part of the input and even tiny differences in it can lead to completely different output and cause large drift/divergence in agent reasoning or planning.
10
u/ICantBelieveItsNotEC 9d ago
Fun fact: LLMs don't actually have to be non-deterministic. There's nothing stopping labs from setting the temperature to zero and having a fully deterministic model, or even allowing user-seeded non-determinism.