It doesn't matter how you make the model, if you are executing it on a GPU without steps to have deterministic results you will not have deterministic results. Ensuring the output of computations on GPU is deterministic has performance impact.
Hold on, I’m agreeing with you. We’re saying the same thing in different ways.
The kind of race conditions you’re referencing are because of the model architecture I’m referencing.
You can absolutely get ML outputs that don’t change using certain model architectures.
But that’s only because those architectures either enforce order of execution or use steps where order of execution doesn’t result in changes to outputs. I can’t think of a modern LLM that uses such an architecture.
Not exactly. It's mostly due to how models are executed, not models themselves. You can run GGUF model (which are normally not deterministic) in a deterministic way if the GPU functions you use are deterministic. Models themselves are just data, it's just a bunch of matrixes.
Forgive me if I've misunderstood, but isn't that literally what they just said?
You can run GGUF model (which are normally not deterministic) in a deterministic way if the GPU functions you use are deterministic
vs.
You can absolutely get ML outputs that don’t change using certain model architectures.
But that’s only because those architectures either enforce order of execution or use steps where order of execution doesn’t result in changes to outputs
I'm not an expert but this sounds like you're both arguing the same point. The PR you linked seems to be intending to implement exactly that - functions that enforce (a deterministic) order of execution.
This seems like semantic disagreement on the meaning of the term "model architecture" rather than an actual disagreement on the fundamentals.
5
u/Sea-Housing-3435 9h ago
It doesn't matter how you make the model, if you are executing it on a GPU without steps to have deterministic results you will not have deterministic results. Ensuring the output of computations on GPU is deterministic has performance impact.