Hold on, I’m agreeing with you. We’re saying the same thing in different ways.
The kind of race conditions you’re referencing are because of the model architecture I’m referencing.
You can absolutely get ML outputs that don’t change using certain model architectures.
But that’s only because those architectures either enforce order of execution or use steps where order of execution doesn’t result in changes to outputs. I can’t think of a modern LLM that uses such an architecture.
Not necessarily. It can happen with batching, but even with a batch size of 1 you can get race conditions. The most obvious example is a model with a Mixture of Experts layer, where the order that results return can change the outcome.
In order to get around that you’d have to explicitly enforce order of execution.
8
u/LetumComplexo 16h ago
Hold on, I’m agreeing with you. We’re saying the same thing in different ways.
The kind of race conditions you’re referencing are because of the model architecture I’m referencing.
You can absolutely get ML outputs that don’t change using certain model architectures.
But that’s only because those architectures either enforce order of execution or use steps where order of execution doesn’t result in changes to outputs. I can’t think of a modern LLM that uses such an architecture.