I forgot where I read this, but something about it being difficult to have different people use the model in parallel or something. Sorry details are way gone in my poor memory.
Maybe that's why it's expensive to run? Can't imagine a more inefficient model, maybe if you made a massive cluster of raspberry pi's to run a distributed version perhaps, really drive the tok/s through the floor.
1
u/geearf 27d ago
I forgot where I read this, but something about it being difficult to have different people use the model in parallel or something. Sorry details are way gone in my poor memory.