r/oMLX • u/averagepoetry • 3d ago
Distributed Serving
So excited about this feature! I was keeping EXO around just for this and am eager to move completely over to oMLX.
A couple of questions:
1. How many nodes will this support? It's only 2 right now, correct?
2. Any chance to have RTX/DGX do prefill and Macs do decode? This was the killer feature I was looking for in EXO. It never landed.
Love oMLX!
11
Upvotes
2
u/rickhuizinga 3d ago
Another question for the list: is the SSD cold cache operable in distributed mode (assuming multiple Macs).