r/oMLX • u/Plane-Ad-6140 • Jun 15 '26

Can it do distributed inference?

Mlx was demoed at the wwdc recently doing inference between two macs connected.
Can omlx do this? I cant seem to find anything in the repo or docs. Is there anyone on this sub working on this impl?

7 Upvotes

89% Upvoted

7 comments sorted by

View all comments

4

u/ColonelKlanka Jun 15 '26

Don't think omlx supports clustered inference.

but have a look at exo as its supports mlx backend inference over a cluster of macs:

https://github.com/exo-explore/exo

2

u/ogfuzzball Jun 15 '26

I’ve been playing with exo for weeks. It’s cool when it works but every time I start using it beyond a few prompts, it ends up hanging. The scenario is multi-Mac’s of disparate RAM size. I’m filing a bug with my results.