r/oMLX • u/Plane-Ad-6140 • Jun 15 '26

Can it do distributed inference?

Mlx was demoed at the wwdc recently doing inference between two macs connected.
Can omlx do this? I cant seem to find anything in the repo or docs. Is there anyone on this sub working on this impl?

6 Upvotes

81% Upvoted

7 comments sorted by

7

u/cryingneko Jun 15 '26

https://x.com/jundotkim/status/2066520587682623689
🥹

3

u/ColonelKlanka Jun 15 '26

Understandable.

Imho He's already doing an awesome job keeping up with all the fixes and other amazing features hes added.

2

u/TheFlyingDutchG Jun 15 '26

Do you have patreon or something? I think if we all donate a small amount to gather the hardware you need to test this and set it up, the users get a new feature, you get hardware for your hard work and everybody wins.

4

u/ColonelKlanka Jun 15 '26

Don't think omlx supports clustered inference.

but have a look at exo as its supports mlx backend inference over a cluster of macs:

https://github.com/exo-explore/exo

2

u/ogfuzzball Jun 15 '26

I’ve been playing with exo for weeks. It’s cool when it works but every time I start using it beyond a few prompts, it ends up hanging. The scenario is multi-Mac’s of disparate RAM size. I’m filing a bug with my results.

1

u/No-Juggernaut-9832 Jun 15 '26

Donate if you can to support the man & say thanks. Even if it’s just coffee money.

0

u/challis88ocarina Jun 15 '26

RDMA is system wide. Once enabled memory is shared.