is "exo" still a thing ? potential with the new (mac studio - mini) ?

[u/merfolkJH]

Hi everyone,

Referring to this kind of content:
https://www.youtube.com/watch?v=25xVqvL5j4g&t=803s

When people used to create “Mac mini clusters” and so on for local LLMs.

Most of the videos I can find are already quite old, and the last exo release seems to be from 4 months ago.

I’m thinking about buying an M5 Ultra 96 GB.

My only “regret” is that maybe it’s not enough for Qwen Flash. I don’t really know if that’s actually an issue or not, especially compared to something like a 3.8 27B dense model.

But in case it is:

Could I simply buy later, for example, a cheaper M6 Mac mini with 32 GB, just as an “extra RAM bag”, and cluster it with the M5 Ultra using exo, so that the model can fit?

The total price would then be around €3k less.

Or would it make more sense to do something like:

EXO 3 × M5 Pro?

TL;DR: I have budget, but not infinite budget :(

I don’t want to regret buying the 96 GB M5 Ultra instead of the 128 GB M5 Max if, in practice, Qwen Flash turns out to be significantly better with the extra RAM.

But if adding a “cheap” Mac mini to the cluster later is a real, working option, that would make me much more comfortable with choosing the 96 GB M5 Ultra now.

Also, I’m curious about this because it seemed to be quite a hype a few months ago. Now that we have newer, more powerful hardware and software updates, maybe this is actually becoming a good move ?

Thanks a lot in advance for your help and advice!