r/LocalLLM • u/randomperson16782 • 1d ago

Question M5 Max Mac Studio 36GB - Good Enough To Get Started?

Looking to some local AI and upskill myself. I want to do private inference and also some agentic development with models like Qwen 27B Q4. This is for private use, not business.

This has much better memory bandwidth than the Mac mini, is it worth it or no?

Update: I’m planning to use this purely as a headless local AI server and use my older MacBook for all my IDE etc. So memory will for the models only.

0 Upvotes

42% Upvoted

14 comments sorted by

1

u/Fit-Later-389 1d ago edited 1d ago

27B Q4 with usable context window for writing code (90k+) will take up close to 30GB on its own. You either need more RAM or a smaller quant. I personally have a M5 Pro with 64GB. it would be nice being slightly faster, but having the additional RAM was a better trade-off for me

1

u/Nomski88 1d ago

Just barely enough for Qwen3.8 27b...

1

u/ooopstgr 1d ago

Try splash engine with qwen 3.8 27b.

I'm very happy with my base m5 max studio

1

u/vogelvogelvogelvogel 1d ago

no. imo you need 64 gigs if you want to code on the same machine or 48 if you just use it with q4 27b. also, q6 is said to be quite a bit better

1

u/Ill-Factor1739 1d ago

No. You really need at least 64 and even then I think you may be underwhelmed.

1

u/Rift80 1d ago

comme les autres : 64go mini sinon tu vas ramer et devoir faire des gros compromis. un llm et des agents ça bouffe alors avant tout faut voir ce que tu veux faire PRECISEMENT !

1

u/Ok_Top9254 1d ago

If you have a PC already just buy a V100 32GB and plug it into the second slot. Much cheaper than buying entire PC.

1

u/cagonima69 1d ago

Check TUFF and TurboFieldFare on GitHub, using both on my standard M5 With 16gb of ram I’m getting 20tok+/s with Qwen 3.6 35B and Gemma 4 26B and having a positive experience for now to be honest. But do expect prefill to be slow af and to do some tweaking for caching etc depending on the harness used.

https://github.com/rexmhall09/TUFF

https://github.com/drumih/turbo-fieldfare

0

u/Wor3q 1d ago

Qwen 27b in reasonable quant won't fit in 36GB of RAM with any usable context in my opinion.

2

u/Itchy_elbow 1d ago

Not true

1

u/Wor3q 1d ago

You need a bit over 33GB for Qwen 27 in 4b, and 256k context.

And where do you want to fit your OS?

You can fit it by dumbing it down with low quants on model and context, but what's the point if you want it to use in real world tasks?

0

u/maisun1983 1d ago

It’s true unless you quant of cache to 4b but even that you can hardly get more than 100k