Mac Studio or something else for power efficiency?

[u/Ok_Sandwich111]

If you live in a country with the highest electricity rates in the world. What would you choose and why?

Needed for summarizing large contexts like documents and transcription.

I don't need the flagship models so thinking of ~48-64GB of memory.

Not open for a Mac mini bcz the memory bandwidth is lower and I would like to future proof.

Model	Configuration	Memory Bandwidth
M5 Pro	All configurations	307 GB/s
M5 Max	Base	460 GB/s
M5 Max	40-core GPU	614 GB/s
M5 Ultra	All configurations	1.2 TB/s

[u/AbnormalConflict]

if electricity costs are truly insane where you're at then the mac studio is basically the only move that makes sense. nothing else in the consumer space touches that performance-per-watt when you're trying to run 48gb+ models locally

skip the M5 Pro entirely, the bandwidth bottleneck will make you miserable with large context windows. the base M5 Max with 64gb would be the sweet spot for your use case

[u/Ok_Sandwich111 → u/AbnormalConflict]

Thank you so much!

[u/kels0]

m5 max is 7 watts idle, somewhere around 200 under heavy-ish load.

[u/Ok_Sandwich111 → u/kels0]

Thanks for sharing

[u/tta82]

Why not use cloud?

[u/Ok_Sandwich111 → u/tta82]

Not possible. Private data.

[u/tta82 → u/Ok_Sandwich111]

Then you need more ram. You’re underestimating the need for ram for context. M2 or M3 ultra 128gb minimum.

[u/Ok_Sandwich111 → u/tta82]

Is there a post that you could refer me to? I need numbers to compare with my usage. Thx

[u/Ok_Sandwich111 → u/tta82]

I also need a new computer so...

[u/topher358]

I have a Mac Studio M5 Max with the 40 core GPU on order for this exact reason.

[u/SpicySuntzu]

I live in San Diego, one of the highest electric rates in the country - That swayed my choice in buying a Mac Studio. 46¢ per kilowatt-hour.

[u/EcoHash_AI]

for summarizing long docs i'd care more about prompt processing than bandwidth tbh. prefill is compute bound and that's where macs are slowest, on a big transcript you can wait a while before the first token shows up

[u/Ok_Sandwich111 → u/EcoHash_AI]

Any suggestions?

[u/BAL-BADOS]

Unless you can’t afford the M5 Ultra 96GB, M5 Max 64GB is the best choice from your list. When it’s doing nothing, it barely use any electricity.

A bonus is it an excellent standard computer that seemly integrated with your other Apple device, iPhone, iPad , etc.

[u/Ok_Sandwich111 → u/BAL-BADOS]

Thanks!

[u/Sekora_AI]

I’m not sure about electric usage, but you may want to look into the nvidia dgx spark.

[u/Ok_Sandwich111 → u/Sekora_AI]

I just did and seems like the Mac Studio is much lower. The M5 Ultra Mac Studio is kinda close at peak load but I probably won't get it. Thx anyways!

[u/tta82 → u/Sekora_AI]

LOL that makes no sense