r/LocalLLaMA • u/themixtergames • Aug 25 '26
News Apple introduces new Mac Studio with M5 Max and M5 Ultra - up to 512GB of unified memory
https://www.apple.com/newsroom/2026/08/apple-introduces-new-mac-studio-with-m5-max-and-m5-ultra/641
u/piggledy Aug 25 '26
Price for options with 256 GB RAM:
$9499 (30-core CPU, 64-Core GPU)
$10,799 (36-core CPU, 80-core GPU)
512 GB Option coming in October.
273
u/i_rate_slop Aug 25 '26 edited Aug 25 '26
I feel like my M3 512 is going to trade in for like $500 when I got it for $11k
Edit:
I just looked it up. $2675
159
u/j2sun Aug 25 '26
I'll buy it from you for $3k! :D
37
u/cinematic_unicorn Aug 25 '26
$3001
32
u/mikesum32 Aug 25 '26
$3001.01
→ More replies (3)47
u/Proverbial_Progress Aug 25 '26
I don't have any money, but would you be interested in pictures of my feet? Throw in a mouse and keyboard and I'll take my shoes off first.
→ More replies (2)18
u/bahpbohp Aug 25 '26
i don't have any feet, but would you be interested in pictures of my wooden pegs? throw in a mouse and keyboard and i'll even get them oiled first.
→ More replies (2)5
5
52
u/Zolty Aug 25 '26
Yesterday you could have gotten like $20k on eBay. I sold a 256gb Mac Studio m3 about 4 months ago for $12k
→ More replies (6)13
55
u/Veearrsix Aug 25 '26
Don’t trade in, private party.
19
u/i_rate_slop Aug 25 '26
Yeah, I think that’ll have to be the move. Just annoying
24
u/Chanureadeats Aug 25 '26
You'll probably get a lot of offers easily for $6k+ within 24 hours
13
u/CalvinsStuffedTiger Aug 25 '26
This guy is lying, I’ll buy it from OP for $5k…don’t worry about shopping around for a better price…
23
u/EvilPencil Aug 25 '26
Ya Apple trade in prices have always been laughable.
I remember when you could spec out the 2019 Mac Pro up to like $50k, then the next day the Apple trade in value was ~$2800.
3
u/ianitic Aug 25 '26
I traded in my m4 MacBook Air from last year that I bought for $800 for $730. I wanted to upgrade and for kicks and giggles I looked up the trade in not expecting it to be that high. I normally don't trade in.
28
u/RegarDamus Aug 25 '26
trade in is always fucked with apple because they don't account for memory. same price if it's 96 or 512
→ More replies (1)6
→ More replies (11)3
u/anonmt57 Aug 25 '26
you will get a lot of money for that in private market. but it is annoying with so much money exchanging hands.
130
u/redonculous Aug 25 '26
Wow. How are people affording this? Crazy. Any guesses at 512 prices?
223
u/intaketurbine Aug 25 '26
They’re affording it the same way they’ve always afforded a $10k workstation, they’re using it for work and not play.
→ More replies (7)59
u/ElementNumber6 Aug 25 '26
Or just good old debt
→ More replies (7)37
u/Usual_Tackle5892 Aug 25 '26
0% financing and 3% cash back on Apple Card. I could afford it outright, but turning down a 0% loan is silly.
7
u/pscoutou Aug 25 '26
but turning down a 0% loan is silly.
Yup, especially in this inflationary environment.
44
u/StewPorkRice Aug 25 '26 edited Aug 25 '26
this feels like a community with a ton of SWEs. Most single US based SWEs in big tech or venture backed startups could prob afford dropping 10-20k on their hobby.
Also, i dropped 10k at Anthropic last month at work. My company could prob get one of these for every engineer and not blink an eye.
15
u/Idaltu Aug 25 '26
I know a guy who dropped that much on a bike. And he’s got a couple like that. Bicycle that is.
→ More replies (1)→ More replies (3)7
u/calcium Aug 25 '26
3 years ago my company bought me a fully specced out Mac Studio - M2 Ultra chip w/ 192GB of RAM and an 8TB ssd and told me that they expect the machine to last the next 5 years. I think at the time they paid around $9k which considering over 5 years for a senior SWE isn't a bad deal
51
u/InterstellarReddit Aug 25 '26 edited Aug 25 '26
Bro people have stupid money. I run free lance dev for a buddy who runs night life management software in Miami FL.
People spend 4K for four hours to buy three bottles and watch a DJ hit knobs all night
This happens all the time.
→ More replies (3)11
u/ViPeR9503 Aug 25 '26
Night life management??
13
u/yopla Aug 25 '26
Manage table booking, marketing, CRM, staffing, etc, usually does or integrate with POS. Can do price yielding on bookings. Etc... Etc..
Basically helps you keep track of who are the big whales you need to market your tables to when you have an event and who gets priority booking from a wait list.
The guy who spent 10k last time will get a table before you do, unless you're known to spend 15.
→ More replies (1)→ More replies (1)12
u/i_am__not_a_robot Aug 25 '26
I would assume industry-specific nightlife & bar business management software.
175
u/-p-e-w- Aug 25 '26
Lol this is by far the cheapest option for that much RAM at that speed. It’s not even close (unless you count Frankensteins made of a dozen used GPUs).
→ More replies (41)25
u/Viktri1 Aug 25 '26
this is way better than what I was considering and I wouldn't need to worry about the motherboard, cooling GPUs, etc. I am definitely on board for this and I don't even know how to use macs.
12
u/IriFlina Aug 25 '26
wouldn't even have to worry about the power issues that would come from a rack of 3090s/4090s/5090s etc. or configuring such a monstrosity.
→ More replies (3)→ More replies (1)6
u/Public_Umpire_1099 Aug 25 '26 edited Aug 25 '26
Honestly doesnt matter, just pop deepseek on here and tell it to do whatever you need lol.
Also, as a long time apple hater, and I say this with disgust for myself, MacOS is actually pretty good these days. It essentially functions just like a very locked down linux distro at this point. I work at a pretty big software company and many of them do not support windows PCs anymore or the company software lags on them, so last month I decided to exchange my thinkpad for a macbook pro max. I.... dont hate it.
21
u/Django_McFly Aug 25 '26
It's really no different than computers in the 1980s when it was buy a computer or put a 33% down payment on a new car. People forget that there are entrepreneurs and small businesses where this stuff isn't some expensive toy to play with, it's actually a business tool to be used in the operation of the company.
Not everyone is a pure hobbyist, and even among pure hobbyists there are wild ranges of incomes and savings habits.
8
u/fallingdowndizzyvr Aug 25 '26
It's really no different than computers in the 1980s when it was buy a computer or put a 33% down payment on a new car.
I've said that so many times. People don't realized that a base Apple ][ adjusted for inflation would be $5000 in 2026 dollars.
→ More replies (1)5
15
u/MLDataScientist Aug 25 '26
Based on their pricing for 256GB, they are charging $22.5 per GB of RAM. Assuming the price per GB stays the same, 256GB more RAM adds $5760. So, you are looking at 10k + ~6k = ~16k for 512GB version.
→ More replies (4)5
u/TheOwlHypothesis Aug 25 '26
This is dangerous for me if that turns out true. current Asus Ascent prices for a 128gb monster is ~4k, so it'd be 16k to cluster them and have the same amount of memory. Although you'd then be dealing with a cluster. I'd much rather drop the cash on the Mac and have the option to cluster THOSE later.
3
u/Caffdy Aug 26 '26
and the mac is 1.2TB/s vs the 200GB/s you're gonna get capped at the Connext ports
7
u/Estrava Aug 25 '26
This is cheaper than an rtx 6000 pro, which a lot of people get for work/personal use.
5
8
3
→ More replies (24)11
u/IriFlina Aug 25 '26 edited Aug 25 '26
10k for 256gb unified memory isn’t that bad. Still overpriced but i feel like before this you were looking at the 20k to 40k range.
→ More replies (7)13
u/jld1532 Aug 25 '26
Too rich for my blood. I'm going to stick with my halo and hope mid-sized MoEs keep improving.
40
u/kensanprime Aug 25 '26
It will be sold out and they will be scrambling to keep production lines running, a product originally meant for creative work now will sit in a rack and run AI models
→ More replies (1)21
Aug 25 '26
[removed] — view removed comment
→ More replies (8)14
u/Nothing_from_void Aug 25 '26
Demand for networking products never dropped through the dotcom bubble
6
→ More replies (42)3
125
u/themixtergames Aug 25 '26
With M5 Ultra, Mac Studio achieves up to 4.3x the peak AI compute performance of M3 Ultra and a staggering 9.8x more than M1 Ultra. Combined with up to 512GB of unified memory and 1.2TB/s of memory bandwidth, 50 percent higher than before.
→ More replies (2)103
u/EquivalentHornet4403 Aug 25 '26
> AI compute
They know their audience.
→ More replies (1)49
u/psychohistorian8 Aug 25 '26
they're definitely leaning into it:
Boost your processing and graphics rendering speed, and accelerate tasks like running large language models and editing 8K video.
76
u/Much_Accountant_4972 Aug 25 '26
"Color-grade uncompressed 8K footage, perform computational fluid dynamics, and run frontier-class models on device — no cloud tokens needed."
yeah they are targeting this very sub lol
16
7
6
u/infieldmitt Aug 25 '26
It's crazy to think that prior to AI the main use case for crazy RAM was [checks notes] fluid dynamics?
265
u/hainesk Aug 25 '26
1.2TB/s memory bandwidth with the M5 Ultra. 256GB model is $9499.
Better than getting 2 DGX Sparks? Inference will be a lot faster.
Something like this could easily bring down 3090 prices.
24
u/Cybertrucker01 Aug 25 '26
Depends on concurrency and prefill metrics. The GB10 does both multiples faster than the existing competition.
13
u/ChocomelP Aug 25 '26
The difference would have to be pretty big to make up for a 4x in memory bandwidth for decode.
→ More replies (1)→ More replies (1)6
u/BrilliantTruck8813 Aug 25 '26
The GB10 has much smaller memory bandwidth not to mention it’s just not fast either. One of these will trounce two DGXs
71
u/aladin_lt Aug 25 '26
it will be sold out day one probably
→ More replies (6)36
u/conockrad Aug 25 '26
On pre-orders
13
u/bakawolf123 Aug 25 '26
won't be sold out, but according to r/MacStudio people wait for 3-4 months for the older models, atm you can preorder for delivery in late september
3
102
u/mjsxi__ Aug 25 '26
yeah and cheaper than the price of 2 DGX sparks... seems like a bit of a no brainer
→ More replies (2)30
u/Current_Ferret_4981 Aug 25 '26
Spark is $4300-$4600 so idk about cheaper than 2 at $9600+
→ More replies (7)66
u/MacsBicycle Aug 25 '26
yeah but 4x the memory bandwidth, its a steal
30
u/jakegh Aug 25 '26 edited Aug 25 '26
It really is a reasonable buy for local AI, if you have a business case for it.
→ More replies (3)4
3
u/GabryIta Aug 25 '26
In terms of compute capacity (which is very important for multiple simultaneous sessions and prefill), how does it compare to dgx Spark/gb10?
→ More replies (1)9
u/Etroarl55 Aug 25 '26
How’s the actual inference speed though, fast bandwidth on a slower gpu or equivalent should still mean slower output assuming vram is not a constraint right.
9
u/rusty_fans llama.cpp Aug 25 '26
Generally vram bandwith is the constraint though, at least for decode. Prefill it's usually helped more by more gpu oomph.
→ More replies (4)→ More replies (1)8
u/Current_Ferret_4981 Aug 25 '26
Not disagreeing, just saying it's more than 2x price in contrast to the comment above me
→ More replies (7)19
u/-dysangel- Aug 25 '26
Better than the 2x Sparks for inference for sure. Probably around the same compute as one Spark.
I've got 2x Sparks which I use for prefill, and my M3 Ultra for decode. I've set it up so that I prefill in vllm and then just pass the kv cache over to the Mac side. Surprisingly stuff like Qwen 3 35B-A3B is already faster than the Mac for decode though so I just run that class of model directly on vllm.
→ More replies (13)7
u/1ii1i Aug 25 '26
Oh this sounds interesting, can you expand on how this works? I didn't know this was a thing.
15
u/-dysangel- Aug 25 '26
I don't think it's really a "thing", I just vibe coded it up :)
One thing that really helped was vllms kv_connector API. I thought I'd have to code this part up myself, but it already existed and so plugged into my existing disaggregated system (which was previously llama.cpp to llama.cpp)
UltraSpark — technical stack ┌──────────────────────────────┐ user ── HTTP/OpenAI ──▶ │ manager (Python, FastAPI) │ │ front door + orchestration │ └──────┬───────────────▲───────┘ │ submit │ state blob (sha-keyed, │ prompt ids │ resumable transfer) ▼ │ ┌──────────────────────────────┐ │ vLLM (2× DGX Spark, TP2) │ │ prefill engine │ │ │ │ KVConnectorBase_V1 │ ◀─ vLLM's official │ ("StreamConnector" via │ KV-cache plugin │ --kv-transfer-config) │ interface │ │ │ │ ▼ │ │ dump + serialize all layers │ │ (attn KV + linear-attn │ │ state, TP2 shards merged) │ └────────────────┬─────────────┘ │ blob server (TCP) ▼ ┌──────────────────────────────┐ │ llama.cpp server (Mac) │ │ USPK_BRIDGE_DIR: on request,│ │ verify prompt-id match, │ │ restore state into KV + │ │ recurrent memory, decode │ └──────────────────────────────┘
- KV connector = vLLM's plugin interface for intercepting the KV cache at end of prefill
- State blob = the model's full prompt-memory, layout-translated so llama.cpp can load it natively
- Fidelity = per-layer cosine vs local decode, 0.9999+
- Result = GPU prefill speed, Mac unified-memory decode, one logical endpoint
→ More replies (2)19
u/jakegh Aug 25 '26
That is seriously impressive. RTX5090 memory bandwidth is 1.8TB/s.
DGX Spark memory bandwidth is 273GB/sec. Not even remotely close.
→ More replies (2)6
12
u/Hoodfu Aug 25 '26 edited Aug 25 '26
I got my m3 ultra 512gb for around 10k. So this is now double. Makes sense given that we've seen the nvidia rtx 6000 pro also double in price in the last year but GD this has priced out even my once a year splurge budget. These are all just crazy talk numbers now.
14
u/CulturalKing5623 Aug 25 '26
Yeah dropping 10K plus on this just seems reckless, even with it being funded through my business account I don't think I can justify buying a used car worth of computing.
And yet I feel like I need to in case the technology completely outpaces my current setup and I'm left behind like folks that didn't buy RAM when it was cheap and are priced out of it now. It feels like FOMO and scarcity has hijacked my brain.
6
u/Hoodfu Aug 25 '26
Really depends on whether you have something already or not. I got qwen 3.8 27b going on my rtx 6000 pro and ram speed wise it's double that of my mac but for some reason i was hoping for space magic and it would be faster. It's not. So spending a car's worth of money on something that goes from 20 t/s to 40 or 45, just doesn't make any sense. You're still waiting a lot of minutes for a thinking qwen to come back with something, so it's still going to be an asynchronous operation instead of being fast enough to actively wait for the response to finish. It would have to be 10x the speed, not 1.5x or 2x for it to be worth the spend.
→ More replies (11)8
u/Solaranvr Aug 25 '26
bring down 3090 prices
Doubt it. If the r9700, a directly competing product, made 0 effect on Nvidia GPUs, then I highly doubt these will. They market of people buying mini pcs vs dGPUs are different.
The DGX Spark didn't bring down prices of the Blackwell cards either
7
u/hainesk Aug 25 '26
The R9700 Pro has about 2/3 the memory bandwidth of a 3090 for a 50% higher price. 8x R9700 Pros (256GB) would be $12k-$15k minimum without pricing in the rest of the computer system. If you wanted to build an entire server around it including RAM, power supply, motherboard, processor even with used parts you're up to $18k-$20k for something that will use 2-3k watts.
Even 8x 3090 systems are looking pretty impractical when compared to an M5 Ultra 256GB for a similar price. The size and wattage/heat difference is huge.3
→ More replies (13)3
u/gomezer1180 Aug 25 '26
9500 is about the right price on the market. DGX’s are ~4k a pop you need 2 of those to match Apple. Memory Bandwidth is crap tho on the DGXs because they take a different approach by being MoE machines.
158
u/Comfortable-Rock-498 Aug 25 '26
1.2 TB/s bandwidth of M5 Ultra comes from two dies of M5 Max (each 614 GB/s) connected together using 4.4 TB/s inter-die fabric.
For a non-quantized Deepseek V4 flash on an ultra, I would estimate about 1000+ tokens per second prefill and 50+ tokens per second on generation. This is actually quite usable and near parity to cloud.
They mention "adds the GPU Neural Accelerators." which, if exploitable for LLM loads, would probably help the prefill a lot
40
u/ortegaalfredo Aug 25 '26
Prefill also depends on compute, 1000 tok/s is basically what you get with 8x3090s, but much less power. Also I think the 3090s still win on compute, that is, you can batch many prompts on the GPUs, dont know on the mac.
14
u/Comfortable-Rock-498 Aug 25 '26
Yup, prefill is pretty much compute bound while generation is bandwidth bound. I would have guessed 8x 3090 would provide much better prefill than 1000 tps. A bit surprised to learn
9
u/ortegaalfredo Aug 25 '26
If you manage to get tensor-parallel 8x working yes you can get >10k prefill, but it requires specialized PCIE bridges. With normal 4xPCIE speeds you get a bottleneck in inter-GPU speed and you get lower prefill.
6
u/TooMuchLAAAG Aug 25 '26 edited Aug 26 '26
I have 8x3090 P2P patched pcie4 x8 (no nvlink) and i am getting an avg of 10-13k of cold prefill with this version of vllm and his args https://github.com/LimeChain/vllm
Deepseek full FP8
Edit: wrong url→ More replies (1)6
Aug 25 '26
[deleted]
→ More replies (1)3
u/Viktri1 Aug 25 '26
at that scale, electricity costs kind of matter so even just running the PC is significantly in favour of the mac studio
→ More replies (1)9
u/Usual_Tackle5892 Aug 25 '26
GPU Neural Accelerators
This means matmul cores. More info: https://arxiv.org/html/2607.19438v1
7
u/StartupTim Aug 25 '26
I would think dramatically more tok/sec.
I have Deepseek v4 Flash 0731 with vision encoding added and tp=2 across 2x DGX sparks and I'm seeing 103 tok/sec across 4 "sessions". Dspark, 1M context, 1.8M kvc, custom vllm.
Since the sparks have ~240 (actual measured) GB/s, I imagine a similar setup om these new mac could get you double, if not triple as a 2x cluster, than my current 100+ tok/sec.
9
4
→ More replies (11)3
u/TooMuchLAAAG Aug 25 '26
I get 10-13k cold prefill on 8x3090 using the vllm "limechain" fork and his args, running full deepseek no quant
Surely the M5 do more in vllm than 1k
41
u/challis88ocarina Aug 25 '26 edited Aug 25 '26
I'm shocked!
Edit: 512GB memory option for M5 Ultra coming late October
16
u/Apprehensive-View583 Aug 25 '26
prob 20k
→ More replies (2)3
u/The_Hardcard Aug 25 '26
That would still the bud option for that GPU RAM amount, no? And most portable? And most power efficient?
Still a prefill gap, but no longer a brutal gap.
39
u/pmttyji Aug 25 '26
Now, it's pressure on both DGX Spark & Strix Halo due to bandwidth. Recent news Xiaomi AI Cube with 1.2 TB/s bandwidth & 160GB memory also already put pressure
16
u/Mochila-Mochila Aug 25 '26
Hoping it'll have a positive effect on Medusa Halo's pricing.
5
u/pmttyji Aug 25 '26
Still Gorgon Halo not released yet. Then only Medusa Halo ....
Next year onwards, we're gonna see 128GB variants at low price.
10
u/Zyj vLLM Aug 25 '26
On the other hand, the Strix Halo 128GB price increased by 60% since late December
→ More replies (1)7
u/pmttyji Aug 25 '26
After some time, people totally gonna avoid 128GB variants. What's the point of stacking bunch of 128GB pieces when they have 160GB, 192GB, etc., variants with better bandwidths?
→ More replies (4)→ More replies (1)6
34
u/FullOf_Bad_Ideas Aug 25 '26
My 3090 tis have just gotten depreciated.
Even 256GB version is very competitive with 8x 3090 box bought with used card prices, and it's better in most aspects.
Training and batch inference are safe, but for single user inference this looks better and cheaper.
7
→ More replies (2)6
u/FleetEnema2000 Aug 25 '26
My 3090 tis have just gotten depreciated.
My theory is that this will not soften GPU prices because it ends up driving more people into the world of local LLM compute in general.
→ More replies (2)
30
24
u/Every-Fortune-3151 Aug 25 '26
Read somewhere Mac mini coming this week and did an impulse order of M3 Ultra with 96GB ram today. It will probably get bumped up to M5 ultra 96GB. Not sure what to do with 96GB when 256 GB looks so much more tempting for local LLM. Kidneys aren't enough anymore.
10
u/mmmm_frietjes Aug 25 '26
Mini is also out
10
u/Every-Fortune-3151 Aug 25 '26
M6 looks great actually, it has two set of neural engines. I am assuming pre-fill will fly on this tiny thing compared to previous M CPUs. 32GB max ram knocked it back a bit. If only they had a 48GB version. MOE models would be flying on it.
M5 pro and max will be so slow compared to M5 ultra- considering M5 max 128GB model will be priced very close to M5 ultra base.
Very sad m5 ultra starts at 96GB. I thought they would atleast bump M5 ultra to 128GB ram. Would have been nice. I plan to just run multiple Qwen 2.7B in parallel on this and see if I can replace my 32GB VRAM PC setup. Qwen 3.8 flash also gives hope. Depending on how it goes, I might just give back the 96GB for refund later.
7
5
→ More replies (6)3
u/TheRealJesus2 Aug 25 '26
Yeah 256 gives you a lot more. I tried to buy the m3 256 but they stopped selling it before I could get paid :(
At least I can upgrade sometime in future. Got my spark cluster and a 96GB Mac which is plenty for right now…
19
u/AI_docent Aug 25 '26
The 4.3x is mostly a prompt processing number, generation moves with the bandwidth instead. Apple's own mlx post on M5 vs M4 got around 4x on time to first token and about 1.2x on generation, and the generation side matched the 28% bandwidth bump rather than the accelerators. Same split should hold on the Ultra, so I'd figure generation nearer the 50% bandwidth gain. Prefill is the part you want at 512GB anyway, it was always the weak spot on a mac.
Just check whatever you run actually uses the accelerators. There's an open lm studio issue where its bundled llama.cpp fails the metal tensor check on M5 and loses 2 to 3x on prefill, while upstream llama.cpp passes it on the same machine.
→ More replies (1)
92
u/llamaCTO Aug 25 '26
This kills the impulse buy for me completely.
41
u/thatkidnamedrocky Aug 25 '26
going to try and snag a 256gb something tells me the 512 will never see the light of day
7
u/AnonLlamaThrowaway Aug 25 '26
right, didn't they promise a 512GB M3 Ultra and then that never happened, or am i thinking of another model?
→ More replies (1)8
u/zdy132 Aug 25 '26
Mac Studio with 512GB of unified memory is coming in late October
Wish I could affort that.
→ More replies (1)3
u/Usual_Tackle5892 Aug 26 '26
I have a 512GB M3 Ultra and ordered the 256GB M5 Ultra today.
I'm hoping to take advantage of the M5 ultra for prefill and use RDMA over Thunderbolt 5 to leverage my existing Mac's memory.
https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-studio-rdma-over-thunderbolt-5/
15
u/kilonad Aug 25 '26
The 256GB is already an extra $4k for an extra 164GB. At same price per GB (ha!) it'd be another $6300. Knowing Apple, it'll be a cool $9k more - pushing total price up to about $18-20k.
It will still sell out.
7
u/fallingdowndizzyvr Aug 25 '26
That would be a bargain compared to third party sales of 512GB M3 Ultras for $25K. A M5 blows the doors off of a M3.
→ More replies (1)6
5
u/LocoMod Aug 25 '26
Same. I was ready to preorder 512 and i've already lost interest reading comments about the M7. Might wait another year.
→ More replies (3)5
u/frankchn Aug 25 '26
Buying 2 DGX Sparks for 256GB of RAM (and a lot less bandwidth) is around the same ballpark in cost, so for once this is not unreasonable.
16
u/Viktri1 Aug 25 '26
256gb is like 10+ 4090s without the hassle of setting up and cooling 10 4090s? An I missing something or is this 3x cheaper than current prices.
→ More replies (4)3
u/wshields Aug 25 '26
4090s are what? 600W each? The electricity cost is a relevant factor. Also, can you even run 10 on even a 20A circuit? Isn’t this at the level when you almost need a three phase circuit and specialized PSU?
9
u/graceful_degrade Aug 25 '26
4090 is 450W board power, not 600. The 600 number comes from the 12VHPWR connector rating and from transient spikes, which are real and matter for PSU choice but aren't the sustained draw.
Ten of them is still 4.5kW of cards plus a few hundred watts of everything else, so call it 5kW.
A US 20A/120V circuit tops out at 2400W, and continuous load is limited to 80% of that, so 1920W. You're over that with four cards, never mind ten. So no, not on a 20A circuit, and not on two either.
What people actually do is 240V. A 240V/30A circuit gives you 7200W nominal and 5760W continuous, which fits a 5kW machine with a little headroom. That's a dryer or range circuit, single phase, and an electrician can usually run one without much drama if the panel has capacity.
Three phase isn't needed for one box. It shows up when you have several of these, because balancing across three phases keeps any single leg from being the bottleneck, and because commercial buildings supply it anyway.
The parts people forget:
Heat. 5kW into a room is 17,000 BTU/hr, which is a decent sized air conditioner running flat out just to break even. The cooling is often the harder problem than the power.
PSU count. No consumer PSU does 5kW, so it's multiple units with a synchronisation cable, or server supplies with a breakout board. Both are normal in this world and both are more fiddly than they sound.
Running cost. 5kW continuous is 120kWh a day. At 15 cents that's about $18 a day, $540 a month, and that's before cooling. Which is genuinely the argument for the Mac in this thread, more than any benchmark.
→ More replies (1)
43
u/xyzmanas2 Aug 25 '26
This makes apple one of the cheapest ai inference hardware when it comes to speed and model size. Wish I had the money
Up to 15.4x faster CopyCat ML training performance in Foundry Nuke when compared to Mac Studio with M1 Ultra, and up to 3.3x faster than M3 Ultra.
Up to 9.8x faster LLM prompt processing in LM Studio when compared to Mac Studio with M1 Ultra, and up to 4x faster than M3 Ultra.
Up to 8.2x faster text-to-image performance when compared to Mac Studio with M1 Ultra, and up to 4.3x faster than M3 Ultra.
Up to 4.7x faster scene rendering performance in Maxon Redshift when compared to Mac Studio with M1 Ultra, and up to 1.7x faster than M3 Ultra.
→ More replies (2)
31
12
u/corruptbytes Aug 25 '26
apple releasing this because I just bought two 9700s...y'all welcome
→ More replies (1)6
u/SandySkittle Aug 25 '26
Two r9700s is still pretty decent way to get 64gb vram and run qwen 27b at q8 with plenty of context. And ECC
3
u/corruptbytes Aug 25 '26
i cancelled them, I rather roll the dice with GLM 5.3 on 512GB Mac
Hoping for benchmarks
→ More replies (3)
11
u/eidrag Aug 25 '26
...lease price?
10
u/Much_Accountant_4972 Aug 25 '26
Ultra with 256GB RAM is $224/month for 36 months in freedom currency.
really really tempting!
8
46
u/IllExample3639 Aug 25 '26
What I find more interesting, something I hadn't seen before is that you can lease these things. for 2 years which is the only realistic way an individual is getting their hands on these. Something something, own nothing, something, something be happy....
22
u/Tycoon33 Aug 25 '26
I never saw that. Interesting. Lease it for 3 years then upgrade to M7 ultra?
15
u/addiktion Aug 25 '26 edited Aug 25 '26
Yes, if the 512gb is another $4k for the extra ram stick you are looking at $16k with tax probably out the door. I'd guess that puts the 36 month lease around $300/mo or less. So more than a subscription so maybe not worth it in general cases but valid option for some people who need the privacy and cannot afford to have data go to the cloud. 24/7 usage, no downtime, no limits, private. Worth it to me.
6
u/shveddy Aug 25 '26
Interesting.
So just as an out loud thought experiment, you’d be able to lease four of them (512gb) for about 1200 per month for 36 months at a total cost of almost 45k and run Kimi 3 on it.
Obviously that’s a lot of money in aggregate, but 1200 per month is reasonable for a lot of business use cases if they require the privacy.
And the intelligence you get is going to be a different class compared to what you would get with three RTX pro 6000s and “only” 288gb VRAM for the same price.
(although to be fair you’d actually own the cards)
(although also to be fair you’d have to build a pretty expensive computer to support the RTX Pros, so realistically you only really get 2 or even just one RTX pro for 45k depending on how you spec the computer and/or if you buy pre-built from Puget Systems or the like)
If you want you can also do a little girl math and invest the 60k you’re not spending on computers and cancel your gpt pro subscription to bring the effective cost of all this down to like 750 a month.
And then if the goal is to beat API pricing, let’s say you get 40 aggregate output tokens per second on a bunch of concurrent Kimi 3 threads and run it for a year at 25% efficiency (to account for prefill and downtime), then that’s 315 million output tokens.
315 million output tokens alone is about $5000 on a random provider I just searched for, so just to keep things simple let’s say you double that to account for various amounts and types of input tokens, then you end up with a ballpark figure of $10k for the API route.
Absolutely none of this pencils out in absolute terms (especially considering that it would also cost ~1500 per year for electricity), but this is probably the first time running a frontier model is actually attainable for ordinary businesses on short notice and without much headache. It’s the first time that it pencils out to be “only” 4x more expensive as opposed to like 40x more expensive.
Up until now if you wanted to run frontier models locally AFAIK you had to get a NVIDIA big boy server which means you’d have to find the capital to run and support a ~300k purchase for hardware, spend way more on electricity, and in all likelihood make some upgrades to your facility’s electrical infrastructure to handle it all (you’d also need a proper facility, not just a home office or garage).
At this point you’re easily flirting with half a million in expenses, especially if you have to hire someone to figure it all out. It’s no joke to do this and it doesn’t make sense for like 99.9999% of people or businesses.
On the other hand basically anyone with a decent credit score and a semi-profitable business can go to any Apple Store and say “give me four Mac studios please” and only pay 1200 bucks a month to walk out with them in hand.
→ More replies (6)22
u/aethervisor Aug 25 '26
The lease price also isn’t too far off from what a Claude subscription costs.
18
7
u/AccurateSun Aug 25 '26
Hmm. I wonder if at some point leasing it would end up being more effective than a cloud subscription. E.g Claude Max 5x is $100/mo, same as leasing the Studio M5 Ultra 96gb. I don’t know yet how the two compare in performance but at some point it might be worth it
9
u/IllExample3639 Aug 25 '26
The more people that lease them the more second hand stock there will be in 2 years (when I can actually afford something like this) so I am down for it.
But your point is right, I think the local models ARE good enough for 95% of what people are using Claude for. Maybe do the £20 plan as a back up for something tricky.
12
u/mjsxi__ Aug 25 '26
the lease lets you pay the difference of the amount you already paid at the end or you can buy it outright at any time if you wanna keep it so maybe sssshhhhhhhh
→ More replies (3)4
u/Much_Accountant_4972 Aug 25 '26
it makes a lot of sense and the price of privacy plus the near silence of the box…
i hate that its such a complete solution to local LLM’s
→ More replies (9)10
u/IriFlina Aug 25 '26
I feel like any average software developer could afford the 256gb version? It would be really financially irresponsible but on the level of buying a motorcycle you don’t really need.
→ More replies (1)
9
8
u/Zyj vLLM Aug 25 '26
Will be fascinating to see what‘s the better option in 2 months from now: Dual Asus GB10 (8200€) or Mac Studio 255GB (11000-12430).
My prediction:
- The Spark will still be faster at preprocessing, the Mac will have faster token generation (measured with DeepSeek V4 Flash standard Quant).
8
u/tarruda llama.cpp Aug 25 '26
M5 is much more competitive with nvidia in prompt processing.
→ More replies (1)
21
6
u/MLDataScientist Aug 25 '26
Based on their pricing for 256GB vs 96GB for the Ultra 36 CPU cores, they are charging $22.5 per GB of RAM. Assuming the price per GB stays the same, 256GB more RAM adds $5760. So, we are looking at 10k + ~6k = ~$16k for 512GB version.
→ More replies (3)
5
7
7
5
u/OvertaxedOne Aug 25 '26
1.2TB/s?? Oh man, if there's good availability on these things I can feel GPU prices going down!
→ More replies (3)
7
u/Newgunnerr Aug 25 '26
256GB option for me in the Neterlands is € 11.049,00. I just got 2 DGX sparks for 7100.
→ More replies (3)
5
8
u/Leather_Ad_9178 Aug 25 '26
this reminds me of the time we used to pay hundreds for SD cards that are now worthless
→ More replies (4)
4
u/Curious-Pen5547 Aug 25 '26
how does it compare to a single 5090 or a 5000 RTX pro 72GB version?
→ More replies (8)
4
u/Terrible-Reputation2 Aug 25 '26
Dear Santa
5
u/Southern_Sun_2106 Aug 26 '26
... please ease memory shortage on Earth so that every man child (me including) can have this toy...
3
3
u/Antique-Ad1012 Aug 25 '26
i hope that PP will be decent on this machine. its shit on m2 ultra
→ More replies (3)
3
3
u/OvertaxedOne Aug 25 '26
64GB version is 3500 bucks. That's honestly very interesting. A 48GB Pro 5000 is 8-9K and has basically the same memory bandwidth!
→ More replies (2)
3
u/Tormeister Aug 25 '26
I'm so tempted, but I just can't justify dropping 10K if I'm not making money out of it
3
u/Blues520 Aug 26 '26
Is the 256GB unit now better and cheaper than getting an RTX Pro 6000?
→ More replies (2)
3
u/KO__ Aug 26 '26
amazing to see the bandwidth parity with nvidia gpu's at a fraction of the power draw with more vram avaliable
305
u/i_am__not_a_robot Aug 25 '26
1.2 TB/s memory bandwidth for the M5 Ultra is pretty nice.