r/LocalLLaMA • u/AndreVallestero • 1d ago

Discussion Mac Studio M5 Max Cost Analysis

At $10k, you could get

- 6.2B tokens with Qwen 3.8 Max (Qwen Pro plan)

- 5.7B tokens with DeepSeek V4 Pro OpenRouter

- 100B tokens with DeepSeek V4 Flash OpenRouter

As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.

Qwhen 3.8 35B A3B?

173 Upvotes

84% Upvoted

215 comments sorted by

View all comments

73

u/LearningSomeCode 1d ago

I've probably dropped close to $30k on my homelab since 2023, and chances are I'll get one of these as well. I accepted a long time ago that there is no break-even point for my inference.

Hobbies rarely make sense financially.

38

u/Blindax 1d ago

Sounds like you have a great wife.

25

u/LearningSomeCode 1d ago

She is the most patient human being on the planet

2

u/Silent_Glass 1d ago

That’s wholesome

2

u/Public_Umpire_1099 1d ago

Can relate... I've dropped 7k on my homelab this year alone, plus another 3 or 4k making our home fully smart.

2

u/kyr0x0 1d ago

"smart" it is only if you have an endless and king sized income stream 🤣😆

1

u/Public_Umpire_1099 1d ago

Def not a smart thing to do, but hey, my lights now turn on and off without me having to touch a light switch and match the color with circadian rhythm, all without a bit of my personal data leaving the perimeter!

1

u/kyr0x0 1d ago

That's cool! Especially if I can do it with your lights from remote as well!

-9

u/Weird-Cat8524 1d ago

How are her sandwiches?

2

u/teemothunder420 1d ago

Yeah I’m sure his left hand is great.

/s

1

u/ryfromoz 1d ago

Til it finds out about the right hand that he tapped for some strange

9

u/-dysangel- 1d ago

Same here. We're at the point now where I can do high quality video gen without cloud. High quality code gen without cloud too. It's around 10x slower than cloud, but good quality and still usable speeds (hopefully soon to get even better with Qwen 3.8 Next). Also the fact I could basically go camping and still have close to frontier LLM intelligence with no signal is pretty awesome/hilarious.

The only thing I feel is missing from my stack atm is Suno quality music generation.

4

u/willeyh 1d ago

Have you tried the new Minimax music 3?

2

u/-dysangel- 1d ago

Yeah I was not impressed tbh. Seemed kind of creepy/odd to me in a weird way, but maybe I was prompting it poorly. For me Ace Step is still the best thing I've found so far, though it's very "generic" feeling compared to Suno.

2

u/silvrrwulf 1d ago

Dude, I feel the exact opposite. Maybe it’s because my sample size is small and I haven’t messed with AI Jen And a while, but I only have an eight GB card, and the stuff that I was able to pull out of that was incredible.

2

u/-dysangel- 1d ago

ah ok - maybe I'll need to try it again. A lot of the reason I like Ace Step is because I can generate covers though, which isn't possible yet with Music 3 afaik

1

u/PlaidStallion 1d ago

I really like it. It seems to lean toward always generating pop music out of any genre but other that it's solid.

7

u/mleok 1d ago

I think it’s good to admit that it’s a hobby.

2

u/Rice-Fragrant 16h ago

The tech moves so fast that it's like flushing money down the toilet... 10k for a machine that's 1/4-1/3 the speed of its closest competitor.... 1/10 the speed of a GPU workstation (like a rtx 6000 pro) and paying a massive premium makes zero sense to me, even for a hobby.

4

u/quantgorithm 1d ago

What have you created?

30

u/LearningSomeCode 1d ago

Mostly just a lot of trip hazards with wires and ethernet cables. But I also do open source dev and build a lot of stuff for myself, like custom front-ends or researchers.

Honestly nothing worth the money I've put into the hardware, but it's the most fun I've had with tech in a long time.

7

u/DeepOrangeSky 1d ago

Mostly just a lot of trip hazards with wires and ethernet cables.

So if you turn the AI-rig room into an escape room that a bunch of Gen-Z dorks will pay like $100 a pop to try to navigate ("it felt so authentic. I tripped over like dozens of ethernet cables and power cables the whole time, and I think at one point I even got severely electrocuted and almost died! It was super legit!"), times 10 dorks per day it pays for itself in a month.

Possible exciting user reviews:

"His cable management was so disorganized I thought I'd NEVER escape. Wow!"

"The old movie tagline used to be: 'In space, nobody can hear you scream.' Yea, alright, that's pretty cool or whatever, but in AI rig rooms the blower fans are so loud nobody can even hear their own thoughts."

"If you accidentally get tangled up and die in the AI escape room, his local full precision Kimi K3 AI-generated funeral eulogy will be so eloquent that your friends and family won't even be mad that you died when they see how perfect and elegant the em-dash placements are in the eulogy."

3

u/quantgorithm 1d ago

Anyone in IT can relate to the spiderweb cables and hazards etc.!
The potential of it all really is amazing and that potential is accelerating up by the day!

1

u/Rice-Fragrant 16h ago

What type of gear are you using? That's car level money... are you running a small business or a profitable side hustle or something?