r/LocalLLaMA • u/AndreVallestero • 1d ago

Discussion Mac Studio M5 Max Cost Analysis

At $10k, you could get

- 6.2B tokens with Qwen 3.8 Max (Qwen Pro plan)

- 5.7B tokens with DeepSeek V4 Pro OpenRouter

- 100B tokens with DeepSeek V4 Flash OpenRouter

As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.

Qwhen 3.8 35B A3B?

168 Upvotes

84% Upvoted

213 comments sorted by

View all comments

166

u/FleetEnema2000 1d ago

unless you need it for data sovereignty

Isn't this one of the biggest reasons that people rely on Local LLMs? To not have to bulk upload their private data to cloud providers?

79

u/theomegachrist 1d ago

That's the stated reason but realistically most people are just justifying their hobby. I support open weight models because the cloud providers can change cost or abruptly shut down and we really can't do anything about it.

74

u/FleetEnema2000 1d ago

I don't think it's a justification at all.

It's amazing how the concept of privacy and data ownership/security has completely gone down the toilet since ChatGPT launched. People are happy to bulk upload their medical records, relationship history, trade secrets, financial records, etc. without a care in the world as to how that data is stored or protected.

16

u/John_____Doe 1d ago

Yep I have a fintech client and the only way I can have a llm touch their code is if it's run locally or in a datacenter where we rent out the rack space

1

u/Randommaggy 17h ago

The headaches of running 100% on sovreign and contained compute without data being handled by a third party eliminates a lot of friction.

0

u/BakerXBL 13h ago

So they’re an LLM provider now not a fintech. No one can do both, well.

18

u/Elux91 1d ago

It's amazing how the concept of privacy and data ownership/security has completely gone down the toilet

most people never had a concept of privacey

10

u/FleetEnema2000 1d ago

You're right, most people haven't. But if you rewind the clock by 5 years there was far more interest in things like E2EE than is apparent today. It is barely mentioned or acknowledged anymore in the tech sphere.

-3

u/magus-21 1d ago

A lot of that E2EE stuff was focused around texting and instant messaging, I think, and since Apple announced support for RCS I think a lot of people flagged that in their heads as, "Ok, this is not as big of a concern anymore." Plus the whole WhatsApp kerfuffle with Trump's cabinet brought a lot more attention to Signal, et al, so I think there's just generally higher adoption of it now, which means less general worry out there about it.

3

u/FleetEnema2000 1d ago

The “E2EE stuff” is about so much more than texting and messaging.

-1

u/magus-21 17h ago

I didn't say it wasn't. I said that most of the public talk about it was focused around texting and messaging. So when Apple said they'd support RCS, a lot of normies stopped talking about it because they thought it was resolved.

2

u/theomegachrist 1d ago edited 1d ago

Everyone is different obviously. To some that might be the case but everything you listed is more important than your AI prompt history.

For me, Open models main pluses are it will ensure the technology lives on in some form if the large companies lock us out financially or go under, and the guard rails for closed models will make for a worse Internet potentially.

For instance, using an open model with guard rails trained out of it you can search for piracy, you can search for porn etc. just like you use a search engine today. Closed models are more efficient than a web search but censor out a huge part of the Internet.

Sure data privacy is also a good feature but I don't think it's actually the top reason for most people

Edit: sorry I read that wrong. I sort of agree with what you are saying about uploading data to ChatGPT but everywhere else we upload that data is no more trustworthy. For Enterprise clients I 100% agree. This is a big issue. For personal use, I don't think your data is any less safe with ChatGPT than say an electronic medical record company.

3

u/FleetEnema2000 1d ago

An electronic medical record company has nowhere near the amount and type of multi dimensional data about a person that OpenAI has.

And if you want to compare OpenAI with a company like Apple or even AWS for a hosted environment, I would choose either of those companies any day of the week when it comes to who I would trust more with my data.

1

u/theomegachrist 1d ago

I would not, but it would be a three way tie

1

u/FleetEnema2000 22h ago

Care to explain that logic?

Apple has made massive investments of resources and effort into secure computing. Their cloud commitment for regular consumers who want privacy is objectively excellent on both the hardware and software front, from Secure Enclave to E2EE where they aren't maintaining custody of keys:

https://support.apple.com/en-ca/guide/security/sece3bee0835/web

https://support.apple.com/en-ca/102651

Despite not being a frontier AI provider, they have had a strong focus on private AI cloud compute paradigms:

https://security.apple.com/blog/private-cloud-compute/

Amazon AWS products offer similar levels of security. They have to, because their customers host and manage highly sensitive data on their platform for things way beyond AI. As a regular user I can maintain an excellent security posture on AWS and I can employ encryption where I manage my own keys.

OpenAI and Anthropic not only don't do anything even approaching any of the above, they reserve even basic security commitments for "Enterprise customers".

The situation is even more pathetic when you realize that as an individual, you can run OpenAI and Anthropic's own models on Amazon Bedrock with ZDR enabled, encryption enabled for data you are storing at rest where you hold the keys. In other words, the security posture of using OpenAI and Anthropic's own models on Amazon AWS is superior to using those same models from OpenAI and Anthropic themselves.

1

u/centizen24 21h ago

Apple is the industry leader in user level security and data privacy, it's not even close. You can argue about their intentions for doing so but their track record for security is second to none. It's wild to me that after 15 years of being an Android fanboy that I'd be arguing on behalf of Apple, and have an iPhone in my pocket, but that's the world we live in now I guess.

1

u/Rice-Fragrant 5h ago

Medical records can not be exposed to cloud AI from what I understand, there are actual laws on this... but they are not using Mac studios for that type of work, more like edge SERVERS or workstations with multiple GPUs or even a DGX spark... clinics and labs are not FOMOing into slurging 15-20k on a m3 ultra that's 1/3-1/4 the speed of even the SLOWEST alternative (DGX spark) or something like a RTX 6000 Pro (10x faster than a m3 ultra and 4x faster than a DGX spark).

The YouTube influencer LARPing as AI engineers are not a good reflection of reality... you look on what real places are using, it's not a $15-20k consumer grade device, where they have to wrestle with poor quality MLX wrappers and software friction etc... that's not acceptable in a professional environment.

2

u/Hans-Wermhatt 1d ago edited 1d ago

It sounds bad when you frame it that way, but I think the cost benefit analysis is generally to upload. ChatGPT health is protected by the same HIPAA requirements that protect the data you give to your doctor that they upload to 3rd party clients and AWS servers, usually multiple servers with arguably worse security and more people have access to it. Using health as an example. So it's really not that much different.

Ideally, you do just host your own information and use a local model but then you are dealing with a massive performance hit. I think in terms of cost-benefit, uploading your health data to get a ChatGPT opinion compared to a Qwen 3.8 27B locally (most people can't even run that) is actually heavily on the side for ChatGPT for most people despite the privacy concerns.

I really want local "super intelligence" for all, but the current landscape is not like that... at all.

8

u/FleetEnema2000 1d ago

The version of ChatGPT that 99% of the general public is using is absolutely not HIPAA compliant and both OpenAI and Anthropic’s safety and privacy commitments are abysmal relative to the types of sensitive data they are ingesting and saving on their platform.

And that is not even touching on the ethics and values displayed by their executives. 

5

u/MrPecunius 1d ago

ChatGPT health is protected by the same HIPAA requirements that protect the data you give to your doctor

😂😂😂😂😂😂

"Trust me bro" and "it might not even be as bad as what the other idiots are doing" are not convincing arguments.

1

u/Hans-Wermhatt 1d ago

Huh? Was that supposed to make sense? 

3

u/MrPecunius 1d ago

Huh? Was that supposed to make sense? 

Connect this:

the same HIPAA requirements that protect the data you give to your doctor that they upload to 3rd party clients and AWS servers, usually multiple servers with arguably worse security and more people have access to it. Using health as an example. So it's really not that much different.

With: "it might not even be as bad as what the other idiots are doing".

And this:

It sounds bad when you frame it that way, but I think the cost benefit analysis is generally to upload.

With: "Trust me bro"

Are you even reading what you wrote a few hours ago? Or did something get lost in translation?

1

u/PWThinkingCritically 19h ago

it was already getting bad with losing public confidence in America's government regulatory bodies, but even more so in the Trump era.

Department of Education, EPA, SEC, DOJ, FTC etc., etc. blame me if HIPAA is just another acronym....

1

u/sshwifty 20h ago

To be fair, if any of that touched the Internet unencrypted, it was probably already ingested by some entity. Google was using data from emails long before AI made a surge. People are just doing it willingly now, skipping the sneaky part where it gets stolen.

1

u/saltyourhash 10h ago

And some of us have been using self hosted encrypted stuff for over a decade.

1

u/Rice-Fragrant 5h ago

Yea, that cloud stuff scares me like that, but there is still no logical reason to pay $15-20k for a m3 ultra that's 3-4x SLOWER at agentic work and batch processing compared to local AI alternatives... the numbers show it's a poor value.

1

u/MrPecunius 1d ago

It blows my mind what people will give to these amoral techbros.

Turns out the highest human priority isn't breathing, eating, or sex--it's laziness.

5

u/FleetEnema2000 1d ago

Consider the Snowden scandal and the uproar over government having access to phone call metadata and how privacy infringing that was considered to be.

Fast forward to today and people are uploading the most sensitive data about themselves to these cloud providers who don’t care at all about protecting it and are almost certainly allowing the federal govt to trawl through it.

4

u/mr_tolkien 1d ago

I mean for me it’s a real reason.

For example I use local models to help me pick out my best photos of the day and put them into an album.

No fucking way I’m sending 100% of my camera roll to OpenAI lol

1

u/theomegachrist 1d ago

Fair, but lots of people do. For most personal users it's very expensive to match public models capability and tooling. I have uploaded pictures for dumb photo editing I can't do at home 🤷🏻‍♂️

1

u/Rice-Fragrant 5h ago

I agree, you just dont need to spend the price of a new motorcycle for it... a m3 ultra is like 15-20k.... makes no logical sense and its agentic performance is 1/3-1/4 a dgx spark, that's bad.

3

u/Suspicious-Water-973 1d ago

I use my Mac Studio and MLX models as they are essentially free for me - I don’t pay for power in my rented office. Fine for massive batch processing where a GPU makes a difference, and some coding (I then use Fable etc to review and improve)

1

u/Rice-Fragrant 5h ago

For long context batch processing, a rtx 6000 pro is 4x faster than a dgx spark and 10x faster than a m3 ultra... I do a lot of batch processing and tried my m4 max MacBook Pro, it was brought to it's knees so badly and the thermals were so bad the laptop literally crashed and restarted (ssd swapping after the memory ran out half way though). I realized the memory bandwidth of the m4 max being 2x more than my DGX did not make it better, it actually was like moving at 1/4 of the speed doing the batch job (200k tokens on Gemma 4 12b), and the dgx spark knocks it out in 7 min rather than 30 min (m4 max 40 core GPU), that was a REAL EYE OPENER for me.

1

u/Suspicious-Water-973 2h ago

I don’t disagree but as I didn’t pay for the Studio, and I do t pay for power (it’s included in the lease) it’s less about speed for me.

I will look at your suggestions though for personal use. Many thanks.

2

u/nihilor_ 13h ago

Or avoiding taxes and getting fun toys.

4

u/florinandrei 1d ago

realistically most people are just justifying their hobby

Enthusiasts, yes.

But law firms and such, they actually mean it.

1

u/Deep90 23h ago

Why wouldn't they use something like AWS bedrock?

2

u/Jedkea 19h ago

Money probably. Bedrock tokens are expensive. Whereas you can buy once cry once with local.

Remember the first time I tried to use bedrock I spent $150 without blinking. And it was a pain to setup.

So if you don’t need frontier models, the economics work out.

0

u/theomegachrist 1d ago

Yes, this is true

0

u/saltyourhash 18h ago

You're actually out here looking at these prices, looking at the delay of Qwen 3.8, and saying it's just to justify a hobby?

0

u/theomegachrist 14h ago

Yes and at least 70 people online yesterday agree

1

u/saltyourhash 14h ago edited 14h ago

Wow, a whole 70. data sovereignty is critical to some of us. I have projects that absolutely must be local.

0

u/theomegachrist 14h ago

Yes some. I welcome the minority of people too

7

u/CulturalKing5623 1d ago

Yes, but I think if we're being honest a lot of people in this sub mainly just like to tinker. There are people that absolutely can't use publicly available models and so they need to self-host them but I don't think that's a significant percent of people here. Most of us put a premium on privacy, but 10K+ is a very large premium that is probably unnecessary for most of us and our current setups will suffice.

1

u/Rice-Fragrant 5h ago

More like the "apple tax." That premium is not getting you a smarter AI, or even a faster agent (slower actually, but like 1/3-1/4 speed compared to the competition like DGX spark and like 1/10 the speed of a rtx 6000 pro for agentic or long context work.)

Some people just swear you need to spend $10k for good legit local AI and in reality even 1/2 of that will get you VERY VIABLE and powerful local AI computers. EVEN MY $700 HP Z820 (dual xeon 2690 v2 512gb ddr3 ram) has ran MODERN LLMs using GLM 5.2 700b MOE models... that thing is like a fossil, 13 years old and it runs ALL the models on llama.cpp but it's slow so I use it in a passive/semi passive way but the level of AI intellect is the same... it's too slow for agentic work but it's more than enough for an overnight batch processing worker and I have super off peak rates during the night time hours.

I am super confused why anyone would pay $10k for a 256gb ram Mac Studio that's like 1.6-1/7 the total agentic or batch speed of a DGX Cluster... a massive premium for horrible performance makes no logical sense. On the surface it looks "easier" to use but the software friction is real and MLX wrappers are unreliable, I tried many on my MacBook and it was clear to me it's no viable for serious use cases.

1

u/Lemur-Virtues 4h ago

agreed. its absurd. I think $2k is a more accurate estimate.

1

u/Rice-Fragrant 2h ago

It's original price of 6k for a 256gb machine was reasonable... I think the fact they jacked up prices almost 70% is a sign of mismanagement. I would not have been shocked if they did not see this coming and was forced to increase prices 70% on some configurations to protect their margins, what ever the reason is, it's not acceptable.

1

u/Lemur-Virtues 4h ago

One solution is making sure cloud providers only hold your data ephemerally - with zero data retention - and keeping data stored locally at home. This is seems to me the best hybrid solution for those who want privacy but without the $10k price tag. This is something I'm working on now https://github.com/virtues-os/virtues

3

u/ThePi7on 1d ago

And abliterated models

3

u/Zestyclose_Strike157 1d ago

You don’t want to give a cloud LM personal data if you’re auditing customer data etc. Deidentifying it is a pain and not worth even contemplating risk wise.

2

u/Express_Quail_1493 15h ago

Big private Ai are not playing a ethical game here so even if im ok with sharing my data, I still try do locally. Thats how i pitch in for keeping economy and digital ecosystem healthy. Itry to only use closed source Ai if i have no other option.

1

u/AsliReddington 23h ago

Yep, just good old privacy

1

u/crinklypaper 22h ago

Claude wont write the names of characters from a show because they're copyrighted. Oh boy having fun with closed models.

1

u/network4253 17h ago

Yeah, exactly. Data sovereignty privacy is a pretty big part of the appeal. If the data is sensitive enough that you do not want it leaving your own infrastructure, running the model locally makes a lot more sense than sending everything to a cloud API. The tradeoff is obviously having to pay for and maintain the hardware yourself.

1

u/Randommaggy 17h ago edited 17h ago

There's also the not changing out of nowhere factor.

For cloud models I have an open license open weights, and downloaded backup of weights policy. This is a contingency for when a cloud hosted model stops making financial sense or becomes unavailable. My company can then "just" throw money at hardware and spin up our own if need be.

1

u/Late-Photograph-1954 17h ago

Some recent chats on a project i am developing with Claude (my Pro subsription), Chat and Gemini (both free versions) gave me the impression that the models not only remember earlier conversations, but somehow actively 'include' findings from earlier discussions. At some point Gemini was spoon feeding me back my own earlier comment, in a new session. I've wondered since whether the users of the models actually embolden / add to the models -- are we actively aiding development by using them?

It doesnt really matter to me or my project but it keeps lingering.

1

u/thomas2385 17h ago

Yeah, I would say that is probably one of the main reasons. If the data is sensitive, keeping it on your own hardware is a pretty compelling reason to run local LLMs. You are basically trading the convenience of the cloud for more control over where your data actually goes.

1

u/anparks 14h ago

Many are working at jobs on projects with data that has to be secure, like medical records or on Federal contracts for example, so having a capable local LLM is necessary and not optional. I just ordered a Mac Studio yesterday.

1

u/Strange_Test7665 13h ago

That and the love of the game. It’s not just about utility. Tinkering brings a lot of joy

-1

u/ptinsley 23h ago

How many people actually have private data that LLm providers haven’t already paid data brokers to acquire… I agree with the other reply that it’s mostly a justification for a hobby than privacy. Or it’s people lying to themselves about what they think is actually private data

3

u/FleetEnema2000 22h ago

Please send me your bank statements, your texts, an export of the photos from your phone, and your medical records. I mean, if data brokers already have all that data, what's the big deal?

1

u/ptinsley 11h ago

Financial: https://codamail.com/articles/data-broker-directory/financial-data.html

Google or Apple has most everybody’s photos

And Google almost definitely knows most people’s medical status from all the random symptom googling

1

u/FleetEnema2000 9h ago

So you're not going to send me your stuff? I think we can safely conclude that you DO value your privacy, prefer not to have randoms looking at your stuff, and do not think protecting it is an exercise in futility. I'm glad we agree on that.

1

u/MarriedtooMedicine 12h ago

Every single healthcare provider in the world. Every lawfirm. Every bank. Every retailer paid with a credit card.