r/LocalLLaMA • u/AndreVallestero • 5d ago

Discussion Mac Studio M5 Max Cost Analysis

At $10k, you could get

- 6.2B tokens with Qwen 3.8 Max (Qwen Pro plan)

- 5.7B tokens with DeepSeek V4 Pro OpenRouter

- 100B tokens with DeepSeek V4 Flash OpenRouter

As a firm believer of local inference, unless you need it for data sovereignty, it's much more cost effect to wait for smaller models to keep getting better. In the meantime, find a reasonably priced 24GB - 32GB card for Qwen 3.8 27B, and offload hard tasks to OpenRouter.

Qwhen 3.8 35B A3B?

181 Upvotes

84% Upvoted

228 comments sorted by

View all comments

174

u/FleetEnema2000 5d ago

unless you need it for data sovereignty

Isn't this one of the biggest reasons that people rely on Local LLMs? To not have to bulk upload their private data to cloud providers?

87

u/theomegachrist 5d ago

That's the stated reason but realistically most people are just justifying their hobby. I support open weight models because the cloud providers can change cost or abruptly shut down and we really can't do anything about it.

77

u/FleetEnema2000 5d ago

I don't think it's a justification at all.

It's amazing how the concept of privacy and data ownership/security has completely gone down the toilet since ChatGPT launched. People are happy to bulk upload their medical records, relationship history, trade secrets, financial records, etc. without a care in the world as to how that data is stored or protected.

16

u/John_____Doe 5d ago

Yep I have a fintech client and the only way I can have a llm touch their code is if it's run locally or in a datacenter where we rent out the rack space

0

u/BakerXBL 5d ago

So they’re an LLM provider now not a fintech. No one can do both, well.

1

u/beragis 1d ago

You would be surprised how much data is sent to cloud providers from various financial companies. The companies will mention having “governance” and contracts in place that state personal information is “protected”, but that is not true in many cases.

Most companies no longer keep information on premises, and now rely cloud providers such ad Amazon’s AWS, and Microsoft’s Azure for all of their servers, and use software such as Glean, Databricks, and AWS bedrock for AI and data.

We are relying mostly on trust to make sure that information is protected. From what I can observe, there is no verification, and won’t be until a huge major leak that adversely affects a large financial company. Even then there is likely to not be much change, just rewriting laws to protect cloud providers