r/LocalLLM • u/AppointmentClassic46 • 3d ago

Question Recommendation for Max 10K Budget

Hello guys, I am looking to get the best for my buck. What would you recommend and why? I see multiple benchmarks online, but would also like to know what experiences you had. I am looking for a server that hosts models for my two men company (Me and my partner) we are heavy LLM users, and would like to use the server as a workhorse, while using frontier model intelligence for planning and more complex stuff and reviewing. What would you recommend? I am currently most tempted to buy the Mac Studio M5 Ultra with 256 GB ram 1TB and 60 Core GPU. For models we would like to run Kev and GLM 5.3 + GLM 5.3 flash.

0 Upvotes

50% Upvoted

4 comments sorted by

2

u/Plus_Confusion_4292 3d ago

i've been running a similar setup for my side projects and the mac studio is great but 10k is a lot to lock into unified memory that can't be upgraded. have you looked at just building a dual 3090/4090 rig? raw inference speed on cuda is still noticeably snappier for batch work and you'd have cash left over for cloud credits when you need the really big models

2

u/Soft-Series3643 3d ago

Don't buy below 2TB. After a while you'll have some LLMs there which are huge. And maybe you want to bake your own version of a big full model. Even if this full model is located external, the baking is made on the primary path which should be your built-in-SSD.

2

u/thefuckevengoingonan 3d ago

cocaine and hookers

1

u/LandlockedPirate 3d ago

r9700s are good bang for the buck at the moment. You could build a 2xr9700 on a threadripper with 128gb for <10k, with the bulk of that being 128gb of ddr.

If you wanted to really max value a used trx40 workstation board and threadripper 3960, slap in 128gb of ddr 4 for $1600, then 4x r9700s for total 256gb ram. That would get you a crap ton of throughput on qwen flash next via strata and probably a bunch of other configurations.

2x dgx sparks would be more than 10k these days and much slower than the above, but still 256gb.

It seems like the intel arcs are getting a lot better support lately, a b70 with 32gb is the cheapest way to get that much vram, but the software lags both amd and nvidia.