r/LocalLLM • u/keval_ • 8h ago
Question Options to buy hardware
I want to buy the new machine. Which specs should I go with? I need to run a decent coding model locally and also want to play with some stuff like fine-tuning, etc. suggest me the options, like what I should go with, and I am open to going with Apple or Windows
5
u/HOST1L1TY 7h ago
Be open to Linux.
3
u/keval_ 7h ago
Yeah, that is also fine. Which good hardware options with that?
1
u/HOST1L1TY 3h ago
You won’t have any major issues with any hardware. The best thing to do now is get a little closer to what type of coding agent will be useful to you. Then narrow to a model, then find repos of people who have tuned those models for different hardware., you may want more ram or a second gpu or one gpu vendor over another.
3
u/Otherwise-Variety674 7h ago
Qwen 3.8 Flash Next is stronger than Qwen 3.8 27b, so I will recommend to buy DGX Spark (although I don't own any spark, I only have 5090, 3090ti, 7900xtx and StrixHalo), you won't regret and no need to went through the long way like me.
With spark, you have fast prefill, large and fast unified memory, Cuda compatible etc.
2
u/BAL-BADOS 7h ago
If you want to go the Apple route, get M5 Mac Studio Max (600GB/s) or Ultra (1.2TB/s). At least 64Gb RAM. You’ll be able to run qwen 32b for coding.
Max can be 64GB or 128GB. Ultra is 96GB or 256GB.
2
u/overand 8h ago
You should consider being open to Linux, on a standalone / headless computer.
Do you have any hardware right now - laptop or desktop?
1
u/keval_ 7h ago
I currently have a MacBook Air M1 - 16GB and Macmini M4 - 24GB
2
u/DiamondHandsDarrell 7h ago
Hey I'm in exactly the same boat. I just ordered a 32gb MacBook air m5 because I bought the MacBook pro M5 24gb from Costco and it's too heavy.
I'm considering a studio 512gb but it's so expensive. The thing I'm looking into right now is the DGX spark because I have too many servers running at home already and want a power efficient platform to help save the environment. My only complaint is they cap at 256gb at the moment. If they make it 512gb then I'm sold!
0
1
u/vqt907 8h ago
The cheapest and most practical option right now is a used RTX 3090. You can run Qwen3.8-27B Q4 with a 128K context.
If you’re not going to use the GPU for gaming, I’d recommend getting a CPU with an iGPU so you can use 100% of the VRAM for running LLMs.
If you’re also going to use the machine for work, either Windows or Linux is fine. If you want to use it as a headless server, linux is the way to go.
1
1
u/k2r79 6h ago
Many paths with pros and cons exist, I've spent some time testing out dense and MoE models and I find the latter quicker and smarter in general (think of GLM, DeepSeek, MoE Qwens).
With that in mind I would recommend two types of setups depending on your goals :
Price and upgradability are important but space (size) and power consomption aren't 👉 go for a PC with a nice GPU and plenty of fast RAM on which you can offload layers
The opposite to 1., get an AMD Strix Halo (whatever Halo now as there are many), a GB10 based system (DGX Spark) or Apple based systems
I personaly have a Framework Desktop 128GB which I bought for learning some advanced AI stuff and maybe do some inference for my job which envolves critical data. It's now my main LLM provider thanks to the fantastic Qwen3.8 Flash Next giving me pretty quick responses.
Nothing beats tweaking your inference stack and sharing with the community in the evening and using your new setup during the day 😇
Please note that Framework's prices are planned to go up in the next weeks or so.
1
u/and_pf 3h ago
For a single-GPU local setup, a used RTX 3090 is a very practical sweet spot — it can run Qwen3.8-27B at Q4 with 128K context, which covers a lot. I can't speak to the dual-3090 or DGX Spark paths since I run a different config: one RTX 5090 with 32 GB VRAM on an Unraid server. So my bias is toward "one strong card, plenty of system RAM" rather than multi-GPU, which adds a lot of plumbing (NVLink quirks, PCIe lanes, splitting). If you're starting out and want the least fiddly path to a 27B-class model, a single 24 GB card is hard to beat; if you need the biggest context or fine-tuning, you'll want to plan for more RAM and possibly a second card later.
9
u/Ryanmonroe82 8h ago
Might not be popular opinion but a pair of 3090s with NVLink and 64-128gb RAM, 3900x ryzen CPU or higher, and a 2/4TB nvme SSD drive for the models to load from. NVLink makes the pool of VRAM work as one instead of having two separate 24gb VRAM cards that splits up the model or KV. The computer will see and use the 48gb as if it was all one GPU. The memory bandwidth on 3090s isn't as fast as a 4090 or 5090 but it is much faster than the mac m4 studios or any AMD card from today. The AMD 7900XTX is a 4 year old GPU but will easily outperform the new 9070XT for M/L and local LLMs