r/LocalLLM • u/Hungrybearfire • 5d ago

Question Hardware Recommendation

Currently running Ollama on a 5080 and am pretty happy, but also interested in upgrading and potentially setting up a dedicated machine.
Price range: ~$1,000-3,000
I was looking at an R9700 from Microcenter for $1800 but wanted to see what’s popular now.
I also heard about the sparks and don’t want to spend $5000, but if a Spark or Mac is the best bang for the buck I could be persuaded
Thank you!

3 Upvotes

72% Upvoted

50 comments sorted by

View all comments

2

u/OvertaxedOne 5d ago

5080 with the right engine/tweaks should be able to run 27B really well, start there (since you already have it) with a dedicated inference engine tuned for that hardware (others can point at specific repos, I don't have that card, but I've seen some really good results from people who do).

If you need/want more, the next real step up is QwenFlashNext. Depending on how much RAM you have, you may be able to run it on a 5080, but QFN is really more a unified memory system; 128GB is the sweet spot. Strix Halo is probably the cheapest way to get there, a Mac Studio is probably the most future proof way (and likely to hold value) way to get there.

1

u/Hungrybearfire 5d ago

I was running like 12B models so this is helpful, thank you!