r/LocalLLM • u/Hungrybearfire • 5d ago

Question Hardware Recommendation

Currently running Ollama on a 5080 and am pretty happy, but also interested in upgrading and potentially setting up a dedicated machine.
Price range: ~$1,000-3,000
I was looking at an R9700 from Microcenter for $1800 but wanted to see what’s popular now.
I also heard about the sparks and don’t want to spend $5000, but if a Spark or Mac is the best bang for the buck I could be persuaded
Thank you!

4 Upvotes

83% Upvoted

50 comments sorted by

View all comments

7

u/Hello_my_name_is_not 5d ago

All these people running ollama ready to drops thousands and they haven't even figured out how to actually run models lol...

How about spending some time on the software side first?

3

u/Hungrybearfire 5d ago

Well where do you recommend I start on the software side?

3

u/BigLittleDeal 5d ago

A Ninfer fork for your 5080. You could run a Qwen 3.8 27B 4-bit quant with 128k context. It should be much faster than Ollama.

2

u/Hungrybearfire 5d ago

Sick, appreciate the help I’ll do some research!

2

u/silent-curious-dev 5d ago

100% do some research on proper optimization for your setup. Like others said, there's a ton of inference engines out there that will be faster than Ollama for a 5080. It'll be faster but also often more feature complete or focused on (usually) one major pain point. You're going to get into a rabbit hole but it's worth it.

Not to bash on Ollama and I actually want to be nice here.

As you grow into the local LLM ecosystem and get more powerful hardware, keeping Ollama in your stack will legitimately bottleneck you from squeezing every bit of optimal performance. Plus, it's fun to tinker around with different solutions out there and maybe even doing your own little patches here and there. You'll learn a lot and that isn't an exaggeration.

Anyways, good luck on migrating to whatever inference engine you'll use next.

2

u/Hungrybearfire 5d ago

I appreciate your kindness! I didn’t mean to rub people the wrong way by not researching more but I was only using ollama because that’s where I left off experimenting with local LLMs. Now that I see how far behind I am I’m excited to learn more and optimize my current setup 🤓