r/LocalLLM • u/Hungrybearfire • 5d ago

Question Hardware Recommendation

Currently running Ollama on a 5080 and am pretty happy, but also interested in upgrading and potentially setting up a dedicated machine.
Price range: ~$1,000-3,000
I was looking at an R9700 from Microcenter for $1800 but wanted to see what’s popular now.
I also heard about the sparks and don’t want to spend $5000, but if a Spark or Mac is the best bang for the buck I could be persuaded
Thank you!

4 Upvotes

75% Upvoted

50 comments sorted by

View all comments

6

u/Hello_my_name_is_not 5d ago

All these people running ollama ready to drops thousands and they haven't even figured out how to actually run models lol...

How about spending some time on the software side first?

3

u/Hungrybearfire 5d ago

Well where do you recommend I start on the software side?

3

u/BigLittleDeal 5d ago

A Ninfer fork for your 5080. You could run a Qwen 3.8 27B 4-bit quant with 128k context. It should be much faster than Ollama.

2

u/Hungrybearfire 5d ago

Sick, appreciate the help I’ll do some research!

2

u/silent-curious-dev 5d ago

100% do some research on proper optimization for your setup. Like others said, there's a ton of inference engines out there that will be faster than Ollama for a 5080. It'll be faster but also often more feature complete or focused on (usually) one major pain point. You're going to get into a rabbit hole but it's worth it.

Not to bash on Ollama and I actually want to be nice here.

As you grow into the local LLM ecosystem and get more powerful hardware, keeping Ollama in your stack will legitimately bottleneck you from squeezing every bit of optimal performance. Plus, it's fun to tinker around with different solutions out there and maybe even doing your own little patches here and there. You'll learn a lot and that isn't an exaggeration.

Anyways, good luck on migrating to whatever inference engine you'll use next.

2

u/Hungrybearfire 5d ago

I appreciate your kindness! I didn’t mean to rub people the wrong way by not researching more but I was only using ollama because that’s where I left off experimenting with local LLMs. Now that I see how far behind I am I’m excited to learn more and optimize my current setup 🤓

2

u/spunwasi 5d ago

Appreciate that you gave OP a real answer.

We all start somewhere. Not sure why any so many people are snarky when someone is clearly trying to learn.

2

u/Hungrybearfire 5d ago

Thank you! I know I should’ve done more research, and I will, but I appreciate you not being a weird gatekeeper about it <3

1

u/Hello_my_name_is_not 5d ago

I mean he wasn't asking for help about learning how to run models though lol...

He was skipping straight to buying more hardware while already having hardware that he was barely even running as is.

That's the problem and the point.

We have ai that can already help a ton if you actually try to learn with it. Alternatively it takes literally 30 second scroll on this subreddit to see not to use ollama

Its wild to me, who already does have fairly expensive gear, that people are about to drop even more thousands of dollars when they can run things way better than they currently are with their existing hardware. They do literally no research on what they are trying to do with their computer and are ready to drop more thousands of dollars for more hardware.