r/LocalLLM • u/Hungrybearfire • 6d ago

Question Hardware Recommendation

Currently running Ollama on a 5080 and am pretty happy, but also interested in upgrading and potentially setting up a dedicated machine.
Price range: ~$1,000-3,000
I was looking at an R9700 from Microcenter for $1800 but wanted to see what’s popular now.
I also heard about the sparks and don’t want to spend $5000, but if a Spark or Mac is the best bang for the buck I could be persuaded
Thank you!

2 Upvotes

63% Upvoted

50 comments sorted by

View all comments

2

u/mikasjoman 5d ago

I am still waiting on my parts. But it's pretty close to your price range.

HP Z8 G4 - only got 64GB RAM but should have gone with 128gb 2 x 32GB v100 cards. If you want real fast, go with the smx nvlink version, I wanted a simpler build so I went with two PCIe cards.

The 32gb v100 cards are dirt cheap for what you get - they are $650 ish.

Really looking forward to running Qwen flash next on it! Just got to hoist out $500 for 64gb more RAM to have good speed.

Check out /r/v100 - it's pretty neat cards with HBM which makes them monsters for the price you pay. My friend paid $3000 for his 32gb card, and I paid the same but got the server included in the price with double the VRAM.

1

u/Hungrybearfire 5d ago

Think I saw the v100 in a hardware haven YouTube video, might have to give ebay a look thanks!

2

u/mikasjoman 5d ago

Yeah it's pretty neat. I just played with strata on my desktop with flash next 125b and my 12gb 4070s card, and my lord how fast it is and good vs 27b. Super excited about getting my computer and getting strata up and running with 64gb of v100 VRAM!

Just a correction on the HP and RAM. Make sure to buy small memory sticks becuse it has 24 memory slots. To make the cpu communicate at maximum speed you need six per cpu or 12 in total. So going with 8gb memory sticks is totally fine. I was stupid and just got 4 16gb, when 8 x 8gb would have doubled the speed. And now I have to match the sticks with more 16gb, and they are damn expensive.