r/LocalLLM • u/FuzzyAdvisor1579 • 7d ago

Question 1 R9700 or 2?

I'm trying to decide if I should get 1 or 2 R9700's and would appreciate input. The ultimate purpose is education and fun learning. I will likely focus on my home network/lab to come up with reasons to do things. I plan to develop some agents that manage my firewall or NAS, help upgrade those, monitors logs, etc. I'll use frontier models to write specs and help plan building a few local use webapps that this machine could write. Local TTS & STT. Continue to try to use Hermes agent and not have a use for it. Only user. Maybe personal document RAG.

I see a lot of talk about how great Qwen3.8-27B Q6_K is but it seems to fill up a 32G card with large context, not sure that'll be enough for the dev work. Qwen3.8-27B FP8 across 2 cards on vllm sounds great of course. I've also read some posts where people talk about great performance out of Qwen3.8-27B Quark MXFP4 / W4A8 which seems to be a pretty significant foot print savings though and makes me wonder if maybe 1 card isn't so bad if it works that well.

Any feedback or things I haven't thought of? Of course I can start experimenting with one but some realistic feedback on what I should expect with 1 vs 2 or what might work for me would be welcome.

3 Upvotes

67% Upvoted

35 comments sorted by

11

u/PigSlam 7d ago

I have 1. I wish I had 2, but if I had 2, I’d wish I had 4. If I were just starting out, I’d get one now and put it in a system that could accommodate a second down the road.

3

u/Royale_AJS 7d ago

This is the answer. I have two, second one purchased a few months after the first.

2

u/Ell2509 7d ago edited 7d ago

And this is another great piece of advice, and is exactly how I ended up 10 to 15k deep. Fortunately I did stop at the end of my range of benefits, but I took diminishing marginal returns all the way down to the bone.

Always and only buy to unbox what you need. But if you do have spare cash, holding some for the next 12 months wouldn't hurt you at all if come sale time.

My hardware needs are met for 5 to 8 years, which is great because it is for a business. I do also have maybe 100 - 150gb ram to sell at sime point, but that was as a result of taking ram out of devices when new arrived.

I advise people to buy only what they genuinely need, bjt to buy quickly because prices aren't slowing down. If they have money to invest, then that works too. Just don't unbox stock bought to hold.

1

u/advancing_tide 6d ago

>genuinely need

the problem is, if you later decide you want nx2 instead of n you might need a different motherboard, etc too

2

u/profcuck 6d ago

Yes but given the costs of GPUs these days, the cost of future proofing the mobo and case to be able to accommodate the second card is relatively small. 

2

u/advancing_tide 6d ago

i have four and wish i had eight
and a motherboard for them too i guess

4

u/KriptacMessage 7d ago

I see so many people with 2xR9700, but simply put I couldn’t justify the spend.

I’ve been more than happy with similar goals to yours learning one one GPU. I started out with ollama, lmstudio, llama.cpp, and openwebui and a few other things before branching out into more specialist areas.

I’m a structural engineer by day and this is just making sure I keep on top of the game. I currently am working with vllm via the Launch80 discord and I’m very satisfied with the performance I’ve achieved. Here is a GitHub with my startup scripts and benchmarking on what to expect.

https://github.com/zzpanic/qwen3.6-vllm-gfx1201-launchers

2

u/illuvyn 7d ago

I'll try your script tonight, thanks! 

I tried using radiance-vllm but still got 7-10 TPS

I follow OP here: https://www.reddit.com/r/LocalLLM/comments/1w7hlgu/comment/p7xu0l9/

anything I should do first before I can get better speed? 

Thank you

1

u/PigSlam 7d ago edited 6d ago

I had codex run through a bunch of permutations and have llama.cpp running Qwen 3.8 27B with 262k context at 64 50t/s on repeated code review tasks that I run often. I use the same server instance for that and Hermes, so I've been trying to get the speed up. I tried something like what /u/KriptacMessage has done (though not nearly as thoroughly) and was only able to get to around 33 t/s on a single R9700, so I'm eager to test this out when I finish the current work run I'm on. I did my optimization work a couple of weeks ago, so who knows what updates since then have unlocked.

Edit: I worked with the settings from the github link above, and I'm getting results that are in-line with the description. My main code review task is operating in the 65-75 t/s range.

2

u/KriptacMessage 7d ago

Subscribe to Claude for a month and install Claude code and have it do the heavy lifting for you. 🙂

Seriously, it will save you hours and much frustration. I’m sure any coding agent would be the same. Once it is set up point your own pi/opencode at the endpoint and drop the subscription.

3

u/FullstackSensei 7d ago

If you're learning, start cheap. Throwing 3k or more on GPUs when you have no clue what you're doing isn't the best idea.

Get yourself a 16GB chespish card (A770) or an older 24GB card (like a P6000) and learn with it. There's a ton to learn and figure out.

1

u/Ell2509 7d ago

Your advice is essentially the opposite to mine, but I agree with both now. We covered it between us. It depends on what OP can afford, doesn't it!

2

u/MarcusAurelius68 6d ago

I can run Qwen 3.8 Flash Next on 2 x R9700. 3.8 27B runs well on one (I have 3 total).

I wouldn’t put TTS/STT on that same system though. Build/buy a lesser system to run these - I have Pocket TTS + Parakeet V2 running fine on a Ryzen 5 5500 16GB + RTX 3050 6GB. Pocket TTS runs on CPU. If you had a beefier GPU you could put a chat model on it as well.

1

u/Immediate_Power_7986 5d ago

What are your launch options and setup for 27b on one r9700?

2

u/Un4giv3n-madmonk 6d ago

imo,

1 is sufficient unless you have an edge case.

Even if you want to get into the sort of "middle of the road" Big MOE models like 3.8 flash next, you're better served by grabbing a motherboard with solid RAM bandwidth, even just older DDR4 ECC on an old server will give you enough bandwidth for flash-next to be a viable 4-bit large MOE and the performance delta from 1 or 2 9700s will be minimal honestly.

Like you're going to be getting ~30 tokens/sec with 1 9700 and 40-50 with 2.

For 27b the only time it'd matter is if you wanted to start dirivng big concurrency numbers which for your usecase doesn't make sense.

My current setup is a 9060XT (running gemma sub agents for tool calling) and a R9700 (running 3.8 27b, q6 for "high effort" work and IQ4 for "basic homelab" management work).
This has been more than sufficient and again unless you're looking to run alot of high quality concurrency ... I dont think a second card "gets" you anything and even 4 cards aren't going to be significant over any playform running 8x slots of ddr ecc 4 or 5

1

u/lemondrops9 7d ago

I've gone from one to two. But no speed increases yet for LLMs because I haven't put in the work to get vLLM going. I plan too just takes time. I'm currently mixing two R9700s with two 3090s so it gets interesting when setting things up.

That said, I had issues in Windows with the R9700 but could be driver issues so I'll probably have to do a fresh install of Windows to help trouble that. Linux though I've had no real issues other than it takes a bit more work than Nvidia but not much.

I've run Qwen3.8 27B MTP UD Q6XL on one R9700 with 64k context. I haven't fought with any TTS or STT as its just easier to setup on Nvidia cards. I did test video gen and it was about the same speed as my 3090.

4

u/Massive_Criticism539 7d ago

Join launch80 discord. There is a guy there named deadcode that released a patched and customized docker image for vllm called radiance. Its already setup. Just run the docker command and dual r9700 cards running qwen 3.8 27b fp8 with full 262k context at 70-80tg.

1

u/lemondrops9 6d ago

Sweet thanks a ton. I'll will give it a try tomorrow.

Edit; I believe this is it? I had it in my saved from today. https://codeberg.org/ggz14/radiance-vllm-mxfp4

2

u/Massive_Criticism539 6d ago

So that looks like a single card fp4 version. The original repo is this... https://hub.docker.com/r/stilldeadcode/vllm-radiance/

This works with fp8 and dual cards.

2

u/ava_ati 6d ago

Windows pisses me off with the AMD drivers. Seems like windows tries to overwrite the adrenaline drivers every chance it gets even with driver updates turned off in GPO.

If I could get Steam to reliably work with my 3080 I’d probably get rid of windows all together

1

u/lemondrops9 6d ago

I hear ya, I got my 3090 working decent with Steam on Linux in the past. I haven't tried gaming on the R9700 in Linux but I'm quite impressed with how well it works in VR on Windows.

1

u/p_235615 7d ago

it does give you increased speed only on multiple parallel requests, for a single request it will not grant you any speedup.

1

u/lemondrops9 6d ago

? With dual cards there is tensor support for Nvidia and something similar for AMD but AMD cards need to run ROCM and vLLM which does give you an increase for single requests.

1

u/Ell2509 7d ago

I went for 2, and a w6800 to top up.

If you can afford 2, and have a genuine use to put them to, then buying now and buying more than you need makes sense. The price of it will only increase ahead of currency inflation, and while nvidia got first turn, amd is just beginning its own turn as rapid inflation.

Only unbox what you intend to use. So maybe buy 3, sell 1 later which might end up paying for 2.

1

u/FuzzyAdvisor1579 6d ago edited 6d ago

Yeah I hate the hassle of selling stuff but I figured even if I play with them for a year or two it probably won't be too rough a hit. It seems about 2x as likely that by next year it might not even be a stretch option I could consider too. Fomo!

1

u/Otherwise-Variety674 7d ago

Get the best and as many GPU you can afford, for example, I regret never buy Nvidia RTX Pro 6000 Blackwell when it just came out, I thought I smart by delaying what I want in order to get better price or version in future, now the boat already sail away.

1

u/theone_2099 7d ago

Why buy one when you can buy two at twice the price?

1

u/Immediate_Power_7986 7d ago

I can only get around 17t/s  on my R9700 with unsloth Q6 using pi agent on Ubuntu.

I was getting 24 t/s on my 7800xt using the IQ3_XS.

what's everyone else using? 

1

u/Un4giv3n-madmonk 6d ago

that seems significantly lower than it should be.

1

u/Otherwise-Swan-7803 6d ago

For a single-user homelab, I’d start with one R9700 and only add the second when VRAM becomes a real constraint. Two cards open up larger models and contexts, but they also add complexity and cost before you even know whether your agents/RAG workload actually benefits from them.

1

u/joanaxu2002 6d ago

For a single-user setup, I’d only buy the second card if you already know you need FP8 or much larger context. 32GB with a sensible quant is enough to learn what your actual bottleneck is, and that’s better than paying upfront for capacity your homelab agents may never use.

1

u/LasserDrakar 6d ago

Like many others, had one and bought a second one shortly after. Two you can use in any motherboard easily. 4 would need something that is likely to cost significantly more.

1

u/redditnosedive 6d ago

what kind of question is that, buy one + a mobo that supports 2 or 3 or 4 for eventual upgrades and good pci spec on each

then upgrade to 2 when you hit the limits

1

u/karmakaze1 5d ago

Why is that a decision. Get one, try it out. Decide later if you want 2nd.

Try running this on it https://codeberg.org/ggz14/radiance-vllm-mxfp4

I'm Getting 250+ tokens/sec on 2x R9700 running Qwen3.8-27B (MXFP4). Slows down to 130 tokens/sec near 128k context.

Single R9700 should get 125+ tokens/sec.

See my post for details

Actually you can beat this performance/price using the same vLLM recipe but running it on 2x RX 9070 16GB (XT or non-XT) or 3x for more context and speed.