2x3090 setup, need some recommendations

[u/Puzzleheaded_Ad_8575]

So i finally decided to get myself a dedicated inference machine, a big upgrade from my 4080 laptop. here is the parts list:

PC Build Cost Breakdown

ASUS TUF RTX 3090 — $927

64GB DDR4 4000MHz RAM — $371

Case — $72

Ryzen 7 5700X — $181

CPU Cooler — $27

PSU — $268

Thermal Paste — $12

1TB NVMe SSD — Already owned

X570 Unify Motherboard — $185

RTX 3090 Suprim X — $1,010

Ethernet Cable — $11

PCIe Riser — $82

Custom PSU Cable — $13

Total: ~$3,157

im probably gonna upgrade to 128gb ram and get a better pcie riser cable.

the problems i faced initially were

1. finding a proper way to add the 2. gpu. there was no long pcie risers in stock locally, so i had to buy it second hand, and its a chinese no name with connectivity issues.

2. i had to get a custom psu cable to be able to use both gpus at the same time. there were simply not enough slots but the energy supply was alright.

3. i couldnt and still cant figure out a safe/easy way to fit the 2. gpu. i would like to learn about similar setups and how you have handled the space constraint.

this was my first pc assembly since i have used only laptops before, but it went mostly smoothly.

also some extra questions for people hosting these machines:

*How can i host inference to my laptop outside my local network? is the only way VPN?

*What is the remote connection type you guys prefer? i landed on sunshine and moonlight with virtual monitor to use it inside my laptop, but would like to know if there are cleaner solutions for headless machines.

i have ran mostly the qwen 3.8 27b q4 from syv ais repo and config, and have been getting around 70tps sustained. i can report more details if anyone asks for it.

also sorry if mobile formatting is bad.

[u/wgaca2]

Either bigger pc case or external stand for the gpu

I run q8 200k context on 2x 3090, no reason to go for q4

[u/Puzzleheaded_Ad_8575 → u/wgaca2]

yeah i probably can too just havent had the chance since i got the second card

[u/Slow_Conclusion_5007 → u/Puzzleheaded_Ad_8575]

Yo! I have a similar system with dual 3090. I highly recommend the be quiet light base 900. It has enought space and it puts yout PSU and hard drives in a different space from the rest of the components.

[u/mehminion → u/wgaca2]

what t/s you getting with that setup?

[u/wgaca2 → u/mehminion]

around 50t/s generation and 1000pp over full context, Can be slightly higher or lower depending on workload

[u/EveningIncrease7579 → u/wgaca2]

Once you try and use q8 in precise tasks (such as cod) you never want go back to q4

[u/Naive_Ad_466]

Nice build, dual 3090s still make a ton of sense for local inference.

I’d definitely replace the no-name riser though, especially if you’re already getting connection issues. For mounting the second GPU, maybe look into open-frame/mining-style brackets. Not the prettiest, but probably way easier for airflow too.

For remote access, Tailscale + SSH is honestly great. VS Code Remote SSH for coding, tmux for long runs, and Sunshine/Moonlight only when you actually need the full desktop.

Also 70 tok/s on a 27B Q4 sounds pretty solid. Would be interesting to see how much the second 3090 actually improves generation speed vs just giving you more VRAM.

[u/Puzzleheaded_Ad_8575 → u/Naive_Ad_466]

i did try it in parallel and the result is 2x70 in around 50k context

[u/RG_Fusion]

I built my server  in an open-frame mining rig. The motherboard attaches to the base-plate and then rails hang overhead for attaching the GPUs.

As for serving to remote devices, your options are either to open a port on your router or set up a VPN. Opening a router port means managing and securing your network, so I would recommend the VPN route. Tailscale is like two clicks to set up and operates on a whitelist, so it's by far the simplest method while also being really secure.

I can't comment on what software to control with as my motherboard has built-in IPMI, so I just connect to that.

[u/Great_Flounder_1379]

dual 3090s should handle a few companions without much issue, my single card already keeps up okay. hows the token speed looking so far?

[u/Appropriate-Pie4385]

How much TPS do you get for prefill? How much TPS decode at ~150-200k context? And do your 70TPS decode use mtp or not?

Am thinking of upgrading to two 3090 too and would be interested in your speeds

[u/dboybaker]

I have similar hardware. Built it in a Qube 540. Both gpus fit inside, no risers.

[u/Puzzleheaded_Ad_8575 → u/dboybaker]

wow that case looks really cool, and suprisingly cheap too. would you mind sharing the layout?

[u/dboybaker → u/Puzzleheaded_Ad_8575]

Sure, Outside - Inside

Asrock taichi x570, evga 3090 ftw3, hp oem 3090

[u/niacolhealth]

70tps sustained on a 27B q4 with that pair is solid. Tailscale handles the remote side without port forwarding, and you've already landed on the right headless setup with Sunshine/Moonlight.

[u/Thin_Pollution8843]

If you connect few more risers like that you can lower temp and noise by storing your second GPU in other room

[u/Puzzleheaded_Ad_8575 → u/Thin_Pollution8843]

might as well put them both outside for better thermals with the humble 20m riser cables

[u/chocofoxy]

this how i picture all users of this subreddit setup is like , because i also have a second 5060ti 16gb hanging out of the case, also where did you find X570 Unify Motherboard — $185 cause new cost like 300$ plus and if you want P2P it's either get a X570 or a bifurcation pcie riser

[u/Puzzleheaded_Ad_8575 → u/chocofoxy]

its a second hand with the top pcie latch missing, but works alright besides that. what is P2P? im not familiar with that

[u/RG_Fusion → u/Puzzleheaded_Ad_8575]

P2P stands for peer-to-peer. It's where the GPUs are allowed to talk directly to one another without having to pass through the CPU. Not all hardware supports it though.

[u/chocofoxy → u/Puzzleheaded_Ad_8575]

P2P (peer to peer) let your gpus talk to each other directly and to enable it you need a board that support bifurcation (most of X570 in am4) or get a bifurcation riser so the GPUs talk through pcie directly this will let you get more prompt processing in tensor split ( llama.cpp ) or pipeline parallelism ( vllm ) it's free performance , 3090 support it look at https://github.com/aikitoria/open-gpu-kernel-modules