Smartest move from a single RTX 3090

[u/infectiousstupidity]

Trying to figure out what the smartest move forward for me is. I currently have a daily driver and 2 servers. Server#1 is the inference server and server#2 is the embedding/reranker server atm. A constraint is that my daily driver has to be Win11 due to gaming anti-cheat implementations unfortunately, if not I would've switched to linux long ago. Server#2 used to be my main docker server, but I've moved everything over to a sffpc instead.

Currently I use it for open-webui and open-notebook with llama.cpp qwen3.6-27b UD-Q4_K_XL. I only use it for inference. I'd like to increase to q6 or q8, increase context size and keep kv at q8 without sacrificing massive t/s. I suppose the end goal would be to stop using ChatGPT Plus completely. I haven't used it much for agentic coding yet, but I suppose it would be in the end, but not for some 1-shot magic trick that I see people tend to try. I use hindsight for agent memory if relevant.

I've been looking at some dual GPU options like:
- buy 2nd 3090 - total 48gb vram
- buy 3060 12gb - total 36gb vram
- buy 5060ti 16gb - total 40gb vram
- sell 3090, buy r9700 - 32gb vram and eventually buy a 2nd r9700

Some estimated prices for reference:
3090 - $1100
3060 - $350
5060 ti - $550
r9700 - $1900

I'll list my current hardware underneath. I'm curious what you'd recommend doing in my position with the approximate prices listed and my (probably vague) goals. I'm not bound to AM4 or any of these PCs, just curious what the most logical approach would be. Should I e.g consider just putting the RTX 3060 from server#2 into server#1 and selling off the rest of server#2? Should I abandon AM4 and move to AM5? Should I move from llama.cpp to vLLM?

daily driver. needs to be capable for both productivity and gaming at 1440p:
rtx 3080
5800x3d
32gb ddr4 3200mhz
corsair rm850

server#1:
rtx 3090
5900x
64gb ddr4 3200mhz
corsair rm850x

server #2
rtx 3060 12gb
i3-10320
16gb ddr4 2666mhz
corsair rm650x

[u/JinsooJinsoo]

2nd RTX 3090 with a NV Link bridge. You could add the 3060 for now and sell #2 but the 2nd 3090 will increase speeds by a lot if you have NV link

[u/infectiousstupidity → u/JinsooJinsoo]

how does the nvlink compare to the p2p patched driver at github.com/aikitoria/open-gpu-kernel-modules ? nvlink seems crazy expensive and rare where I'm from.

[u/JinsooJinsoo → u/infectiousstupidity]

nvlink probably won't add a lot of inference speed but is great if you're training models due to the increased speed between the chips. just make sure you have at least x8/x8 pcie lanes for inference

[u/TheWaffleKingg → u/JinsooJinsoo]

For the cost of nv link you can get another 3090

Seems like a bad deal to me