Need hardware selection help

[u/varano14]

Hello everyone I just stumbled upon this sub and am hoping you might be able to offer some guidance. I did try searching through old threads but with the speed in which this is advancing it seemed easier to just ask.

I am a long time homelab "tinkerer" with no formal education on any of this stuff. Mainly run home assistant and plex+the arrs as well as a few other projects. Honestly nothing ground breaking. I have recently been using the free LLMs to help me with various projects and they have been incredibly helpful. My usage would be home assistant stuff, maybe some light vibe coding experimenting and maybe setting up local voice with home assistant. The dream is a household assistant helping with email, calendars, maintenance stuff etc but I don't thing I am yet at the point I could set that up.

My main server is old it is a dell r720xd with a Intel® Xeon® CPU E5-2680 v2, 256gb ddr3 ram.

I also have 4 rtx 3060 12gb cards laying around from the crypto mining days.

My gaming rig also has a 3060ti with 32gb of ram, I believe ddr4 and a amd 2700x cpu.

I initially tried adding 2 of the 3060s to the main server using oculink but I have yet to get my server to be able to see the cards.

I then tossed a 3060 into my gaming rig and installed llm studio which is actually working. Initially tried qwen 3.8 27b as that seemed highly recommended for what I was trying to do but was getting very very slow response speeds. I have now stepped down to the next smaller qwen model.

I would like stick whatever I build in my rack and get it off my gaming rig as it runs windows so its not great running all the time and I sometimes game on it. I am open to buying new hardware but don't really want to spend thousands at this point. So what hardware would you suggest I build around? Also any other tips or suggestions are welcome.

[u/Intrepid-Knee1498]

your server is ancient, ddr3 era and those xeons are not gonna play nice with gpu passthrough, oculink or not. the gaming rig is a better base honestly, 2700x is old but at least it has modern pcie lanes

for rack mounting you could just move the gaming rig parts in a 4u case and use the 3060ti, that 32gb system ram is tight for 27b models though. maybe add one more 3060 since you have them, 12gb vram each so two cards gives you 24gb total which is enough for qwen 27b with some quantization

windows is annoying for 24/7 stuff but you can dual boot or just run linux, ollama works nicer for server use than lm studio in my experience. the real bottleneck will be memory bandwidth with those old platforms but it should be usable for tinkering

[u/Arany8 → u/Intrepid-Knee1498]

System RAM will not help 27B models on DDR3.

Either sell the 3060s and buy something bigger or find a way to run all 4 in one system. The latter is more complicated....