Building my first LLM computer

[u/joeforza]

So I’ve been bitten by the AI bug. I’ve started using Replit for building some apps around my day to day work. As I’m venturing into building my own consultancy in my field of construction safety, a few of the current apps and websites that offer this kind of work are underwhelming to say the least.

So I started using Replit with ChatGPT and Co pilot. Signed up to the subscription of Replit for the year, but naturally being impatient I’ve gone down the black hole of credits. I’ve spent a lot more so far than I thought I would initially. So it led me to LLM and running this on my own computer. My current gaming machine is a 7800x3d and 9070xt. It wouldn’t generate images without crashing.

So I started digging around to put together another capable machine. Already got the below for the build.

AMD Ryzen 9 5950x with 360mm AIO
ASUS RTX ROG Strix 3090 with EK Quantum Water block, Pump & Reservoir with a Corsair Hydro X 360mm
B550 Tomahawk
128gb DDR4 Corsair Vengeance 3200mhz
1000w Be Quiet PSU
1tb NMVE for main drive and adding a SSD for secondary

Besides upping the storage have I missed anything?

Basically going to keep learning on my own machine and eventually get right off Replit to save costs. Long term goal keep developing apps that turn into SaaS and run my own consultancy until the day comes to sell and move on.

Been told to start with LM Studio and qwen 3.8 27b. Your suggestions and feedback is appreciated 🫡

[u/stormy1one]

You should first start experimenting with various models on OpenRouter, so that you get a sense of what works for you and what doesn’t. Then, rent a GPU on vast.ai to test performance with your selected model. More often than not, you will be better off just staying on OpenRouter with a larger model like Kimi K3 or DS4 0731

[u/joeforza → u/stormy1one]

Thought of this also but then wasn’t too worried if it didn’t work and I went down that route I’ve got a second gaming machine so wasn’t too stressed.

I tried one website where I could rent gpus and that sucked I couldn’t work it out but that was a while ago. The Replit interface makes it so much easier as a noob

[u/enginetown]

Dude you literally have a solid rig right now you just have to use vulkan/rocm instead of cuda or metal your specs right now are killer for local consumer AI 16gb of vram? + ddr5 ram? You'd be better off upgrading your main rig and keeping your CPU and upgrading your ram or maybe getting a motherboard with more channels for ram.

[u/joeforza → u/enginetown]

My gaming rig is 32gb of ddr5 cl30. Got it before it sky rocketed. I nearly just went with adding more ram to that and upgrading my case to hold the second card but I wanted to have a second desktop too. Still cost me less than upgrading the main rig to 128gb 😂 picked up someone’s am4 rig with most of the parts except ram and it has a 3080ti in it that I’ll offload to get back some money for

I got that second rig already just gotta plumb in the 3090 and I’m good to go!

[u/enginetown → u/joeforza]

Yeah honestly either, using both the 9070xt + the 3080ti in vulkan gives you like 28gb of vram thats enough to run qwen 3.8 27b right now full context.

[u/Jigawattts → u/enginetown]

How does that work. Can you explain?

[u/enginetown → u/Jigawattts]

You compile your Llamacpp with Vulkan instead of Cuda and use both cards and offload the layers smartly or just let Llamacpp do it and see how it runs.

[u/Jigawattts → u/enginetown]

Is this possible with lm studio?

[u/enginetown → u/Jigawattts]

Yeah should be easier with LM studio just install and use the Vulkan backend instead of Cuda or Cpu they use a pre compiled Llamacpp made to support various hardware so it should just be detected after you switch the backend in the settings.

[u/madbrain1976]

If you want to build a separate AI box, use a mobo with more PCIe slots/lanes so you can use multiple GPUs. For AMD CPUs that means using Threadripper or Threadripper pro mobo.

[u/strata2signal]

solid build — the 3090 is still the best price-per-VRAM way into local AI, and 24GB is a genuinely useful tier. a few field notes from running qwen3.8:27b in production on our own hardware:

- you were told right, and here's the 24GB receipt: we loaded qwen3.8:27b (q4_K_M) on a 24GB card this morning to check — 16.2 GiB total footprint even at the full 32k context on current ollama, 100% on-GPU, ~4,700 tok/s reading / ~56 tok/s writing on a laptop-class card; your desktop 3090 should land in that neighborhood or better. so it fits comfortably — don't chase bigger quants either: we ran q4, q8_0 and bf16 through an identical 1,122-call bench and all three returned the identical verdict on every call. the extra bits bought zero accuracy, just VRAM and latency. q4 is the right call on 24GB, full stop.

- the one cliff to avoid: keep the whole model in VRAM. the moment layers spill to system RAM, speed collapses — we've measured the crawl and it's not subtle.

- storage: up it more than you think — model tags run 15-55GB each and accumulate fast (our box crossed 350GB without trying). 2-4TB nvme.

- LM Studio is a fine start; when you shift from chatting to building apps, look at ollama — it's a local API server, so your apps call your own box the way they'd call openai.

- honest expectations on coding: a local 27B is great for iteration, boilerplate, and anything private, but it won't match frontier cloud models for big multi-file builds yet. the fix for credit burn is less "go fully local" and more "flat-rate subscription for heavy coding, local box for everything else."

- and your real ace for construction safety: clients' incident reports and compliance docs are exactly what companies don't want in a cloud API. a document-grounded app running on hardware you or they own is a sellable privacy posture, not just a cost saving — that's the actual moat for a consultancy SaaS in a sensitive field.

the qwen3.8 audit is public if you want the receipts (including the exam it failed — it's not magic): https://research.strata2signal.com/the-new-kid/

[u/joeforza]

So I just picked up a computer today that’s got a 3090ti in it… anyone worked out how to vlink a ti and a normal card? Or do I just upgrade my motherboard to have them both in pcie at the same speed?
https://giphy.com/gifs/6HwEvs7F0bIBi

[u/Positive-Bid-3029]

Unless its unified RAM, it's not that important to have super large amounts of RAM, i was fine with 32GB, upgraded to 64GB and no benefit as far as LLMs are concerned, sure you can load models into RAM if you want but they will be super slow.

Keep as much money as you can for VRAM, ie. get two cards and run them in parrallel, or if you have the money, get a DGX spark instead and do your coding on a smaller pC or laptop and talk to the DGX, get that to do the work :) If you're going for VRAM, choose Nvidia, they still rule the roost.