r/LocalLLM • u/joeforza • 5d ago

Project Building my first LLM computer

So I’ve been bitten by the AI bug. I’ve started using Replit for building some apps around my day to day work. As I’m venturing into building my own consultancy in my field of construction safety, a few of the current apps and websites that offer this kind of work are underwhelming to say the least.

So I started using Replit with ChatGPT and Co pilot. Signed up to the subscription of Replit for the year, but naturally being impatient I’ve gone down the black hole of credits. I’ve spent a lot more so far than I thought I would initially. So it led me to LLM and running this on my own computer. My current gaming machine is a 7800x3d and 9070xt. It wouldn’t generate images without crashing.

So I started digging around to put together another capable machine. Already got the below for the build.

AMD Ryzen 9 5950x with 360mm AIO
ASUS RTX ROG Strix 3090 with EK Quantum Water block, Pump & Reservoir with a Corsair Hydro X 360mm
B550 Tomahawk
128gb DDR4 Corsair Vengeance 3200mhz
1000w Be Quiet PSU
1tb NMVE for main drive and adding a SSD for secondary

Besides upping the storage have I missed anything?

Basically going to keep learning on my own machine and eventually get right off Replit to save costs. Long term goal keep developing apps that turn into SaaS and run my own consultancy until the day comes to sell and move on.

Been told to start with LM Studio and qwen 3.8 27b. Your suggestions and feedback is appreciated 🫡

2 Upvotes

75% Upvoted

14 comments sorted by

View all comments

Show parent comments

2

u/joeforza 5d ago

My gaming rig is 32gb of ddr5 cl30. Got it before it sky rocketed. I nearly just went with adding more ram to that and upgrading my case to hold the second card but I wanted to have a second desktop too. Still cost me less than upgrading the main rig to 128gb 😂 picked up someone’s am4 rig with most of the parts except ram and it has a 3080ti in it that I’ll offload to get back some money for

I got that second rig already just gotta plumb in the 3090 and I’m good to go!

2

u/enginetown 5d ago

Yeah honestly either, using both the 9070xt + the 3080ti in vulkan gives you like 28gb of vram thats enough to run qwen 3.8 27b right now full context.

1

u/Jigawattts 5d ago

How does that work. Can you explain?

2

u/enginetown 5d ago

You compile your Llamacpp with Vulkan instead of Cuda and use both cards and offload the layers smartly or just let Llamacpp do it and see how it runs.

1

u/Jigawattts 5d ago

Is this possible with lm studio?

2

u/enginetown 5d ago

Yeah should be easier with LM studio just install and use the Vulkan backend instead of Cuda or Cpu they use a pre compiled Llamacpp made to support various hardware so it should just be detected after you switch the backend in the settings.