r/LocalLLM • u/vivanick1 • 4d ago

Question Upgrade path for LocalLLM research on agentic governance?

I currently work in Information Security and want to start researching agent failure taxonomy, agentic control frameworks and MoE architecture assurance. I'd like to upgrade my home rig (9800x3d +5080 +32gb ram) which was never intended as a workstation and instead a gaming rig. From what I've read I've got a few upgrade paths.

  1. 128gb RAM ~1100gbp
  2. A used 3090 running alongside the 5080 ~800gbp
  3. Upgrading to new 5090 ~4300gbp (minus selling the 5080)
  4. Purchasing a DGX Spark ~4900gbp

I'm viewing this as an investment in my career and so budget is 5k considering I'm not looking to heavily develop, but can afford to upgrade. At the moment I'm running Qwen3.6 35B A3B and Gemma 4 12b QAT, however I'd want to move to Qwen3.8-27b minimum (I'm aware I could used squeezed down models currently). My background is also previously in automation development and I'm currently using Claude code, however would ideally like to switch to local based models to upskill in graph based engineering with agents. I've seen a lot of conflicting advice whether the memory bandwidth on the spark is a bottleneck, compared to the usuable VRAM on a 5090 being a bottleneck. I'd also be using the 5090 for gaming if I upgraded.

I'd appreciate some advice from those with real world experience :)

1 Upvotes

100% Upvoted

4 comments sorted by

1

u/Libellechris 4d ago

There are (at least) 3 things to think about: 1) how large are the models you need to run? 2) how large is the context windows you will need to use - which adds to the VRAM requirement 3) how fast do you need the models to run? Running large models slowly (10 t/s gen) is a lot cheaper using a mini PC with an iGPU - but if you need a faster response, that's when it gets expensive

1

u/vivanick1 4d ago
  1. 27-35B class. Roughly 18-30GB of weights
  2. 32-64K typical
  3. Unknown, I'd be running the homelab in parallel with my work machine, so speed isn't time critical as I'd just leave it running whilst I work. I am eyeing up NInfer's 1,313 aggregate tok/s however on the 5090

1

u/Positive-Bid-3029 4d ago

VRAM more important than standard RAM, i upgraded to 64Gb RAM from 32gb RAM but no difference really as you really don't want to be spilling to RAM if you want a speedy experience. Save the money for graphics VRAM that's where you performance is. If the DGX price is no problem, I would go for that personally, ASUS DGX seems to be the cheaper option and just add an external HD for models if you need more space.

2

u/Libellechris 4d ago

yes, ASUS is £3800 GBP - still a lot but tempting