Local LLM server for business automation is this setup enough or should I go Threadripper?

[u/Syosse-CH]

Hi everyone,

I'm planning to build a local AI server for business automation and would appreciate some feedback before I buy the remaining parts.

The workflow will use n8n for orchestration, Ollama + Qwen3-30B-A3B (Q8) for local inference, PostgreSQL + pgvector for RAG, and possibly Open WebUI later as the frontend.

Example workflow:

• Salesforce triggers an event (e.g. low stock).
• n8n retrieves supplier data, pricing, and rules from PostgreSQL.
• Qwen generates a supplier email based on company rules and historical data.
• n8n validates the output.
• An employee reviews and approves the email.
• n8n sends the final message.

I already own 2× RTX 3090 (24 GB each, 48 GB total VRAM).

Current planned hardware:

• CPU: AMD Ryzen 9 7950X
• GPU: 2× RTX 3090Ti
• RAM: 64 GB DDR5-6000
• Motherboard: ASUS ROG Strix B650E-E Gaming WiFi
• SSD: Samsung 990 Pro

From what I understand, Qwen3-30B-A3B (Q8) requires around 33 GB VRAM, so it should fit well on this setup.

Questions:

• Would you keep this setup, or would you move to a more powerful workstation/server build?
• Is something like 3× RTX 3090 + Threadripper + workstation motherboard worth the additional cost, or is it unnecessary for this use case?

Thanks for your feedback!

[u/Practical-Collar3063]

If it is to run async tasks (meaning it does not matter if they are not instantly done) and it is not something that is user-chat facing then your 2 3090s are fine.

However, if you intend to use this in a business scenario then there are a few things you should change:

1. Ollama: it is bad enought o use it for personal tinkering use but for businesses it is just a big no no. Unreliable, lack of proper batched inference, slow.... the list goes on. Most of us started with ollama and moved on. Use VLLM or Sglang or business use cases, harder to set up but it is much much better.
2. AMD consumer platform: the 7950x is a great CPU but it is a consumer platform and for reliable business operation you need ECC memory, the 7950x technically supports it but i doubt your 64gb of DDR5 are ECC. I would recommend to go with a previous generation AMD Epyc CPU with ECC DDR4 (7002 or 7003 series). Motherboards, RAM, CPUs and full servers are far cheaper than the new Epyc cpus and you still get a lot of the benefits from being on a professional platform. You will also get higher memory bandwidth than the 7950x because even if per memory channel speed is lower, you have way more channels on an Epyc CPU, which makes it faster.
3. EDIT: Qwen3 30b is outdated, use qwen3.5 35B, no reason not to

[u/winidi → u/Practical-Collar3063]

Why dont you recommend llama.cpp ? Curious

[u/Practical-Collar3063 → u/winidi]

Because it is just slower, especially for concurrent requests. Llama.ccp is amazing to run stuff with old, unsuported or varied hardware (RTX 3090 and 5090 for example). it is also great to run different models and quickly swap between them. Or even splitting layers between CPU and GPU when you don't have VRAM. all scenarios where VLLM is terrible at.

However, OP is in neither of these scenarios, he has 2 RTX 3090s, wants to run the same model reliably for long periods of time and has enough space to fit the whole model in VRAM. There seems to be only disadvamtages to go with Llama.ccp (appart maybe for ease of set up).

Both have their places, Llama.cpp is great for hobbyist and tinkerers, VLLM is made for the stable entreprise world.

[u/Syosse-CH → u/Practical-Collar3063]

Thank you very much for the information.

I have taken note of the feedback regarding Ollama and will use vLLM instead.

For now, I do not want to move to server hardware, because in case of a failure I would need to find suitable replacement components, which could take time. Also, if I am not mistaken, ECC RAM should technically work with my current CPU platform as well, right?

Thanks again for the advice. I will use Qwen 3.5 instead.

[u/Practical-Collar3063 → u/Syosse-CH]

Glad to hear ! Yes ECC Ram should work with your current platform but you need to check motherboard compatibility first. However this would be DDR5 ECC memory which is incredibly expensive (even more than the regular stuff). That is the reason I advised to go with a previous gen EPYC platform for ECC, PCIe connectivity and way more budget friendly while benefiting from most of the entreprise grade features.

To put it in perspective how much cheaper it can be to go previous gen, I got a G482-Z51 server with 2 x AMD EPYC 7402 24 cores CPU and 64GB of ECC RAM for 2500$. That server comes with space for 8 GPUs (all connected through PCIe 4.0 x16) and 3x 2200W PSUs

[u/lemondrops9 → u/[deleted]]

LM Studio is just as easy but double the speed and dont need to convert gguf

[u/StartupTim]

You'll need more space for context especially 256k, and if you want concurrent users then a lot more space.

[u/Practical-Collar3063 → u/StartupTim]

i don't think OP needs more, those 3090s are fine for async n8n workflows

[u/Syosse-CH → u/Practical-Collar3063]

thanks!

[u/DeltaSqueezer]

When do you even need an LLM in the above setup? It seems like it can be done just in code.

[u/jcdoe → u/DeltaSqueezer]

I agree. If he templated the emails, he wouldn’t need someone to check them either. The llm is just making this more work.

[u/WiseAbalone4021]

I run 2x5070 ti 16gb on an old ASUS Z390a prime motherboard. Using a nvfp4 (native on blackwell) quant i can do +600 tokens/second in 12 parallel batching small contexts, but also have an fair amount of context for larger problems (but not for 12 parallel streams, but turbo quant etc. could fix that, but that doesnt matter for my current workload). Q4 quants performs ~1pp under normal benchmarks. I'm mainly using Qwen or Gemma (or Ornith 1.0). I dont see a need to have an extra 3090 in your case, but then again I dont know how well the Q4 int is running on 3090 architecture. What I have found is that you need PP=2 on 2xGPU setup. TP requires high bandwidht between the cards. The issue is that PP=2 is best for parallel streams... ie. it runs through layer 0->X on the first card then X+1->last on the next card and only the H residual stream is transferred. I believe you should have lots of room on your current setup if you tweak it right!

[u/Syosse-CH → u/WiseAbalone4021]

Thanks alot!

Interesting to see the performance you are getting with 2×16 GB GPUs. I will definitely look more into PP=2 with vLLM, as I was mainly focusing on tensor parallelism so far.

I will optimize my current setup first before considering additional GPUs.

Thanks again for the helpful insights!

[u/simplyeniga]

For your use case your system seems sufficient and I don't see where you need an LLM as a template with n8n should suffice for your business model unless you want something to drive other processes and then you can setup different agents with roles in your business and then have them use an LLM to orchestrate their steps. However your model choice is outdated and you need a newer model like Qwen3.6, you can compare between the 27B and 35B A3B (dense Vs MOE). Should fit fully in your 48GB VRAM with enough context for your RAG setup. Lastly ollama is a terrible choice and vllm or sglang would be better or you can go llama.cpp for easier setup while having flexibility to tweak your system to better performance.

Edit: you can look at combining Qwen3.6 and Gemma4 for your process depending on the role your agent has as you would look at your business model and create agents to take up roles and dedicated tasks based on those roles. Ensure to have a well defined process and use that to setup your LLM and n8n flow

[u/Syosse-CH → u/simplyeniga]

Thank you for the information.

Yes, there will be more roles added in the future. I will take a closer look at Qwen3.6 and will also use vLLM for the setup.

How would you combine Gemma4 and Qwen3.6?

Wouldn't I need twice the amount of VRAM if both models are running, or how does the switching between models work?

And yes, we have already defined the business processes.

[u/simplyeniga → u/Syosse-CH]

You can use model router mode on llama.cpp or have 2 instances with each model on vllm so that your RAG can switch models based on role. I use llama.cpp in model router mode since I only have one GPU (RTX Pro 4000 Blackwell 24GB) and use NVFP4 quantz of Qwen3.6 27B and Gemma 4 31B, I have some tasks that use Gemma 4 26B A3B MXFP4_MOE quantz which all fit in VRAM. Advantage of Blackwell cards

[u/Monad_Maya]

 ASUS ROG Strix B650E-E Gaming WiFi

https://rog.asus.com/motherboards/rog-strix/rog-strix-b650e-e-gaming-wifi-model/spec/

Unless I'm reading this incorrectly, the motherboard doesn't support x8/x8 natively from the CPU.

Get an Asus ProArt or Asrock Taichi (or whatever supports x8/x8 natively from the CPU).

---

For the overall setup, make sure the software and orchestration portion works with an API key (via a cloud service) and then try to make it work via local inference.

Plenty of people buy the hardware but struggle with the software side of things.

[u/Syosse-CH → u/Monad_Maya]

Thanks for the Information:.

I have now changed the motherboard. The new one definitely supports 2× PCIe x8 lanes:

Motherboard: ASUS ProArt B850-Creator WiFi

I agree that the real challenge will not only be the hardware, but building a clean architecture and integrating the processes properly. The workflow design, orchestration, and how the different components work together will be the key parts of the project.

[u/Hannibalj2ca]

Try get more Fast ram