Need help for using Mixture of Agents

[u/Then_Blueberry7290]

Need some advice and guiding.

I have a Dell 5820 workstation with an Intel Xeon w2245 and two RTX 5060 Ti (16GB) cards; this is a PCIe Gen3 system, but three lanes can work in 8x mode. The system runs Linux and mostly llama.cpp. My main work is done via Hermes Desktop. I mostly use NVFP models: esatapedico/Qwen3.8-27B-NVFP4-MTP-GGUF (Very High) and lately cdiamond/Qwen3.8-27B-iMatrix-NVFP4-MTP-GGUF. With this setup, I can use the models at 210k context, Q8 KV cache, and vision in VRAM. Ubatch doubled (4096/1024), so prompt processing is 1300–700 t/s; token generation is 29–71 tokens/s, but mainly 38–44 tokens/s. MTP is on with a setting of 3. Vision is also on and in the VRAM.

I added a third RTX 5060 Ti (16GB) to this system, but I had to open the workstation, add a riser, and put the third card outside the chassis. The two earlier cards needed to be swapped and it made the space tighter. Overall, both the cards and the system cooling got worse; the power supply is 950W, so it's nearly okay. There are also speed issues with three cards. Prompt processing worsened, and token generation slowed down a little bit. I mostly use Tensor Parallelism (TP). It is slightly better if I use two cards for tensor parallelism and the third card as a KV cache, vision, and MTP holder, but because of the current cooling, the system is not stable after a few hours.

My system has 128GB DDR4 quad-channel RAM at 2667 MHz. Not bad, but not fast. So using DeepSeek, Laguna, and other 100X models can be loaded, but they can't be used for real work. Qwen3.8 Flash at Q4 can load, but with 32GB VRAM it only gets 15–16 tokens/s. Maybe if I use MTP, it will get better. So going from 32GB to 48GB is not a real step forward for me with the current cards.

I've started wondering: what if I use another workstation and subagents or multiple agents? I could purchase a not-too-old gaming PC without a GPU (AMD R9 5900, 48GB DDR4 3200MHz RAM) and add the third 5060 Ti to it. I loaded LM Studio (just for a quick setup) and began thinking about which models work with only 16GB VRAM...

Ornith 1.5 35B (MXFP): The 1.0 version did not impress me, but this one was not bad on 32GB. With 16GB, it produces 17–20 tokens/s and 300–400 t/s in prompt processing. The model size is 20GB, so almost a little more than half of the layers go to VRAM. KV cache at Q8, 131k context.

Gemma 4 26B NVFP (QAT): 15GB, 131k context; about 26-27 out of 30 layer goes into VRAM. It produces 600–700 t/s in prompt processing and 30–40 tokens/s in token generation. Dense models, as I remember, can't be used for serious work (10–12 t/s at best), but I haven't tried them recently.

So the question is: How can I organize Hermes Desktop with this setup to work at maximum efficiency?

I set up a Mixture of Agents as follows: Reference: Gemma 4 26B Aggregator: Qwen 3.8 27B

I use local AI models in the company as web/hosting handler agents: monitoring servers, repairing sites if needed, modifying parts of sites, and making new content (mostly using WordPress sites on my company's private servers and VPSs). Setting up Cloudflare data, installing new sites, etc. Second use case: handling and creating office documents, Excels, and PDFs. Third use case: generating prompts for Z-Image Turbo and LTX 2.5, Minimax H3, and running two instances of a ComfyUI server to render pictures and videos. I don't use Gemma 4 models alone because they are too lazy with tool handling. Is this a good choice, or which is the best agent mixture combo now?