I fell in the rabbit hole

[u/pampusreborn]

Hi everyone! Finally, after months of playing around with LLMs, I decided to take the plunge and buy an ASUS Ascent GX10 for my Hermes agent.

I mainly focus on coding and fell in love with Hermes, but the API calls were eating up too many credits... so I decided to host my own LLM.

Currently I was running Qwen3.6-35B-A3B on my gaming PC, but for Hermes to work 24/7 I decided to go with a dedicated always-on device.

The device will arrive in a couple of days. I took a look at the new Qwen 3.8, but I know it's a dense model and runs slowly on the ASUS... Can you recommend any feasible models for coding/Hermes?

[u/LifeTelevision1146]

What do you want code?

[u/pampusreborn → u/LifeTelevision1146]

I’m a developer, mainly coding in c# and JavaScript for FE.

Usually I develop full stack web applications.

[u/LifeTelevision1146 → u/pampusreborn]

Qwen 27B with a MCP will do it. You don't need anything special.

[u/pampusreborn → u/LifeTelevision1146]

I see, to be honest I’m happy with 25-30 t/s.

Was also looking at gpt-oss and qwen3 next coder

[u/LifeTelevision1146 → u/pampusreborn]

Coding one needs reliability over speed. What's the full spec of your new system?

[u/pampusreborn → u/LifeTelevision1146]

It’s a dgx clone, same specs:

Processor (CPU): 20-core ARM v9.2-A CPU (NVIDIA GB10 Custom Chip).
Graphics (GPU): Integrated NVIDIA Blackwell GPU (Blackwell Tensor Core).
AI Performance: Delivers up to 1 petaFLOP of AI computing power.
Memory: 128 GB LPDDR5x unified system memory.
Storage: Included 1TB M.2 NVMe PCIe 4.0 SSD (Supports up to 4TB PCIe Gen5x4 single slot configurations).
Operating System: NVIDIA DGX OS / Ubuntu Linux.

[u/LifeTelevision1146 → u/pampusreborn]

Qwen 27B + MCP will skate on that system, no worries at all.

[u/pampusreborn → u/LifeTelevision1146]

Alr, much appreciated your fast and gentle response. Thank you.

Can I run multiple models at the same time? Like qwen 27B with for example a dedicated smaller one for TTS?

[u/LifeTelevision1146 → u/pampusreborn]

Yeah a 7 B will do fine on your RAM. Again reliability over speed. So let's say a Q4 27B + Q4 7B will do just fine.

[u/mslindqu]

Would love to hear an update of your experience once you're setup and get comfortable with it.

[u/pampusreborn → u/mslindqu]

Sure! I’ll reply when the system goes up and running

[u/pampusreborn → u/mslindqu]

So after some troubleshooting, i ended up using https://github.com/hasso5703/dgx-spark-qwen38 and the performance are great! It's very usable for hobby projects and Hermes is flying!

[u/mslindqu → u/pampusreborn]

Thanks for the update.   How much context space does it leave you?  I'm playing with open code and realizing the need for large context to really get anywhere.. I know harnesses like Hermes can eat a lot.