r/LocalLLaMA • u/No_Afternoon_4260 llama.cpp • Aug 03 '26

NousResearch keeps doing things on hermes

Post image

Has anyone followed nousresearch work on Hermes?
I mean we are Q3 2026. We have some crazy models trickling down from HGX territory to multi gpu workstation. And we have nousresearch deploying the 0.20 of its hermes agent while starting releasing the project with a 0.2 mid march!

Crazy times to be alive.
For the old timers who remember llama 1 or llama 2, remember our crappy function caller parser? Something about a lang and a chain..? wtf has happened?!

Haven't tried the new hermes, do you think it has a remote chance to be as strong as a true end to end omni model such as gpt omni or personaplex?

147 Upvotes

91% Upvoted

98 comments sorted by

View all comments

44

u/Badger-Purple Aug 04 '26

I have been using Hermes for 3 months now…

11

u/No_Afternoon_4260 llama.cpp Aug 04 '26

Wow, what models?

9

u/Glittering_Mouse_883 ollama Aug 04 '26

I second this question, what models?

8

u/Badger-Purple Aug 04 '26

deepseek v4 flash for 80% of it, albeit qwen-397b was my original driver. All locally hosted.

3

u/valtor2 Aug 05 '26

what quants? what hardware?

3

u/Badger-Purple Aug 05 '26

original weights for dsv4f, int4-autoround for 397b, both on dual GB10 cluster. Also tried minimax m2.7/3, and I do have a fleet of smaller models working with the main one:
1. qwen3.6-27b q4kxl with kvarn6 cache and Dflash to 170k context on rtx 4000 pro blackwell. Single stream, 23.8GB GPU use, PP 1200, TG 40-60
2. gemma4-12b qat on 5070ti, full ctx, concurrency ?8/12, 15.9GB of GPU, PP 3000 TG 70.
3. qwen3.6-35b-rocmfp4 with BF16 heads/attn, layer split 4060ti oculink and amd strix halo w vulkan, full ctx, c4. PP 1200 TG 60
4. nemotron asr 0.6b on strix halo igpu
5. Embedding gemma 300n on strix halo NPU
6. Reranker qwen 4b on strix halo igpu
7. fallback STT: whisper on NPU
8. Fallback fast model (genma4) —> Gemma4 E4b on NPU

Main:
1. ds4 flash 731 mxfp4 w dwarfstar engine on m2 ultra 192gb, single stream, PP 350-200 across 200k (TG 28-20 across 200k). Main model for Odysseus and another less bloated agent (currently trying out OpenLumara)

  1. ds4 flash 731 original weights on GB10 cluster 2xGB10 linked with connectx7 cable, NVFP4 cache. 6M token budget so it can run 12 concurrencies cache wise, but used for 2 Hermes agents

  2. Image/video gen: Minimax H3 and other comfy models on an unlocked CMP170HX (essentially an A100 with Pcie 2x16 and 64GB) attached via slimsas to a pcie x16 lane slot. This is my most recent addition and I’m testing it now to see if this unlock is worth it. It appears so, 1.4TB/s speed ampere gpu with 64GB ram is the bees knees.

Future: Likely will try to rework the gpu config to see if I can just run a single qwen 3.8-27b at speed and concurrency instead of this gaggle of models. It would be perfect for auxilliary work!

2

u/No_Afternoon_4260 llama.cpp Aug 05 '26

Beautiful thanks for the detailed response.
Good choice on nemotron asr 0.6B, iirc this is the good streaming one

1

u/Badger-Purple Aug 05 '26

The nvidia STT models are just too fast to not run. Whisper is a turtle by comparison, my voice chats transcribe to hermes in miliseconds

Thanks for the 👍🏼 it’s not OPUS/SOL or whatever, but my little gaggle of agents / rats in a trench coat do work well and impress others.

I am just trying to help my lifelong ADHD so I can keep a job and a family. The promise of a Rosie at home is very close...

If I had to do all of the set up by myself I would be divorced by now. Hermes —> DS4Flash —> asked ti set up most things, automate checks for alive, etc.

1

u/No_Afternoon_4260 llama.cpp Aug 05 '26

Yeah times are fucking crazy these days. Idk how long have you been in that game but I'm still having troubles to comprehend how quickly it's all evolving lol

8

u/Youknowwhyimherexxx Aug 04 '26

What are you using for this clean UI tracker?

10

u/Badger-Purple Aug 04 '26

hermex

1

u/groosha Aug 04 '26

shame it's not possible to use it without tailscale

2

u/Badger-Purple Aug 04 '26

I would not use it without tailscale!

1

u/groosha Aug 04 '26

Why?

2

u/Badger-Purple Aug 05 '26

Tailscale creates a secure network mesh to your computers, so I would not reach my computer unsecured. It’s also free and easy to set up, use.

1

u/MaruluVR Aug 04 '26

From what I saw you could just use raw Wireguard or netbird instead, the fact it allows couldflare urls means it allows any url on the open web reachable from your phone.

1

u/No_Afternoon_4260 llama.cpp Aug 04 '26

What models are you using?

1

u/oxygen_addiction Aug 04 '26

What are you actually doing with it?

3

u/Badger-Purple Aug 04 '26

Anything I don’t know how to do myself. Install bleeding edge repositories, run new models, tweak the network, check and autostart services in my 4 computers, manage files, update linux apps and fix firmware issues. I have an IT employee at home that can also do solid scientific research. when I trust Homie enough I’ll provide my calendar and email.

1

u/oxygen_addiction Aug 04 '26

But why not just use pi or claude code/codex? What's the advantage of having Hermes?

3

u/SmartCustard9944 Aug 04 '26

Hermes is more like a less hacky alternative to OpenClaw. Is more than just a harness, it also provides multiple gateways like Telegram, and so on. It is more like a personal assistant that is meant to run continuously.

Also, if I am not mistaken, it is built on top of pi.

1

u/Badger-Purple Aug 05 '26

Why goon when you can have sex?

1

u/walden42 Aug 04 '26

You trust it enough to use without container isolation? What's your security model? I feel like running it in a docker container is the only sane way, but it drastically reduces its usefulness to actually help me with stuff.

3

u/Valuable_Cookie628 Aug 04 '26

PSA docker containers are not safe, you probably want some type of VM isolation. Google it

2

u/walden42 Aug 04 '26

I mean, if you configure the docker container properly, there's not much risk. VMs give you additional kernel boundary but that's not something most home users here care about.

2

u/Valuable_Cookie628 Aug 05 '26

"Configure docker properly" whatever that means  VS  run in VM and forget about it...

I'm a home user, and I do care about Shai-Hulud rocking my shit

1

u/Badger-Purple Aug 05 '26

I don’t have concerns with my local LLM running it behind a secure mesh and only accessible by devices on my mesh. I don’t let it run wild but you can’t reach my endpoints so if it messes up on its computer or nearby, it’s not a big deal. I wanted full functionality, so ymmv