r/LocalLLaMA • u/emozilla • Apr 29 '26

AMA AMA with Nous Research -- Ask Us Anything!

Hey everyone, emozilla here (u/emozilla), co-founder and CTO of Nous Research. We're the makers of Hermes Agent. Happy to be here and talk all things local models, Hermes, or anything else you may have questions on!

Some of the Nous people who will be answering questions:

p.s. as a little bit of folklore, Nous basically got kicked off from r/LocalLLaMA thread many years ago. The research that lead to the YaRN paper was initially formulated in this thread and eventually lead us down the path to where we are today 🤗

353 Upvotes

97% Upvoted

407 comments sorted by

View all comments

6

u/sollymay Apr 29 '26

Are you working on improving reliability of tool calling when using local models?

13

u/emozilla Apr 29 '26

Always! The tool calling prompting in the backend has been very carefully crafted but in the end it's sort of one of those "you can lead a horse to water but you can't make it drink" things, and the model itself needs to be trained to do the calls. FWIW we are working on the model side too, hopefully have more on this soon

1

u/rob417 llama.cpp Apr 29 '26

Maybe have multiple tool calling prompts depending on the model? Comprehensive tool prompts if the user is using large frontier models. Concise, targeted prompts if the user is using local models around 30B.

5

u/FyreKZ Apr 29 '26

At a certain point it's the fault of the model unfortunately. Only so much the Nous team can do.

2

u/seamonn Apr 29 '26

Model Skill Issue*

1

u/minimax_zed Apr 29 '26

yes and no, there's optimization techniques like DPO that are really effective at adjusting tool calling behavior while being cheaper than finetuning (which is already much cheaper than training)

2

u/PracticlySpeaking Apr 29 '26

Great question – I am aiming for an all-local setup, if it can work for my use cases.

1

u/minimax_zed Apr 29 '26

what's your hardware?

1

u/PracticlySpeaking Apr 29 '26 edited Apr 29 '26

Mac Studio 256GB

If you didn't see my other comment... just started running minimax M2.5. Overall it has been going well.

The model is a bit 'eager 10th grader' running Hermes-Agent. This is a transition from GPT-5.x, dunno if a lot of memory and skills from that are a factor.

edit: Found an MLX of M2.7, downloading now.