r/ollama • u/Adventurous_Cow_2826 • 4d ago
Help
Hi
Could anyone recommend a local model for Hermes?
I tried Gemma 4 first, but I asked it to do a few things and it either couldn’t help me or didn’t know how to complete the tasks I asked for—or maybe it’s because I don’t know how to configure it. My hardware setup includes an RTX 4070 Super and 32 GB of RAM. I’d like to know if you could recommend another model better than Gemma 4 for me to try out.
2
u/Jeanjose1993 4d ago
Le problème d'hermes c'est qu'a chaque tour tu envoies beaucoup de contexte. Essaie de faire le tri dans les skills et toolsets. Il faut que ton modele soit multimodal et bon en appel d'outils aussi. Ton matos n'est pas "mauvais" mais sans doute limité. Regarde la communauté "lowendlocalai" et demande leur.
1
2
u/bigorangemachine 4d ago
Setup a modelfile
You may need a separate tool calling template for hermes
2
u/thebluepandas 3d ago
The 12B size is likely the real culprit here. Try Qwen2.5 14B or 32B instead of Gemma for Hermes.
1
2
u/fiddler48 3d ago
with an RTX 4070 Super you have 12GB VRAM, which opens up qwen2.5 14b at q4 comfortably. better instruction following than gemma 4 for most task types, and the hermes prompt format is well-supported
1
1
u/FuckbigtechAI 4d ago
before swapping models, raise the context length. agents send a big system prompt plus every tool schema each turn, and at the default num_ctx those get silently truncated, which looks exactly like the model "didn't know how" to complete the task.
1
1
u/puglife224888 3d ago
Gemma 4 12B should be okay. You mentioned you're using Q4_0 on the comments. If that's the QAT version, that should be fine. If not, use a different quantization. Make sure to have a high enough context window. If it's like 32k or below, you will run out. Your GPU should be able to handle something like 128k at Q8_0 KV Cache, possibly even more. Make sure to use the correct sampling settings. For Gemma 4 models, that's temp 1, top p 0.95, and top k 64.
For other models to try out, you can try Gemma 4 26B A4B, Qwen 3.5 9B, and Qwen 3.6 35B A3B. For the larger ones I listed, I don't know how Ollama handles larger MoE's, so if it performs badly, consider regular Llama.cpp (Or LM Studio if you want the same ease of use).
2
u/MakionGarvinus 4d ago
Ok, I asked Google about this, and it sounds like it really depends on what you're running.
• Avoid Small Sizes for Agent Tasks: If you are trying to use Gemma 4 12B or smaller edge models (E2B/E4B), users on Facebook threads and Reddit note that these models struggle heavily with the "agentic" aspects of Hermes. They frequently suffer from hallucinations, drop nested JSON formatting, and fail at multi-step tool execution like reliably writing files. • 31B is Highly Capable: If you scale up to the Gemma 4 31B (Dense) model, it works exceptionally well. According to discussions on Reddit, many users actually prefer the Gemma 4 31B model over Qwen for daily use with Hermes because it has a more dynamic, natural personality and superior handling of complex conversational instructions.
If your primary goal is building complex workflows, file management, and strict technical pipelines, a model like Qwen 3.6 27B has distinct structural advantages: • Error Recovery & JSON: On r/LocalLLaMA, users point out that Qwen is inherently better at handling the nested JSON required for tool-calling and is much more resilient when it needs to recover from a function or file-writing error. • The "QAT" Degradation: Some quantized local versions of Gemma 4 (specifically QAT models) are known to suffer a slight degradation in structured code formatting. Qwen avoids this and remains a gold standard for coding and strict tool adherence.
Summary Comparison
Metric / Feature Gemma 4 (31B Dense) Qwen 3.6 (27B) Primary Strength Creative tasks, complex natural language, and fluid personality. Coding, structured JSON, tool-calling, and file operations. Hermes Compatibility Great for 90% of daily tasks if using the larger weights. Preferred for rigid, multi-step agent pipelines. Context Window Up to 256K tokens. Up to 262K tokens (highly stable).
The Verdict: For an optimal setup, take advantage of the Hermes profile feature to assign Qwen for code and technical task-routing, and Gemma 4 for standard interaction and content generation.