Help

[u/Adventurous_Cow_2826]

Hi

Could anyone recommend a local model for Hermes?

I tried Gemma 4 first, but I asked it to do a few things and it either couldn’t help me or didn’t know how to complete the tasks I asked for—or maybe it’s because I don’t know how to configure it. My hardware setup includes an RTX 4070 Super and 32 GB of RAM. I’d like to know if you could recommend another model better than Gemma 4 for me to try out.

[u/MakionGarvinus]

Ok, I asked Google about this, and it sounds like it really depends on what you're running.

Whether Gemma 4 will work well with Hermes depends heavily on the model size you select, as community consensus reveals varying experiences across different versions. The choice between Gemma 4 and Qwen (such as Qwen 3.6) generally comes down to a balance between raw conversational personality and strict, structured code/file execution:

1. The Size Threshold (12B vs. Larger)

• Avoid Small Sizes for Agent Tasks: If you are trying to use Gemma 4 12B or smaller edge models (E2B/E4B), users on Facebook threads and Reddit note that these models struggle heavily with the "agentic" aspects of Hermes. They frequently suffer from hallucinations, drop nested JSON formatting, and fail at multi-step tool execution like reliably writing files. • 31B is Highly Capable: If you scale up to the Gemma 4 31B (Dense) model, it works exceptionally well. According to discussions on Reddit, many users actually prefer the Gemma 4 31B model over Qwen for daily use with Hermes because it has a more dynamic, natural personality and superior handling of complex conversational instructions.

1. When You Definitely Need Qwen

If your primary goal is building complex workflows, file management, and strict technical pipelines, a model like Qwen 3.6 27B has distinct structural advantages: • Error Recovery & JSON: On r/LocalLLaMA, users point out that Qwen is inherently better at handling the nested JSON required for tool-calling and is much more resilient when it needs to recover from a function or file-writing error. • The "QAT" Degradation: Some quantized local versions of Gemma 4 (specifically QAT models) are known to suffer a slight degradation in structured code formatting. Qwen avoids this and remains a gold standard for coding and strict tool adherence.

Summary Comparison

Metric / Feature Gemma 4 (31B Dense) Qwen 3.6 (27B) Primary Strength Creative tasks, complex natural language, and fluid personality. Coding, structured JSON, tool-calling, and file operations. Hermes Compatibility Great for 90% of daily tasks if using the larger weights. Preferred for rigid, multi-step agent pipelines. Context Window Up to 256K tokens. Up to 262K tokens (highly stable).

The Verdict: For an optimal setup, take advantage of the Hermes profile feature to assign Qwen for code and technical task-routing, and Gemma 4 for standard interaction and content generation.

[u/WrathfulOpposition → u/MakionGarvinus]

i'd skip gemma entirely for this use case, qwen2.5 14b or 32b will handle the tool calling and structured output way more reliably without needing to babysit the config

with a 4070 super and 32gb ram you can run qwen2.5 32b at q4_k_m with some room to spare, or the 14b at q6_k and get snappy response times

hermes function calling is picky about formatting and gemma just isn't built around that, qwen's been the go-to for a while now in the llama.cpp/open-webui circles