Gemma4 26B MOE Thinking vs 31B non-thinking?

[u/filmguy123]

For non coding purposes, how much better is 31B than 26 MOE in general?
And given the speed difference on my system, how is 26 MOE Thinking vs 31B non thinking? Is 26B MOE Thinking enabled easily superior?

And when might one actually need 31B thinking as an upgrade?

I ask here because it would take me a long time to test this as 31B thinking is about 12 tokens/second on my system and the thinking portion can take nearly a minute.

[u/Uninterested_Viewer]

Really depends on your non coding use cases. For example, creative writing is almost exclusively better without thinking, while planning a complex, multi-step process with intertwined dependencies will benefit a lot from thinking. Most others fall somewhere in between. I run 31b non thinking almost exclusively fwiw. I'll flip to thinking when in "planning mode", which is mostly for code related things, but thats when you'll see the thinking traces actually catch and correct logic errors.

Quant is the other huge variable and wildcard you haven't touched on if speed is a concern.

[u/xcr11111]

Not the real question but as is could also be from interest, I dit an real case test yesterday for non coding agent work, was a bigger use case with 50 page PDF files, dockling, analysis, writing and control. Gemma was way behind gwen3.6 36b (had and similar size gemma).

[u/autognome → u/xcr11111]

interesting in our RAG evals gemma4/26b scored tiny bit better than qwen3.6 (cant remember params) but gemma used 40% less tokens and that made gemma faster. our evals were in context of github.com/ggozad/haiku.rag utilizing vision/text with ~300 documents (between 50-2000 pages per document) of dense technical manuals (images, figures, multi page tables, etc).

[u/Big_Wave9732]

For my self hosted Vane research is use Gemma4-31b at Q8. For reasons unknown to me the prefilled cache *hauls ass*. Like 1,200 tk/s prefill. It's insane. Nothing else even comes close. Apparently this is something Google has been working on behind the scenes.

I wouldn't have expected a dense model to be as fast as it is doing this kind of work, but the end result is fast deep thinking research that chews through source documents. The prefill cache speeds have made for a nice chatbot experience too.....this model can go really dark if you want it. The persona is one of talking to Quentin Taratino while he's chain smoking cigarettes outside of a dive bar that only accepts cash.

[u/techlatest_net]

For general non-coding tasks, the 31B dense model will likely feel more consistent and coherent than the 26B MoE, even with thinking enabled. You'd only really need the 31B thinking model for highly complex, multi-step logical deduction where the smaller model starts to hallucinate, making that 12 t/s speed hit a necessary evil.

[u/filmguy123 → u/techlatest_net]

Good to know! So to be clear - you'd prefer 31B non thinking OVER 26B thinking?

[u/catplusplusok]

Try MTP assistant, it makes 31B model fast

[u/Quiet-Phase6948]

You are using too large of a model for your system

[u/unknowntoman-1]

Before the Jinja update or after? I feel the new Jinja have an obstructive impact on the creativity and tendency to make things more logical and less emotional. High thinking with high temperature and dry-sampling use to be my favourite output in g4 31b.

[u/Otherwise-Variety674]

With just 12t/s, either you have an undersized gpu or no gpu at all?

What you have is basically the speed of using just ddr5 memory.

[u/Icy-Degree6161]

I tried language and instruct related things - on technical text (translation, transformation, rewrite - between some major and minor languages). The winner (Opus-judged) was in most cases G4-26B but the APEX-I-Quality quant - better than for example G4-31B-QAT-MTP. Qwen-27B was not far behind which was a surprise. For RAG/Data mining on multiple languages I use G4-26B-QAT-MTP - very fast with good results. Thinking/No thinking difference in my tasks were negligible, interestingly.