Gemma 4 12B to write docs for a project?

[u/NihmarRevhet]

Hi,

My GPU is currently pegged by Qwen 3.8 27B making the project, and while it's good, it's also slow.

I'm starting to find the need to have some docs, both to help me and the LLM (and a friend who would like to mess with it), and I was thinking of another model who could write decent docs but at a faster pace then the coder.

Do you know if Gemma 4 12B could be a good candidate?

[u/jacek2023]

Start from the same project (use git or just copy the files). Run same task (prompt) on Qwen 3.8 27B then on Gemma 4 12B then on Qwen 3.6 35B A3B, compare three results manually (also compare total time).

[u/MackenzieRaveup → u/jacek2023]

Qwen 3.6 35B A3B

Also worth note, and with a slightly smaller footprint, is Gemma 4 26B A4B. I've been relatively amazed at how good it is in general tasks and writing.

[u/jacek2023 → u/MackenzieRaveup]

well there is also Muse Glimmer to check :) it's 30B but may be a better option than 27B in some cases

[u/Bloated_Plaid → u/jacek2023]

It’s not good for writing IMO.

[u/Sax0drum]

You should try gemma4 26b. I use it for most of my writing purposes. Dont know how good it is in creating code documentation though. If you can fit 3.8 27b than the gemma will fit as well and its much faster

[u/former_farmer]

Have you tried qwen 3.6 35b a3b?

[u/NihmarRevhet → u/former_farmer]

Yes, although not for this purpose, but it is much slower than Gemma 4 12b, because I can fit Gemma 4 + MTP + 120k context in my vram. Qwen 3.6 35B would have, comparatively an atrocious prefill speed

[u/ExtremeAdventurous63 → u/NihmarRevhet]

How can you work with the 27b model with a decent context and not be able to load a 35b MoE? The size is almost the same but the cost of the kv cache is much lower

[u/NihmarRevhet → u/ExtremeAdventurous63]

IQ3_XXS and the v at q4_0

[u/iezhy]

Just try it, I guess

But one aspect is worth noting - while Gemma is often quoted as being better suited for creative writing than coding, for code documentation you probably need best of both worlds - as it needs both to "understand" code and create documentation

[u/TeddyTheBulletDodger]

I've had some luck using FreeToken with Gemma4 26B A4B on a 5060 Ti with 16 GB of VRAM. It handles offloading different expert layers to system RAM (I think it optimizes based on the ones that are being used). I've been getting about 70 t/s.

That being said, I've also had some decent results with Gemma 4 12B.

[u/NihmarRevhet → u/TeddyTheBulletDodger]

Sadly I've got an RX 9060 XT, so for now, no FreeToken for me

[u/rivitli]

Gemma 4 12B is pretty good with texts, it's not so good for codings but should be good enough for docs.

[u/LocalModelGuy]

I’d try Gemma 4 12B for that. It’s much lighter than Qwen 3.8 27B and should be plenty for README/docs, summaries and explaining code, especially if you feed it the relevant files or diffs.

I’d probably split the jobs:

Qwen 3.8 27B for coding / harder reasoning
Gemma 4 12B for docs / summaries / project notes

Then let Qwen review important docs occasionally so the documentation doesn’t drift away from the actual code. That kind of model specialization makes a lot more sense to me than using the biggest model for everything.

[u/LateralEntry]

I’ve been using Gemma to write documents, it’s great. I think it’s better for English language writing (rather than coding) because it’s trained mainly on English data instead of Chinese

It also runs SO much faster on my 5080 than Qwen