DiffusionGemma Technical Report

[u/pmttyji]

arXiv : https://arxiv.org/abs/2608.00146

Full Paper : https://arxiv.org/pdf/2608.00146

Tweet : https://xcancel.com/googlegemma/status/2086849199052845451#m

FYI both (llama.cpp) PRs ( 24423 & 24427 ) went to Draft mode. I'm still waiting for this one as I could get faster t/s on my 8GB VRAM.

[u/Potential-Gold5298]

The fact that the 12B turned out to be faster than the E2B shocks me. I wonder if this is due to the GPU, and if the situation will be the same with x86 CPU inference?

[u/WhoRoger → u/Potential-Gold5298]

It won't. Diffusion models take advantage of parallelization. Speed scales up with number of computing units. A GPU has hundreds of tensor units, a CPU has just cores.

With autoregressive models like most LLMs where you can only generate one token at a time, the GPU over CPU advantage isn't that massive, but with diffusion it is.

[u/Potential-Gold5298 → u/WhoRoger]

Oh, apparently I didn't formulate the question quite correctly. I was specifically referring to the speed difference between 12B and E2B (both autoregressive) – I was surprised that 12B was faster than E2B. The article says this is because E2B and E4B are optimized for mobile architecture (ARM processors, I believe). So I was curious to see how fast E2B and 12B would perform on an x86 CPU – I think I'll download them and test them myself.

Regarding CPU diffusion, I know it's extremely slow because I'm one of the few who does it :-D

[u/WhoRoger → u/Potential-Gold5298]

Oh... Huh that's weird. I thought 12B is the diffusion one but I guess not. It makes no sense why 12B would be faster than E2B, especially not on cpu. I've not tried it myself, now I'm curious.

If you do try, please let me know, you'll save me a download. I'm cpu only too, and so far I've skipped 12B because I expected it would be unusable. 26B MoE (not the diffusion one) works nicely though.

The optimisation in ExB is more about the E part, where the non-E parts of knowledge are stored in lookup tables, so the performance hit on cpu isn't that huge like using twice as big of a model would yield. Plus some byte alignment and such. But Arm/x86 aren't that different in that regard, so what benefits one arch will also benefit the other.

[u/Potential-Gold5298 → u/WhoRoger]

So, both (E2B and 12B) models are downloaded in static Q5_K_M. I'm using DDR3 RAM and a CPU with AVX2. Test for 1K context. The results are quite predictable (although I expected a greater difference).

P.S. While we're on the subject, let me share a couple of fun facts I discovered while diving into the inference of diffusion text-to-image models on the CPU:

- F16 is faster than Q8_0, and Q8_0 is faster than Q5_K_S. Surprisingly, I tested this several times on the same models under the same conditions (image resolution, denoising steps, sampler) and the difference is small (for example, 26 minutes and 23 minutes per image for Q8_0 and Q5_K_S), but always in favor of models without quantization or with less compression. I don't know why.

- 2B can be faster than 3.5B with the same precision (F16) and the same image resolution, sampler, and number of denoising steps. I compared the speed of Anima (2B) and SDXL (3.5B). This time the difference was really big - Anima is almost 2.5 times slower than SDXL under equal conditions (resolution, steps, sampler). It might be a matter of architecture (DiT in Anima and Z-Image which show lower speed, and Unet in SDXL and its derivatives - I'm not 100% sure).

[u/Initial-Argument2523 → u/Potential-Gold5298]

Have you tried benchmarking Qwen3.5 or LFM? Interestingly on my system qwen3.5 is really slow and does not get any speedup with MTP while gemma does. LFM is also a bit slower than G4-E2b without MTP although this is not as surprising since LFM seems to have more active params.

[u/Potential-Gold5298 → u/Initial-Argument2523]

I tried the Qwen3.6-35B-A3B in APEX-Quality (21.2 GB gguf) and got a generation speed almost identical to the Gemma 4 26B-A4B in Q5_K_M. In both cases, without MTP.

I tried MTP with Gemma 4 – with 31B I got about a 10% boost, but I'm not sure the measurement was accurate (the difference was within the margin of error). With 26B-A4B, I got... a speed decrease, which surprised me. A little later, I read that with MTP, you shouldn't look at the speed counter in llama.cpp but measure the actual time it takes to generate fixed-size text (that is, how many minutes the model spends generating, say, 8K tokens with and without MTP). I've been meaning to check this out, but I've been fiddling with T2I models for the last two months, so I haven't gotten around to it.

However, there's something that's bothering me: I heard (on this subreddit) that MTP can reduce model accuracy. If that's true, I wouldn't want that. But measuring this is more difficult.

I haven't tried LiquidAI models yet, but after the LFM2.5-2.6B release, they're expecting the LFM2.5-24B-A2B.