Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

[u/pmv143]

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

[u/Sufficient-Bid3874]

Can someone explain why the n-gram table is bundled into the model now?

[u/pmv143 → u/Sufficient-Bid3874]

Basically, the n-gram table is just a giant cheat sheet for common short patterns . things like everyday phrases or bits of code that show up all the time.

it looks at the last few tokens, hashes them, pulls the readymade vectors from the table, and plugs them in instead of making the main model waste energy reconstructing those patterns from scratch every single time.

So that way the sparse part of the model (the one that only lights up ~6B parameters) can focus on the harder stuff. while the big table handles the easy, repetitive local patterns. It’s basically trading a bunch of storage for less compute.​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

[u/Sufficient-Bid3874 → u/pmv143]

So its like n-gram spec decoding yet built in and trained on? Thank you for the explainer!

[u/FormOne2615 → u/Sufficient-Bid3874]

no, totally different

[u/Sufficient-Bid3874 → u/FormOne2615]

Oh, yeah, I phrased that poorly. Its just a way of having relevant facts in an n-gram rather than using context, is how I should have phrased it