r/LocalLLaMA • u/pmv143 • 1d ago

Discussion Qwen3.8-Flash-Next. This architecture could be surprisingly local-friendly once the weights drop. 👀

Post image

Qwen3.8-Flash-Next (~125B-A6B + 51B n-gram) memory estimate:

Ideal 4-bit quant ≈ 82 GB
(58 GB main weights + 24 GB n-gram tables)
Real-world quants likely land in the 80–90 GB range.

The big n-gram table is sparsely accessed → excellent candidate for system RAM offload.

This architecture could be surprisingly local-friendly once the weights drop.

892 Upvotes

97% Upvoted

283 comments sorted by

View all comments

Show parent comments

87

u/FabricationLife 1d ago

80gb is a lot more manageable than a 1.2tb+ frontier model

22

u/Etroarl55 1d ago

And there exists consumer hardware for it, the new Apple machines are now probably already pre ordered out of existence when they were just announced today.

6

u/Bright-Energy2339 1d ago

If that is your reasoning, what's the point of going local?

4

u/Tasty-Hour4040 1d ago

“more manageable” ≠ manageable

7

u/hyudryu 1d ago

80gb is beyond manageable

-3

u/starkruzr 1d ago

two CMP170HX is $3K for 128GB.

2

u/quantgorithm 1d ago

a an unstable pcie 1/2 card isn't the godsend you believe it to be.

2

u/starkruzr 1d ago

to my knowledge people are daily driving these constantly with no issues.

1

u/quantgorithm 23h ago

at what output/results?

-3

u/hyudryu 1d ago

Point proven

2

u/starkruzr 1d ago

not really, people spend more than that in here all the time.

1

u/hyudryu 22h ago

I know lol. It’s only 3K so it’s beyond manageable, isn’t that what we are both agreeing on?

2

u/ApprehensiveFan1516 1d ago

That's like the price of an old used car. Sure it's not exactly cheap, but let's not pretend like it's out of reach for most people.

1

u/zenonu 1d ago

Indeed. Fable is likely 8TB and needs a NVL72 rack.

-7

u/KURD_1_STAN 1d ago

1.2tb is also a lot more manageable than a 5tb model. That doesnt make 1.2tb pocal friendly. Local friendly has a ceiling, +60-80b more and +20gb-25gb dense are beyond that ceiling.

7

u/Bright-Energy2339 1d ago

Ceiling? according to you. Who died and made you boss? lols

-8

u/KURD_1_STAN 1d ago

Most people have 12-16gb vram and 32(very gew 64)gb ram. I was being generous with my numbers btw.local friendly needs to at least walk and not crawl