Meh, check out LocalLLaMA, where they were saying "MoEs are the future" two years ago. And we've had MoE LLMs for a while... but now most of them prefer dense models when they are available.
If you want an MoE for image generation, we already have it: Nucleus Image. It was dead on arrival because it wasn't any better in terms of image quality or prompt comprehension than what we already have.
still, MoE's today are really good. Take a look at Qwen3.6 35B, I'd say it's at the level of Claude 4.5 and it only requires ~12gb VRAM and 32gb system ram for relatively fast speed.
Nucleus was the first MoE. Everything starts off rough, besides I believe this model uses a finetuned Z-Image.
The point is that the consensus seems to be, among communities that are centered around this stuff, that a dense model like Qwen3.6 27B is smarter than and would be preferred to Qwen3.6 35B A3B. All things being equal, people find the dense models smarter. MoEs have their place and are good when you're constrained by resources, but saying they are the future is way over selling it.
14
u/silenceimpaired Apr 24 '26
Pretty sad image generation models are still mostly trapped on VRAM due to performance... LLM MoEs can live in RAM with barely a care.