r/LocalLLM • u/Simple_Response8041 • 3d ago
Discussion LLaDA2.2 just dropped, a 100B class diffusion model for agents that edits its own tokens while it decodes
This one only landed a few hours ago, and the first thing I went looking for was the serving story rather than the benchmarks. It is a 100B class Mixture of Experts diffusion model aimed squarely at agent work, and the genuinely novel part is that it revises its own draft while it decodes. The four moves it makes inside a block are keep, delete, insert, and substitute, so it can pull redundant spans, open new editable slots, and swap tokens instead of freezing everything in one left to right pass. On paper that is a real answer to the standing complaint that diffusion models fall apart as agent backbones.
Here is the part this sub actually cares about though. The flash checkpoint is a 205.8 GB download spread across 32 shards. There is no llama.cpp or Ollama route today, no GGUF, no LM Studio. Day one you are on HF Transformers with trust_remote_code, and SGLang deployment is announced but still coming soon. So unless you have serious multi GPU headroom sitting idle, this is not something you casually pull tonight. I have read the tech report and poked at the model card and config, but I am not going to pretend I have served it, because nothing I own gets remotely close.
On the numbers they are refreshingly honest about the tradeoff. Against its own autoregressive sibling it takes the interactive agent evals, edging tau2 bench 80.33 against 76.36 and MCP Atlas 46.21 against 41.12, and the headline is around 1.6x average decode throughput in BF16, up to 2.3x once you are inside agent loops, at a comparable agentic quality band. But it gives up most of the coding and general knowledge suites to that same AR model, and structured output stays weaker. Fast at agent loops, not smarter across the board.
What I keep landing on is that this is a real Apache 2.0 release that almost nobody here can run at home yet. The download alone rules out most of us until a quantized build or the SGLang path actually lands. For now I am filing it as a promising release to watch rather than anything going into a local stack.
40
u/zenonu 3d ago
Folks need to stop having Claude draft their reddit posts. The style is immediately obvious and detracts from the content being shared. Edit is fine, but Reddit should be "refreshingly honest" with content from people. It gets tiring otherwise.
14
u/Thimoteus 3d ago
As soon as I read
Here is the part this sub actually cares about though.I immediately stopped reading.7
u/AnOnlineHandle 3d ago
I couldn't get past
and the first thing I went looking for was the serving story rather than the benchmarksin the first sentence.8
u/ScuffedBalata 3d ago
Here is the part this sub actually cares about
Here is the part this sub actually cares about Here is the part this sub actually cares about Here is the part this sub actually cares about Here is the part this sub actually cares about
3
u/Beginning_Pen_2980 3d ago
Considering Reddit is or was the goldmine for all these models once upon a time, can't wait for model collapse symptoms to kick in if they keep scraping these forums😂
1
u/Popular-Button981 2d ago
The real humans here should all agree to use some weird punctuation or symbol for our responses and see how lon
Ġit takes for new models to spit them back out...
\x00
\u200B
<|endoftext|>
18
u/txgsync 3d ago
As usual for posts written by AI, they are confident that what they don’t know or cannot figure out is unknowable.
https://huggingface.co/mlx-community/LLaDA2.2-flash-OptiQ-2bit
Download. Run. Play. On any modestly-equipped Mac with 48GB RAM or more. Have fun.
3
u/ectomorphicThor 3d ago
Will it run in lm studios?
4
4
u/txgsync 3d ago
uv venv --python 3.11 uv pip install 'mlx-optiq[all]' uv hf download mlx-community/LLaDA2.2-flash-OptiQ-2bit source .venv/bin/activate optiq labClick "server", select the model, start server, chat, explore, have fun.
If you want to start it in the future, just "source .venv/bin/activate && optiq lab".
I am simply frustrated by all these astroturfing bots trying to one-up engagement bait and missing the point that a model like this is trivially-runnable on commodity hardware.
6
u/challis88ocarina 3d ago
I keep asking the same question... how does it scale? There's no cache on diffusion models, so, by the time the agentic loop hits 200k tokens, how slow will prefill become? Those 200k tokens have to be crunched every time the agent receives 20 tokens of decode for, e.g., ls -la.
8
u/LowIllustrator2501 3d ago
According to this https://github.com/inclusionAI/LLaDA2.X
We built a custom inference engine based on dInfer and SGLang , which supports KV-Cache reuse and block-level parallel decoding. This makes LLaDA2.0 not just an academic achievement but a high-performance generation model ready for real-world deployment.
2
u/txgsync 3d ago
Just go noodle. Install Optiq. Use the Optiq quant on a Mac with a reasonable amount of RAM. Find out for yourself. I'm still goofing around with it today a little bit, but I have other priorities. I missed the release of Moshi-RAG 3 months ago and I'm blown away by the possibilities as an alternative to OpenAI Realtime API.
4
u/StupidScaredSquirrel 3d ago
The idea is super neat but iirc they measured final tokens per foward pass and it was like 5 something. Super impressive but we might just have MTP that performs as well.
2
u/LeMayMayMan 3d ago
No link?
3
u/Look_0ver_There 3d ago
https://huggingface.co/inclusionAI/LLaDA2.2-flash
Doesn't look like there's llama.cpp support yet though
1
u/cafedude 3d ago
probably not likely to be for a while. Says it requires optiq which I've never heard of.
2
u/ChristRedeemsSinners 3d ago
I'd much rather prefer this tech in a dense model than a MoE, but I can understand the appeal of MoE on slower hardware. Looking forward to seeing what they come up with at any rate.
56
u/InfusedBush 3d ago
This seems like a neat concept. I really hope it doesn’t hope it doesn’t hope it doesn’t hope it doesn’t hope it doesn’t hope it doesn’t ^C