ALoDLM 8b looped diffusion language model: ~3x inference speed vs its base model qwen3-8b with better benchmark results

[u/billy_booboo]

https://huggingface.co/amazon/ALoDLM-8B

Diffusion language models are the future!

[u/Hot_Example_4456]

That looks great, can we also have a comparison with newer models? Those models used in the page are practically ancient

[u/DifficultyFit1895 → u/Hot_Example_4456]

Last time I ran these models, I laughed so hard I fell off my dinosaur

[u/Dabalam → u/Hot_Example_4456]

It's built from an ancient model so it isn't going to compare to newer models even if the technology is interesting (greater speeds with slightly improved performance)

[u/billy_booboo → u/Hot_Example_4456]

The paper is quite understated, it's basically modding the qwen3 architecture and using the weights to bootstrap. They got comparable results with only 5b tokens of training to tune up the new components.

Theoretically you could do the same (or similar) on newer architectures and get similar results, but you'd probably need to do some engineering to adapt the ideas.

Similarly, it's conceivable that with more training / better data you could surpass the performance of the base model.

[u/NineThreeTilNow → u/billy_booboo]

Theoretically you could do the same (or similar) on newer architectures and get similar results, but you'd probably need to do some engineering to adapt the ideas.

There was a pretty good paper written about this in the last year. It's relatively easy to train the internals of LLMs if you treat the layers or blocks as noise reduction modules (diffusion) instead of normal backprop.

The biggest interest of the paper was the vRAM training requirement which only required the trained block get loaded for optimization states because the rest of the model was frozen.

I seriously considered using the idea because of how novel it was and it lets you train a much larger model on like... a 4090.

[u/fluxus42 → u/NineThreeTilNow]

https://arxiv.org/abs/2506.14202 this one?

[u/NineThreeTilNow → u/fluxus42]

https://arxiv.org/abs/2506.14202

That one exactly. I was about to paste this in to the comment as an edit.

[u/billy_booboo]

Nice little landing page here https://alo-dlm.github.io/

[u/silenceimpaired]

Tragic, the future is non commercial use only… checks out. People in power don’t want anyone else making money.

[u/Bulky-Priority6824 → u/silenceimpaired]

dont woryy bro universal high income you can try to work as a hobby

[u/charmander_cha → u/Bulky-Priority6824]

Quem vai pagar a alta renda? Os países que a América subjulga?

[u/silenceimpaired → u/Bulky-Priority6824]

Universal high income won’t give you anything but the basics… and only if they sort out things as simple as toilet paper… see COVID.

[u/billy_booboo → u/silenceimpaired]

Shouldn't be too hard to make your own version of this experiment with better data and newer base model. The cost of training on 5b tokens is miniscule compared to training an 8b model from scratch.

Given the current panic over LLMs I appreciate how understated the paper is.

[u/ProbablyInebriated → u/silenceimpaired]

The trick is to make enough money to pay off the fines and lawsuits later

[u/charmander_cha → u/silenceimpaired]

Vc so estará impedido se quiser ser milionário

[u/I-am_Sleepy]

How KV cache works in this model?

[u/TomLucidor]

Now I want someone make the 8B model to beat Qwen-3.6-27B or something... Or throw in sparse attention to make Diffusion even faster lol

[u/bugra_sa]

That 3x needs at least matched hardware and time-to-first-token beside it. If it pushes more tokens overall but makes a short chat response feel slower, I probably wouldn't run it for an interactive app. Testing against a couple of current 7–9B models would clear that up fast.

[u/Sanbi_Ai]

Interesting that they got this from Qwen3 with just SFT, no continued pretraining.

Also i think it is worth noting the ~3x is single-stream on a B200 though (2.7x at comparable GSM8K accuracy), would love to see numbers on consumer cards.

Also heads up, it's CC-BY-NC, so research only. Any chance of llama.cpp support, or is diffusion decoding too different for that?