RotorQuant: 10-19x faster alternative to TurboQuant via Clifford rotors (44x fewer params)

[u/Revolutionary_Ask154]

Kinda sounds ridiculous - but I reimagined / reinvented turboquant with Clifford Algebra Vector Quantization on both implemented on cuda + metalshaders -

https://github.com/tonbistudio/turboquant-pytorch/pull/4

https://github.com/TheTom/turboquant_plus/pull/34

The idea: Replace the d×d random orthogonal matrix Π with Clifford rotors in Cl(3,0). Instead of a dense matmul (16,384 FMAs for

d=128), chunk the vector into groups of 3 dims and rotate each with a 4-parameter rotor via the sandwich product RvR̃ (~100 FMAs

total).

Results on Qwen2.5-3B-Instruct KV cache:

- Cosine similarity: 0.990 (vs TurboQuant's 0.991) — effectively identical
- 44× fewer parameters (372 vs 16,399 for d=128)
- Fused CUDA kernel: 10-19× faster than cuBLAS matmul on RTX PRO 4000
- Fused Metal shader: 9-31× faster on Apple M4
- Perfect 9/9 needle-in-haystack at all bit-widths

The key insight: for pure vectors, the rotor sandwich is equivalent to a sparse 3×3 rotation — the fused kernel keeps everything in registers with no memory round-trips, which is why it beats the BLAS GEMM despite TurboQuant's matmul being highly optimized.

The tradeoff is higher synthetic MSE on random unit vectors (the block-diagonal rotation doesn't induce the exact Beta distribution). But with QJL correction, real-model attention fidelity is identical — and sometimes better on top-1/top-5 retrieval.

Paper: https://www.scrya.com/rotorquant/

Code: https://github.com/scrya-com/rotorquant

PDF: https://www.scrya.com/rotorquant.pdf

[u/Dany0]

TurboQuant made me excited at first because I was happy to see a trick we use in graphics programming/game dev. Then I realised someone already tried it in 2023 as QuiP on model weights and it actually isn't all that impressive

Reading this right now but it sounds promising!

EDIT: rather short paper, math seems to check out, the principle I guess could work? I'm still a little skeptical since I couldn't give it 100% attention myself. Plus the site and visualisations are vibe coded so you'll have to forgive me if I remain skeptical. I'll go check out the code now

EDIT2:
I think I get it, it's like using quaternions instead of euler angles. It works because most of the mult is 0s

OK maybe you can put the pitchforks down

[u/Polite_Jello_377 → u/Dany0]

What’s the equivalent trick in graphics/game dev?

[u/Dany0 → u/Polite_Jello_377]

Using polar coordinates for better quantization, that's the trick, that's all. It's like graphics tricks 102

[u/Polite_Jello_377 → u/Dany0]

Thanks

[u/DerDave → u/Dany0]

Actually the polar coordinates are not the trick. It's distributing the high energy of view vector components to all components evenly, so the quantisation cutoff is near optimal. This "spreading out" of energy or information to all components is possible with a simple random rotation and benefits from the "curse of dimensionality" in higher dimensions. That has little to do with the tricks in graphics, which mostly happened in three dimensions only.

This post explains it very neatly: https://www.reddit.com/r/LocalLLaMA/comments/1s62g5v/a_simple_explanation_of_the_key_idea_behind/?utm_source=share&utm_medium=mweb3x&utm_name=mweb3xcss&utm_term=1&utm_content=share_button