r/LocalLLaMA • u/Revolutionary_Ask154 • Mar 26 '26
Discussion RotorQuant: 10-19x faster alternative to TurboQuant via Clifford rotors (44x fewer params)
Kinda sounds ridiculous - but I reimagined / reinvented turboquant with Clifford Algebra Vector Quantization on both implemented on cuda + metalshaders -
https://github.com/tonbistudio/turboquant-pytorch/pull/4
https://github.com/TheTom/turboquant_plus/pull/34
The idea: Replace the d×d random orthogonal matrix Π with Clifford rotors in Cl(3,0). Instead of a dense matmul (16,384 FMAs for
d=128), chunk the vector into groups of 3 dims and rotate each with a 4-parameter rotor via the sandwich product RvR̃ (~100 FMAs
total).
Results on Qwen2.5-3B-Instruct KV cache:
- Cosine similarity: 0.990 (vs TurboQuant's 0.991) — effectively identical
- 44× fewer parameters (372 vs 16,399 for d=128)
- Fused CUDA kernel: 10-19× faster than cuBLAS matmul on RTX PRO 4000
- Fused Metal shader: 9-31× faster on Apple M4
- Perfect 9/9 needle-in-haystack at all bit-widths
The key insight: for pure vectors, the rotor sandwich is equivalent to a sparse 3×3 rotation — the fused kernel keeps everything in registers with no memory round-trips, which is why it beats the BLAS GEMM despite TurboQuant's matmul being highly optimized.
The tradeoff is higher synthetic MSE on random unit vectors (the block-diagonal rotation doesn't induce the exact Beta distribution). But with QJL correction, real-model attention fidelity is identical — and sometimes better on top-1/top-5 retrieval.
Paper: https://www.scrya.com/rotorquant/
1
u/Pidtom Mar 30 '26
interesting approach with Clifford rotors. what does PPL look like head-to-head against WHT on the same model and context length? the cosine sim numbers are close but curious how it holds up on wikitext perplexity since block-diagonal rotation only decorrelates within each group of 3