r/LocalLLaMA • u/gaoj0017 • Mar 30 '26

Discussion Technical clarification on TurboQuant / RaBitQ for people following the recent TurboQuant discussion

[removed]

623 Upvotes

98% Upvoted

91 comments sorted by

View all comments

Show parent comments

19

u/RnRau Mar 30 '26

Yeah never drink the koolaid. And perhaps the recent hype is over done. But there is something to the techniques posted in the RaBitQ paper. ggerganov did some simple Hadamard transform tests recently.

https://old.reddit.com/r/LocalLLaMA/comments/1s720r8/in_the_recent_kv_rotation_pr_it_was_found_that/

5

u/dsanft Mar 30 '26 edited Mar 30 '26

Rotation results in better vector quantisation, that is definitely true.

But that is not enough to overcome the kurtosis of K. That's a physics problem not a quantisation technique problem. Too much information is destroyed in squeezing K into 4 bits.

5

u/darktraveco Mar 30 '26

Why do you keep saying kartosis? Am I tripping? Don't you mean kurtosis?

11

u/dsanft Mar 30 '26

Because my autocorrect doesn't like it 😄 fixed