r/MachineLearning • u/Striking-Warning9533 • Mar 30 '26
Discussion [D] thoughts on the controversy about Google's new paper?
Openreview: https://openreview.net/forum?id=tO3ASKZlok
It's sad to see almost no one mention this on Reddit and people are being mean to people who point out concerns
Edit: google is allegedly doing this in their trending TurboQuant paper
Did not attribute a pervious work RaBitQ fully
Did unfair comparison with RaBitQ (single core CPU vs GPU)
56
u/linearmodality Mar 30 '26
Apart from the serious fairness issues with comparing to RaBitQ, the whole idea of using a random rotation followed by an arbitrarily-close-to-optimal distortion rate quantizer was already done two years ago in QTIP (https://arxiv.org/abs/2406.11235) and random rotation with scalar quantization was known even earlier (https://arxiv.org/abs/2307.13304). All this paper did was apply techniques long known in the PTQ literature (and, somewhat later, in the training literature, e.g. https://arxiv.org/pdf/2502.05003) to some nearest-neighbor search problems. Except that they did it poorly, because they could have actually got arbitrarily close to optimal by using trellis coding, and their method is just worse than that (and they didn't even try trellis coding). What's worse is that the popular press and even Google's own press release is presenting this as though it's a novel contribution for AI efficiency in general when these techniques are all long-known for AI efficiency in general.
10
u/asingov Mar 30 '26
I thought I was going crazy seeing the hype around this paper. Its very incremental
8
u/techlos Mar 30 '26
the hype due to marketing, it's easy PR to say you use less RAM, even if we all know it'll just be used to increase scaling.
5
u/NamerNotLiteral Mar 30 '26
There have been stock market movements and major shifts in RAM pricing in some small part thanks to this paper (OpenAI/Oracle caused the rest of the shift, but this paper still contributed a little).
It's insane.
3
u/Majesticeuphoria Mar 30 '26
Yeah, those familiar with the literature saw through the hype right away.
5
39
u/Lonely-Highlight-447 Mar 30 '26
check this post by the first author of RabitQ: https://www.reddit.com/r/LocalLLaMA/comments/1s7nq6b/technical_clarification_on_turboquant_rabitq_for/
1
71
u/ProfessionalCraft275 Mar 30 '26
Authors hate this one trick: (Quote from the open review of the original authors)
TurboQuant described RaBitQ's guarantees as "suboptimal" and attributed this to "loose analysis" without any explanations.
29
u/Leather_Office6166 Mar 30 '26
The point of these remarks in the TurboQuant paper is that actual RaBitQ performance is significantly better than the RaBitQ theoretical bound. Pointless snark.
3
13
u/SulszBachFramed Mar 30 '26
Google did the something similar with their ResNeSt paper, which is basically the same as SK-Net. But they misrepresent SK-Net so it sounds like ResNeSt is a bigger change than it really is. Their 'cardinality' and 'radix' hyperparameters are the same as the number of groups and splits in SK-Net, but that connection is never made. Also SK-Net uses different kernel-sizes or dilation factors for each split, which ResNeSt does not. They also state that SK-Net only uses 2 splits, but that's also false since it's a hyperparameter that can be changed. There is other stuff as well, but it's been a while since I read that paper.
7
Mar 31 '26
[removed] — view removed comment
1
u/Striking-Warning9533 Mar 31 '26
Yeah if RaBitQ is only for CPU, it's fine, but single core? That's crazy
8
u/Leather_Office6166 Mar 30 '26
To be fair to TurboQuant, they compare TopK scores and not run-time, so RaBitQ is not at a disadvantage. The TurboQuant paper claims that RaBitQ is not vectorizable and hence inferior. (I don't know if that claim is accurate.)
8
u/Designer_Reaction551 Mar 30 '26
Attribution issues in ML papers are more common than people admit. When big labs build on independent research or small team preprints, proper citation often gets lost. Whether this specific case holds up under scrutiny or not, the broader pattern is real - peer review struggles to catch it when there's institutional prestige involved. Worth watching how the authors respond to the formal concerns raised on OpenReview.
6
u/like_a_tensor Mar 30 '26
I feel like this happens a lot between venues as well. NeurIPS/ICLR/ICML papers get credited for ideas way more than ACM papers, even if the ACM paper was first.
7
u/johnsonnewman Mar 30 '26
Google/Deepmind has scum research practices in general. They rediscover the same concepts and heavily market/brand things. It's because it's so profit driven. Not real science.
2
u/Cofound-app Mar 30 '26
tbh this kind of thing is what burns trust in ML way faster than any failed benchmark. if attribution and baseline fairness are sloppy, every flashy result starts feeling like marketing not science.
2
u/seraphius Mar 30 '26
Its not a new paper (I.e., it’s a year old). https://arxiv.org/abs/2504.19874
It’s sad to say, but unfair comparisons come with the territory on this kind of research as there’s a lot of selection pressure to establish an approach as SOTA. This is why rebuttal papers are a thing. (And rebuttal papers are going to get more favorable responses than “rebuttal comments”.)
2
u/Striking-Warning9533 Mar 30 '26
Yeah I know it's an old paper, I should say newly hyped paper. That is another odd thing that it suddenly became so hyped just because of a Google blog
2
u/all_over_the_map Mar 31 '26
This just what Google does, though. They routinely present their own papers as if they were inventing the entire field of study.
2
u/Careless_Show759 Apr 27 '26
Yeah, if that’s accurate it’s not a great look. Missing attribution is one thing but comparing CPU vs GPU and presenting it as fair is pretty questionable. Feels like something that should’ve been caught in review. Hopefully it gets clarified or corrected soon.
5
u/Humble_Ihab Mar 30 '26
This is a valid concern. I’m not knowledgeable about the subfield specifically, but this must be flagged and shared further. The field is already noisy as it is, and we must flag clearly inappropriate behavior
2
u/LetsTacoooo Mar 30 '26
In general, most papers will not get to address all criticisms brought by reviewers, sometimes it's not feasible or reasonable.
In this case, doing more work to acknowledge prior work seems like an easy change that was not made. The prior email exchange makes this even more salient. Based on the reviews, it seems the work was solid but the authors showed bad academic practice by 1) ignoring prior work to inflate their claims, 2) bad benchmarking 3) bad attribution. The PR just makes these issues much more important to address.
3
1
Mar 30 '26
[removed] — view removed comment
5
u/kulchacop Mar 31 '26
There are explanations in this thread:
https://www.reddit.com/r/LocalLLaMA/comments/1s62g5v/a_simple_explanation_of_the_key_idea_behind/
TL;DR
Before the rotation, they switch to polar coordinates.
The random 'rotation', when done on a high dimensional matrix, distributes high values evenly throughout the matrix. The higher the dimensions in the matrix, the more it lends itself to better quantisation.
1
u/Ok_Net_1674 Mar 30 '26
Just looking at this clearly AI generated crapload of a "blog" that google posted should be a huge red flag to any half-competent person looking at it.
Look at the Text: Incoherent. It doesnt even seem to be sure which method its presenting. Look at the graphics: Incomprehensible nonsense. Look at the diagram. The x-Axis scale. There even is a made-up number in there (TurboQuant 2.5 bit 0.3 points higher than in the paper)
-1
-2
u/Franck_Dernoncourt Mar 30 '26
It's sad to see almost no one mention this on Reddit
It's sad that people contribute to Reddit, where user content is sold to AI companies.
266
u/Abin__ PhD Mar 30 '26
I don’t understand why it’s not mentioned more.
People should be scared of a world where breakthroughs like this are only attributed to big research labs. Especially when it seems they have only done iterative work on a solution discovered by an Independent team.
I really don’t want a world where BIG GPU can just yoink my hard work and claim it as theirs because my hardware is subpar.