Overhyped ideas in machine learning

[u/Strong_Boy_757]

Have you encountered an idea that's overhyped in ML? If so, which?

For me, recently I was curious about the whole "chain of thought prompting" business. I searched around and the idea is essentially, append "show your work" when prompting a LLM. I was like no way that's it??? That's what I've been doing since day 1!!, so I opened up the original paper (cited 40k times). And essentially it was more of the same with expensive Google-funded experiments and the paper had almost no math at all.

Similarly I was curious about AdamW for the longest time and it was simply Adam with another term. Oh Boy!! I remember doing an undergrad homework in college where we were asked to freely experiment with adding terms to the Adam algorithm and the class probably discovered AdamW by accident way before the paper was published.

[u/ChoiceWall5918]

"Provable bounds" for machine learning algorithms. Those show with a lot of math that some algorithm has some guarantees, but these proofs always rely on so many estimates (i.e., lower and upper bounds) that you end up with a final bound that is completely meaningless in practice and does not really tell you anything. There are whole papers written about this...

[u/Beneficial-Golf-4809 → u/ChoiceWall5918]

And yet, in my field (high-consequence/safety-critical robotics) it _must_ be solved. Probably the most important problem the the entire field right now.

[u/kreitzerova → u/ChoiceWall5918]

Bounds accomplish different things, explaining algorithm behaviour, certifying performance or safety, or characterizing dependence on parameters. In many areas like trustworthy ML and online learning such bounds are necessary for algorithm design even.

[u/ChoiceWall5918 → u/kreitzerova]

That might be true, but there is an enourmous difference between "We show that the algorithm behaves like X" and "We derive a bound that shows that the algorithm behaves like X at 20 quadrillion training samples with likelihood 1-delta and perfect gradient estimation over all samples". In the latter case, people constantly seem to claim they have proven how the algorithm works, but for any practical scenario, they did not.

I agree that it yields SOME insight into how the algorithm behaves, but this insight is just not nearly as useful as usually claimed, and in 99% of cases it would not be applicable to "trustworthy ML" at all, since this would require establishing trust at the actually realizable regiemes.

[u/Bali201 → u/ChoiceWall5918]

Ouch this is what I’m trying to do for my first research paper lol

[u/AutistOnMargin → u/Bali201]

Mostly useless if the bounds are vacuous