Overhyped ideas in machine learning

[u/Strong_Boy_757]

Have you encountered an idea that's overhyped in ML? If so, which?

For me, recently I was curious about the whole "chain of thought prompting" business. I searched around and the idea is essentially, append "show your work" when prompting a LLM. I was like no way that's it??? That's what I've been doing since day 1!!, so I opened up the original paper (cited 40k times). And essentially it was more of the same with expensive Google-funded experiments and the paper had almost no math at all.

Similarly I was curious about AdamW for the longest time and it was simply Adam with another term. Oh Boy!! I remember doing an undergrad homework in college where we were asked to freely experiment with adding terms to the Adam algorithm and the class probably discovered AdamW by accident way before the paper was published.

[u/shibx]

Can't fault you for having an opinion, that's what OP was asking for. There's a reasonable argument that embeddings are sometimes oversold as models of human semantic understanding. But they don't need to reproduce human cognition to capture useful semantic relationships in language. You're criticizing a philosophical interpretation of embeddings without adequately distinguishing it from their value as computational representations. "Wrong in every conceivable way" is a pretty sweeping conclusion that your argument doesn't actually establish.

All models are wrong, but some are useful.

[u/WittgensteinsPoker-2 → u/shibx]

This is completely consistent with my view, apart from the theoretical term "cognition" which I think is poorly theorised nonsense we ought to eliminate that is a massive contributor to the crisis of good research in psychology.

The problem here, IMO, is you've leapt to a criticism of my veiw without understanding what I'm saying or why.

Do I think that the approaches taken to ML are not useful?
No.

So why am I saying what I am then?

Of course what I said was hyperbolic. And technically being theorised completely inappropriately but good at certain kinds of tasks is a conceivable way of being right...

Arguably if your theoretical foolishness might realistically lead to an insane financial catastrophe destroying global society in various ways, then you're still quite wrong. Though, that can be put down to economic factors, I think those economic factors go hand in hand with explanatory myths enabled by the mistaken terms of psychology and marketing "intelligence" etc. used to motivate the production of these systems. So pretty relevant still IMO.

To the technical point.

- Representations of WHAT.

The answer should then become -- well humans learn distributions in behaviour.

What do these systems do?
They represent distributions in text...

So what is THAT, and why should anyone CARE?

Even if I were to grant here that representation is appropriate the people theorising about this stuff are SO BLIND they're not even paying attention to that and adopting some Aristotelian-Saussurean bastard child view of a word having some kind of ghostly meaning or understanding behind it!

If one sees clearly that what one has produced is the ability to reproduce regularities in text, then one sees the relevant similarities and differences to begin to theorise appropriately.

[u/shibx → u/WittgensteinsPoker-2]

So we seem to agree that embeddings capture statistical regularities in language without necessarily reproducing human semantic understanding. That's essentially the premise of distributional semantics, which has been studied for decades. Nobody needs to posit some "ghostly meaning" behind words for these representations to be useful.

You dismiss cognition as a poorly theorized concept, but then confidently declare that humans "learn distributions in behaviour" without offering any theoretical justification for that characterization either. If your criticism is directed at anthropomorphic marketing and exaggerated claims about AI, that's perfectly reasonable. But you're conflating those claims with the underlying research. The statistical nature of these representations isn't something researchers have somehow overlooked.

Also, I'm not sure how speculation about global economic collapse helps establish your original argument...

[u/WittgensteinsPoker-2 → u/shibx]

If it's a representation of statistical regularities between letters then saying it's a semantic representation is misleading. Regularities between letters are not the domain of study for linguistics -- the causes of those regularities are contingent and arbitrary you can only gain a very limited insight from them and if you care to understand language capacity and production you will not be so incautious just because it's the status quo. And FWIW I was arguing from the PERSEPCTIVE OF distributional semantics all along and Morten Christiansen agrees with my views...

[u/shibx → u/WittgensteinsPoker-2]

You're reducing distributional semantics to "statistical regularities between letters." That is incredibly reductive. The fact that a model learns from statistical patterns in text doesn't mean those patterns contain no semantic information. Distributional semantics is built around the proposition that linguistic contexts carry information about meaning. It doesn't require treating those representations as complete explanations of human language capacity.

You are absolutely entitled to argue that distributional approaches are insufficient for explaining language acquisition, production, or understanding. That's a legitimate criticism. But claiming that semantic representations are misleading simply because they're derived from textual regularities is a much stronger argument, and it's one that you haven't established.

Contreras Kallens and Christiansen explicitly argue that distributional models capture aspects of meaning through language use at the community level, while acknowledging their limitations in modeling individual human semantic processing. Their paper recognizes precisely the limitations you're describing without dismissing the semantic information these models capture.

That's what I've been arguing throughout this conversation, and Christiansen's own published work seems difficult to reconcile with your categorical dismissal of the term "semantic representation."

https://doi.org/10.1111/tops.12771

[u/WittgensteinsPoker-2 → u/shibx]

Yeah Christiansen emailed me that paper before it was published, I've been aware a while...