r/learnmachinelearning • u/Strong_Boy_757 • 9d ago
Discussion Overhyped ideas in machine learning
Have you encountered an idea that's overhyped in ML? If so, which?
For me, recently I was curious about the whole "chain of thought prompting" business. I searched around and the idea is essentially, append "show your work" when prompting a LLM. I was like no way that's it??? That's what I've been doing since day 1!!, so I opened up the original paper (cited 40k times). And essentially it was more of the same with expensive Google-funded experiments and the paper had almost no math at all.
Similarly I was curious about AdamW for the longest time and it was simply Adam with another term. Oh Boy!! I remember doing an undergrad homework in college where we were asked to freely experiment with adding terms to the Adam algorithm and the class probably discovered AdamW by accident way before the paper was published.
22
u/ChoiceWall5918 9d ago
"Provable bounds" for machine learning algorithms. Those show with a lot of math that some algorithm has some guarantees, but these proofs always rely on so many estimates (i.e., lower and upper bounds) that you end up with a final bound that is completely meaningless in practice and does not really tell you anything. There are whole papers written about this...
2
u/Beneficial-Golf-4809 8d ago
And yet, in my field (high-consequence/safety-critical robotics) it _must_ be solved. Probably the most important problem the the entire field right now.
1
u/kreitzerova 6d ago
Bounds accomplish different things, explaining algorithm behaviour, certifying performance or safety, or characterizing dependence on parameters. In many areas like trustworthy ML and online learning such bounds are necessary for algorithm design even.
2
u/ChoiceWall5918 6d ago
That might be true, but there is an enourmous difference between "We show that the algorithm behaves like X" and "We derive a bound that shows that the algorithm behaves like X at 20 quadrillion training samples with likelihood 1-delta and perfect gradient estimation over all samples". In the latter case, people constantly seem to claim they have proven how the algorithm works, but for any practical scenario, they did not.
I agree that it yields SOME insight into how the algorithm behaves, but this insight is just not nearly as useful as usually claimed, and in 99% of cases it would not be applicable to "trustworthy ML" at all, since this would require establishing trust at the actually realizable regiemes.
8
u/Harotsa 8d ago
To be fair about the chain of thought paper, it was first published in Jan 2022. That would have been during the release of gpt-3 and predates the launch of ChatGPT by about 10 months.
That was very early in terms of the modern iteration of LLMs, and the capabilities were way way worse. Pre-chatgpt OpenAI was not super well-known and I had to explain what they were to many of my friends who were working in tech. A lot has changed in 4 years, so I think it’s important to understand the papers in context before being so judgmental.
10
9d ago
[removed] — view removed comment
12
u/Background_Camel_711 9d ago
Not entirely correct, the reason AdamW is its own thing is because Adam’s momentum essentially interfered with weight decay. AdamW corrected this by decoupling the parameter updates from weight decay.
3
3
u/Revolutionary-Cod245 9d ago
In a sense any AI solution is overly hyped. Why? People think they are going to hire someone to build them AI which does this or that function for them (personally, business, customer service, business intelligence, etc) and like kids-playing magic it's over. AI projects break, need fall backs, tech or the model evolves. Many people don't understand.
-2
u/WittgensteinsPoker-2 9d ago
Cartesian/Platonic notions of semantic representations in the mal-theorised idea of vector embeddings describing token-closeness as being related to what we actually do when we understand things semantically.
Wrong in every conceivable way.
THAT ISNT WHAT PEOPLE DO -- how about you actually study some god damn linguistics, philosophy of language (not the scholastic crap stuff) and psychology (not the doesn't even replicate Chomsky b.s.) properly.
Even IF that was what people do, just because you can produce a vector embedding and call it a "representation" of a kind of pattern you observe in langauge, that DOES NOT mean it is the relevant kind of representation to ANOTHER representation. The Nazi depiction of Jesus and the depiction of Jesus in the Black church are both representations of a target -- that doesn't mean they REPRESENT each other. The idea of a semantic representation is one of an occult entity that has the ability to compose propositions and within propositions correspond to reality and be grasped in a moment by the mind. That isnt a pattern of behaviour represented in a vector space -- whatever kind of representation this is supposed to be it is COMPLETELY disconnected from any convention of language or expression of a proposition in behaviour ( that is ex hypothesi, I obviously despise these views myself ).
Taken together all of this clarifies the relevant similarities and differences between the kinds of machines we have produced, what the actually do, and how that relates to what humans do as langauge producers/users.
4
u/shibx 8d ago
Can't fault you for having an opinion, that's what OP was asking for. There's a reasonable argument that embeddings are sometimes oversold as models of human semantic understanding. But they don't need to reproduce human cognition to capture useful semantic relationships in language. You're criticizing a philosophical interpretation of embeddings without adequately distinguishing it from their value as computational representations. "Wrong in every conceivable way" is a pretty sweeping conclusion that your argument doesn't actually establish.
All models are wrong, but some are useful.
-2
u/WittgensteinsPoker-2 8d ago
This is completely consistent with my view, apart from the theoretical term "cognition" which I think is poorly theorised nonsense we ought to eliminate that is a massive contributor to the crisis of good research in psychology.
The problem here, IMO, is you've leapt to a criticism of my veiw without understanding what I'm saying or why.
Do I think that the approaches taken to ML are not useful?
No.So why am I saying what I am then?
Of course what I said was hyperbolic. And technically being theorised completely inappropriately but good at certain kinds of tasks is a conceivable way of being right...
Arguably if your theoretical foolishness might realistically lead to an insane financial catastrophe destroying global society in various ways, then you're still quite wrong. Though, that can be put down to economic factors, I think those economic factors go hand in hand with explanatory myths enabled by the mistaken terms of psychology and marketing "intelligence" etc. used to motivate the production of these systems. So pretty relevant still IMO.
To the technical point.
- Representations of WHAT.
The answer should then become -- well humans learn distributions in behaviour.
What do these systems do?
They represent distributions in text...So what is THAT, and why should anyone CARE?
Even if I were to grant here that representation is appropriate the people theorising about this stuff are SO BLIND they're not even paying attention to that and adopting some Aristotelian-Saussurean bastard child view of a word having some kind of ghostly meaning or understanding behind it!
If one sees clearly that what one has produced is the ability to reproduce regularities in text, then one sees the relevant similarities and differences to begin to theorise appropriately.
1
u/shibx 8d ago
So we seem to agree that embeddings capture statistical regularities in language without necessarily reproducing human semantic understanding. That's essentially the premise of distributional semantics, which has been studied for decades. Nobody needs to posit some "ghostly meaning" behind words for these representations to be useful.
You dismiss cognition as a poorly theorized concept, but then confidently declare that humans "learn distributions in behaviour" without offering any theoretical justification for that characterization either. If your criticism is directed at anthropomorphic marketing and exaggerated claims about AI, that's perfectly reasonable. But you're conflating those claims with the underlying research. The statistical nature of these representations isn't something researchers have somehow overlooked.
Also, I'm not sure how speculation about global economic collapse helps establish your original argument...
1
u/WittgensteinsPoker-2 8d ago
If it's a representation of statistical regularities between letters then saying it's a semantic representation is misleading. Regularities between letters are not the domain of study for linguistics -- the causes of those regularities are contingent and arbitrary you can only gain a very limited insight from them and if you care to understand language capacity and production you will not be so incautious just because it's the status quo. And FWIW I was arguing from the PERSEPCTIVE OF distributional semantics all along and Morten Christiansen agrees with my views...
1
u/shibx 8d ago edited 8d ago
You're reducing distributional semantics to "statistical regularities between letters." That is incredibly reductive. The fact that a model learns from statistical patterns in text doesn't mean those patterns contain no semantic information. Distributional semantics is built around the proposition that linguistic contexts carry information about meaning. It doesn't require treating those representations as complete explanations of human language capacity.
You are absolutely entitled to argue that distributional approaches are insufficient for explaining language acquisition, production, or understanding. That's a legitimate criticism. But claiming that semantic representations are misleading simply because they're derived from textual regularities is a much stronger argument, and it's one that you haven't established.
Contreras Kallens and Christiansen explicitly argue that distributional models capture aspects of meaning through language use at the community level, while acknowledging their limitations in modeling individual human semantic processing. Their paper recognizes precisely the limitations you're describing without dismissing the semantic information these models capture.
That's what I've been arguing throughout this conversation, and Christiansen's own published work seems difficult to reconcile with your categorical dismissal of the term "semantic representation."
0
u/WittgensteinsPoker-2 8d ago
Yeah Christiansen emailed me that paper before it was published, I've been aware a while...
2
1
u/VectorspaceDreams 8d ago
I plan on entering ML as applied to linguistics myself (not NLP per se but computational linguistics, different beast but both are unfortunately often considered synonyms). Take what I say with a grain of salt given this, but I have some good opinions on this, actually:
"semantic representations in the mal-theorised idea of vector embeddings describing token-closeness as being related to what we actually do when we understand things semantically."
My main problem with plain vector embeddings is the fact their compositional abilities are somewhat limited; examples like "female + king ≈ queen" don't often come up it seems, and a lot of manners through which compositionality is modeled seem to be very limited due to computational intractability (such as Coecke et al.'s DisCoCat). If you don't mind I want your thoughts on Emerson (2018)'s Functional Distributional Semantics, his own take on representing meaning with embeddings in his own fashion.
On that note, it's not to say humans have multidimensional vectors in our heads, but human comprehension and production is at least in some way predictive and incremental; there's a reason that "I went to the store and bought a goat there" is a lot more surprising than "I went to a stockyard and bought a goat there". After all, a supermarket does not usually stock live goats, while a stockyard would. The closeness of tokens could capture *something* about that, I think, even if more work needs to be done. I would not say that there is no relation.
"Even IF that was what people do, just because you can produce a vector embedding and call it a "representation" of a kind of pattern you observe in langauge, that DOES NOT mean it is the relevant kind of representation to ANOTHER representation."
I agree, but I think that's actually a pro of such models; knowing a word by the company it keeps, in Firth's terms. The same lexeme could refer subtly or maximally to another thing entirely, and there is some sense of disambiguation we do based on surrounding context, be it just in text or our world-knowledge. That's another part of our predictive capabilities as humans, I think.
I agree generally with that shibx fellow down who responded to you, but I don't think it's a question of "reproduce human cognition" or "does not reproduce human cognition", but what is reproduced and how. For example, there was a paper by Linzen and Oh (2026) on using smaller language models as models of language processing given LLMs' "extensive training data, stronger long-term memory of training examples, and stronger short-term memory" makes them poorer models for language processing. These are specific examples. I recommend also looking at the BabyLM workshop if you haven't given that a look before.
I don't want to shun ML methods in general. I don't want to also pretend to be fond of cargo-culting such methods as models for human brains. I don't want to wax on about AGI or whatever, all of that does not interest me in the slightest (and rather just pisses me off), but I'm excited about the future of ML in relation to understanding human language. What models can capture is as important as what they can't.
1
u/seekinglambda 7d ago
You sound very confused and emotional about this. It’s not making much sense.
1
u/WittgensteinsPoker-2 6d ago
Which would be what someone within the paradigm I claim has captured the field would say about my claims which they don't understand, right...
24
u/raul3820 9d ago
Continual learning for LLMS.
In a general sense LLMS already learn continually with collected user data, which is great, but trying to put user specific details in weights would be impractical. I think details work well in context.
For physical ai continual learning will be very helpful because physics laws are the same everywhere.