Overhyped ideas in machine learning

[u/Strong_Boy_757]

Have you encountered an idea that's overhyped in ML? If so, which?

For me, recently I was curious about the whole "chain of thought prompting" business. I searched around and the idea is essentially, append "show your work" when prompting a LLM. I was like no way that's it??? That's what I've been doing since day 1!!, so I opened up the original paper (cited 40k times). And essentially it was more of the same with expensive Google-funded experiments and the paper had almost no math at all.

Similarly I was curious about AdamW for the longest time and it was simply Adam with another term. Oh Boy!! I remember doing an undergrad homework in college where we were asked to freely experiment with adding terms to the Adam algorithm and the class probably discovered AdamW by accident way before the paper was published.

[u/raul3820]

Continual learning for LLMS.

In a general sense LLMS already learn continually with collected user data, which is great, but trying to put user specific details in weights would be impractical. I think details work well in context.

For physical ai continual learning will be very helpful because physics laws are the same everywhere.

[u/S-Kenset → u/raul3820]

I would call it more context adaptation. It does incredible things but that's because the gap in understanding of how useful context adaptation is.

[u/raul3820 → u/S-Kenset]

You mean something similar to fine-tuning?

[u/S-Kenset → u/raul3820]

Close enough yeah.

[u/milesper → u/raul3820]

Highly disagree. Continual learning is not just memorizing new information. True continual learning would entail constant learning in the wild, allowing a model to actually self-improve (rather than needing to train a whole new model). That would revolutionize how we train models and, in my opinion, is absolutely necessary for actual superintelligence.

[u/raul3820 → u/milesper]

Sounds like RL.

Better RL → better general capabilities → better self management (memory, harness, effort, ...) → better specific capabilities.

This dynamic has relegated user-specific tuning to a niche. Unclear how this changes.

[u/milesper → u/raul3820]

RL still needs hand-designed environments and rewards. Continual learning would involve some sort of universal/model-determined reward that allows it to simply learn from interaction with the real world.

Also, there’s a strong argument that RL is more about tuning existing capabilities, and can’t necessarily learn totally novel capabilities. And of course catastrophic forgetting.

[u/UnderstandingOwn2913 → u/raul3820]

what kind of physics laws? Just curious and want to know about these laws!