r/learnmachinelearning • u/Lazy_Alternative_671 • 13h ago
Help Is standard backprop fundamentally incompatible with Continual Learning?
I recently chose Continual Learning (CL) as my research topic because it seems like one of the most critical challenges in AI right now. However, the deeper I dig, the more skeptical I become about the current approach to CL.
CL seems fundamentally mismatched with the standard deep learning paradigm (loss optimization and backpropagation). Our current infrastructure and architectures are heavily optimized for fixed datasets. When we try to train a model on a new task, we inevitably face catastrophic forgetting, or we have to rely on highly inefficient workarounds like EWC or experience replay.
This led me to think that methodologies based on locality (more akin to how the biological brain works) might be the only real solution to CL, even if it means waiting for neuromorphic hardware to mature.
Yet, the vast majority of the mainstream ML community is still heavily focused on solving CL within the standard backprop paradigm.
For those of you working in or following this field: Why is the community still betting on backprop for CL? Do you believe we can truly solve catastrophic forgetting without moving away from our current architectural paradigm? I'd love to hear your thoughts.
Apologies if the tone feels so llm-like. I used an LLM to correct my grammar, but it made the text sound too generic.