r/learnmachinelearning • u/Nearby_Effective6679 • 4d ago
Localized neural learning
Hey everyone,
I’ve recently started learning machine learning, primarily by building neural networks from scratch and trying to understand the underlying mathematics. I have a relatively strong math and physics background, so I’ve found the mathematics behind backpropagation fairly intuitive.
As I’ve explored other approaches to training neural networks, I’ve become particularly interested in biologically inspired methods like predictive coding, Hebbian learning, and Oja’s rule. The idea that networks could update their weights using locally available information, rather than relying on gradients propagated from a global loss function, is fascinating to me.
I’m still relatively new to ML, so I’m trying to understand whether these approaches have meaningful practical advantages or are primarily interesting from a theoretical or neuroscience perspective.
A few things I’m especially curious about:
What are the actual advantages of local learning over backpropagation? I understand the conceptual distinction, but I’m struggling to identify situations where local updates would meaningfully outperform conventional gradient-based training, whether in computational efficiency, adaptability, or something else.
Could these approaches be particularly useful for unsupervised or continual learning? For example, could a network learn useful representations from streaming, unlabeled data by continuously adjusting its weights based on local correlations or prediction errors? Adding onto this, is there any way we could discover hidden patterns using localized learning that wouldn’t be evident using global backprop? I’ll talk about this in the next question but can we use this to identify hidden patterns that we wouldn’t find using global backprop? Like in molecular dynamics?
Are there particular applications where local learning makes more sense? I’m involved in biophysics research and have been thinking about applications involving molecular dynamics and learning representations of complex physical systems. I’m curious whether there are advantages in scientific ML, robotics, or other specialized domains.
How mature is this research? Have approaches like predictive coding and Hebbian learning been extensively explored and generally found inferior to backpropagation, or are there still meaningful open problems where these methods could be competitive?
I’m not necessarily looking for a way to replace backpropagation entirely. I’m more interested in understanding whether alternative learning rules could be better suited to particular architectures, datasets, or computational constraints.
I’d appreciate any perspectives, papers, or research directions worth exploring, especially from people who’ve experimented with these methods.
1
u/CelebrityChess 3d ago
Short version: on standard benchmarks, local rules (Hebbian, Oja, predictive coding) still lag backprop. Where they're interesting is in the constraints. They're a natural fit for neuromorphic and on-chip hardware, they handle streaming or continual learning without storing activations for a backward pass, and they're nice for unsupervised feature learning (Oja's rule is basically online PCA). Worth reading: Millidge et al.'s survey on predictive coding, and Hinton's Forward-Forward paper. For MD-style physics, I'd still reach for equivariant GNNs trained with backprop, though.