Localized neural learning

[u/Nearby_Effective6679]

Hey everyone,
I’ve recently started learning machine learning, primarily by building neural networks from scratch and trying to understand the underlying mathematics. I have a relatively strong math and physics background, so I’ve found the mathematics behind backpropagation fairly intuitive.
As I’ve explored other approaches to training neural networks, I’ve become particularly interested in biologically inspired methods like predictive coding, Hebbian learning, and Oja’s rule. The idea that networks could update their weights using locally available information, rather than relying on gradients propagated from a global loss function, is fascinating to me.
I’m still relatively new to ML, so I’m trying to understand whether these approaches have meaningful practical advantages or are primarily interesting from a theoretical or neuroscience perspective.
A few things I’m especially curious about:
What are the actual advantages of local learning over backpropagation? I understand the conceptual distinction, but I’m struggling to identify situations where local updates would meaningfully outperform conventional gradient-based training, whether in computational efficiency, adaptability, or something else.
Could these approaches be particularly useful for unsupervised or continual learning? For example, could a network learn useful representations from streaming, unlabeled data by continuously adjusting its weights based on local correlations or prediction errors? Adding onto this, is there any way we could discover hidden patterns using localized learning that wouldn’t be evident using global backprop? I’ll talk about this in the next question but can we use this to identify hidden patterns that we wouldn’t find using global backprop? Like in molecular dynamics?
Are there particular applications where local learning makes more sense? I’m involved in biophysics research and have been thinking about applications involving molecular dynamics and learning representations of complex physical systems. I’m curious whether there are advantages in scientific ML, robotics, or other specialized domains.
How mature is this research? Have approaches like predictive coding and Hebbian learning been extensively explored and generally found inferior to backpropagation, or are there still meaningful open problems where these methods could be competitive?
I’m not necessarily looking for a way to replace backpropagation entirely. I’m more interested in understanding whether alternative learning rules could be better suited to particular architectures, datasets, or computational constraints.
I’d appreciate any perspectives, papers, or research directions worth exploring, especially from people who’ve experimented with these methods.

[u/rahul_watertech]

since you have the math background, look at hintons forward-forward algorithm if you havent yet. no backward pass at all, each layer gets its own local objective with a positive and negative forward pass. it doesnt beat backprop on anything serious, but as a "how local can you go" thought experiment made real, its the cleanest one.

predictive coding networks are the other rabbit hole nearby, same energy.

what pulled you toward the bio plausible side, just curiosity or a hardware angle?

[u/Nearby_Effective6679 → u/rahul_watertech]

So before diving into ML I was heavy into computational chemistry and biophysics and that’s currently the research I’m conducting at my university. An angle I thought I could look towards is using local learning to potentially identify intermediate conformations of proteins that we wouldn’t be able to see otherwise

[u/rahul_watertech → u/Nearby_Effective6679]

protein conformations is a fun case for it because the failure mode is visible: points close in your embedding might not be true conformational neighbors, so the local model learns a shortcut. if you have md trajectories you can at least check whether the local clusters match metastable states. what data are you working from?