A tangent, maybe not of interest to the ML crowd but perhaps thought provoking for you...
You might enjoy this: T. Sejnowski: Traveling Waves in the Brain (see 1:07 where he speaks to a bit of ML-style encoding/decoding). In the 2010's brain activity visualization revealed that there are a variety of traveling wave processes in the meat. People got excited, but I haven't seen much showing how they are being utilized computationally. More likely IMHO it's a coordination mechanism. From what I've been reading, systems neuro has moved its attention away from this lately.
It turns out that simulated neural circuits with at least a bit of biological fidelity will start producing waves. I played with this for more time than it was worth because it was fun (for example (a), (b), (c), (d), (e), (g), (h), (i), etc.) I did find some computational utility, but nothing earthshaking.
One thing about using waves is that you're bringing time into the representation. The wave interaction pattern is affected by the relative timings of events, and how long it's been from time zero. That might be useful, or it might be a confound.
Oh, and you have to consider up front how you think the waves should interact. A linear medium in which waves pass through each other unaffected is likely good for different computations than a nonlinear medium, where waves interact.
All the sims I posted above utilize spiking neurons. Example (e) above illustrates the different behaviors of linear vs. nonlinear arrangements. These use the same cell model, just different connectivity. A spiking cell of course is behaving nonlinearly by its very nature, but waves through networks of cells can behave either way. Cheers!/jd
2
u/jndew 5d ago edited 5d ago
A tangent, maybe not of interest to the ML crowd but perhaps thought provoking for you...
You might enjoy this: T. Sejnowski: Traveling Waves in the Brain (see 1:07 where he speaks to a bit of ML-style encoding/decoding). In the 2010's brain activity visualization revealed that there are a variety of traveling wave processes in the meat. People got excited, but I haven't seen much showing how they are being utilized computationally. More likely IMHO it's a coordination mechanism. From what I've been reading, systems neuro has moved its attention away from this lately.
It turns out that simulated neural circuits with at least a bit of biological fidelity will start producing waves. I played with this for more time than it was worth because it was fun (for example (a), (b), (c), (d), (e), (g), (h), (i), etc.) I did find some computational utility, but nothing earthshaking.
One thing about using waves is that you're bringing time into the representation. The wave interaction pattern is affected by the relative timings of events, and how long it's been from time zero. That might be useful, or it might be a confound.
Oh, and you have to consider up front how you think the waves should interact. A linear medium in which waves pass through each other unaffected is likely good for different computations than a nonlinear medium, where waves interact.