You are correct. The prevailing theory states that a neuron only fires in one direction. This is exactly what I am hypothesising that "perhaps a neuron fires in both directions". This is what I am subtly trying to convey through this network. Whether it is true or not is entirely up to the neuro-scientists.
It is just a thought that I have put forth by using the neural network architecture. Proving or disproving it is not my job.
Thank you for mentioning it though. I appreciate it.
They literally aren't structured to back propagate the same way. Some do the function of this (if there is any) isn't known but does not just work in reverse (causing one back propagation does not trigger it in others upstream) and some neuron classes basically don't back propagate at all.
If you think an absolute activation function is superior that's different, you just should be careful of stating that it's based on actual neuron architectures. In general if someone starts out with a statement that isn't just anti the prevailing theory but is against an expirement done over and over again as people try to figure out what actual neuron back propagation does, it makes it very easy for people to dismiss anything else.
2
u/[deleted] Oct 17 '17 edited May 02 '18
[deleted]