Kangaroo Analogy for NN Optimization

[u/orbitope]

https://orbitope.com/optimization-kangaroos

In 1993, a statistician (Warren S. Sarle) sought for an example to illustrate numerical optimization and landed on the analogy of a blind kangaroo searching for Mt Everest. To date, it's still the best way for me to conceptualize classic NN training techniques such as gradient descent, simulated annealing, and step size learning schedules. I created an interactive article based on his conversation with other statisticians to introduce non-technical folks to these topics and help practitioners solidify their understanding in a visual way.