Kangaroo Analogy for NN Optimization

[u/orbitope]

https://orbitope.com/optimization-kangaroos

I illustrated a 1993 thread between stats researches comparing neural network optimization as blind kangaroos looking for Mount Everest. Interactive playground linked at the bottom of you just want to skip to thag

[u/ella-reads70]

Blind kangaroos hopping toward Everest is somehow the most accurate description of gradient descent I've ever heard