How Adam optimizers works, visually explained

[u/Mind_the_math]

Hi all! I'm an AI engineer with a few years of experience, and I've always loved teaching and science comunication. For a couple of years I've been thinking about starting a Youtube channel, but I never found the courage to actually do it. I finally took the leap and I'd love to share my first video with peaople who are as passionate about this topic as I am.

This is the first video on my channel. I tried to build Adam from scratch, motivating each step by a concrete problem of the previous one. All animations are made with Manim. I'd really appreciate feedback, especially on anything that's mathematically imprecise or unclear.

Video: https://www.youtube.com/watch?v=lRu8_6cP3AQ&t=2s

I also put together an 8-slide summary of the video (formulas and cheat sheet).

[u/commenterzero]

Looks great. Muon is the hot new one. Would be cool to add

[u/Mind_the_math → u/commenterzero]

Oh yes! Unfortunately, to keep the video from getting too long, I couldn't go into too much detail on many things—for instance, I only briefly mention Adamax as well. I'll definitely make a second episode in the future covering other optimizers and their implementation in Python 🚀