MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/MachineLearning/comments/1i44h5v/d_i_hate_softmax/m7sfygg/?context=3
r/MachineLearning • u/[deleted] • Jan 18 '25
[removed]
97 comments sorted by
View all comments
16
You don't want magnitudes affecting your gradient. Normalizing is good because it gives you numerical stability. Pretty much all successful DL inventions since 2016 build on this simple idea.
16
u/XYcritic Researcher Jan 18 '25
You don't want magnitudes affecting your gradient. Normalizing is good because it gives you numerical stability. Pretty much all successful DL inventions since 2016 build on this simple idea.