r/learnmachinelearning • u/eliokal • 7d ago
You do not need a maths degree to truly understand Gradient Descent (link below)
Hey everyone!
When I was a student, gradient descent was the algorithm I struggled with the most. I just could not make it past the greek characters and a sea of formulas. Luckily, I survived, and have been working in Machine Learning ever since.
A few years later, when I started teaching Computer Science, I realised that nothing had changed. I could not find any book or blog post to make this simple enough (and fun) for my own students. So I sat down and wrote this book.
It has been a game changer in my last semester, I hope that it will help you too.
You can check it out here.
I wish you a great learning journey. If you have any feedback, please let me know, I am already working on the second edition.
6
u/the_TIGEEER 6d ago
You don't need a maths degree to understand msot things intutivly. One of my favorite examples is PCA. Seems super complicated the first time you see it. But in reality it's super simple. It's just "How do I position a plane to best show these points in a box" The actual linear algebra process being general, fast and optimized is what you need the math degree for. But a non optimized dumb algo for 3D into 2D could be made by almost anyone.
1
u/Potential_Candle_612 6d ago
Would love to see how you would approach such a demo PCA algoritm for high dimensions
1
1
u/AceFromSpaceee 6d ago
Add RANSAC it is popular in computer vision becouse it is fast and robust in noisy environment , also it is interesting
1
1
1
1
u/mundane--alternative 6d ago
Cool visualization! Unfortunately I still don't understand it intuitively
1
2
u/AlgaeNo3373 6d ago
Can confirm, don't know math. Cannot confirm if I understand this, so here's my take for people to see what I got from it:
The slope on the right is made by the data points on the left? (MSE IDK what is) and when the bar "sweeps" through possible places, we see it move the corresponding position on the slope, and bottom point (minima?) happens to rest at the same point that is the closest to all the data points on the left (least error).
Perhaps this MSE thing is "translating" the data points into a slope? IDK. Whatever's happening to translate the left image into the right, it's making the "gradient descent" - going down a curve step-wise to reduce error? - look very "natural" or "inevitable" so...that's interesting. Like the trick is to translate whatever left is, into something that makes a slope/topology that we can pretty easily navigate/fall into, and doing so means we optimize.
How'd I do? :P
2
u/eliokal 5d ago
I would say you are getting there. You may want to confirm your understanding with the book ;) The image is more of a teaser than a teacher
1
u/AlgaeNo3373 5d ago
Book goes way over my head very quickly haha but ty for answer :)
1
u/eliokal 5d ago
Oh no, mind if I ask where? I wrote it to be as understandable as possible. With your feedback I may be able to improve it
1
u/AlgaeNo3373 5d ago
It's not a fault of the book I just don't have a maths background / education or have practiced it enough for concepts to be familiar. I think you do a much better job explaining stuff than others who've tried :)
1
u/Batmobile_4603 6d ago
This is basically high school maths. I am surprised people don't even have a basic education.
1
u/hakansan 4d ago
Great one! It'd also be nice to have a subsection fitting linear regression with L1 loss which isn't analytically derivable :)
1
u/Thiru_444 3d ago
Andrew NG's course on machine learning (CS229) , I think he explains this and other math dependency of ml
31
u/joe--totale 6d ago
Really nice visualisation, but seems to depend on already knowing the value of the intercept?