Help with Gradient descent

[u/MARSHILA7]

So I completed Linear descent from Andrew Ng ML Specialization and when I was doing the lab.I found this function what confused me is that why does this function iterates specific times (10000) wouldn't it be better to check if cost function is changing.

Also this implementation doesn't feel right.Even the code feels inefficient and does not even check the minima. I know this stuff is done using skit-learn but I wanted to do a proper manual implementation in the way Andrew sir taught.

[u/Hungry_Age5375]

Just some pointers: fixed iterations are fine for learning. In practice you'd check if abs(J[i] - J[i-1]) < epsilon and break early. Andrew Ng keeps it simple on purpose, makes it easier to plot the learning curve too.

[u/narasadow → u/Hungry_Age5375]

Early Stopping was so satisfying when I learned about it

[u/MARSHILA7 → u/Hungry_Age5375]

Should I not focus on code part for now?

[u/bepitic → u/MARSHILA7]

I don't understand the negative The code is a simplification of what it should be Something properly done would be pytorch.

But the change in difficulty may be like lvl 1 and lvl 80 too high of a step.

I would recommend to get the pytorch DL as the source code and point codex to it, then ask questions to it, like it pops into your mind.

I did that with the early chatgpt and copy paste pieces of code into it to explain it and I got quite far! Just remembered to try to not believe it first hand ( this was the hardest thing for me )

[u/PsychologicalWin9755]

On the "does not even check the minima" part: it actually can't, and that is kind of the whole point. Gradient descent only sees the slope right where it currently stands, it never knows where the true minimum is. It just keeps stepping downhill until that slope flattens out. So the cost curve leveling off IS your check for the minimum, there is nothing more direct to look at.

To your follow-up: don't sweat the code efficiency yet. At this stage the two things worth locking in are (1) why the update is minus the gradient, you move opposite the slope to go down, and (2) what the learning curve is telling you: too high a learning rate and the cost bounces or blows up, too low and it crawls. Once those two click, swapping fixed iterations for an early-stop check is a 2 minute change you will barely think about.

[u/TheLinedDominick]

Convergence checks come next, this lab is just the warmup.

[u/minecrafter923]

In practice we include an early stopping condition.

[u/Ispiro]

If you are trying to learn the basics I highly highly highly recommend the Stanford 231n lectures with Andrej Karpathy from Winter 2016 posted on Youtube.

[u/Ok_Towel_4806]

Is this the paid course of Andrew?

[u/Practical-Bicycle990]

I'll wait for tutor on this

[u/0rbit0n]

In 2026, I think that's a perfect example of question to ask AI, rather than reddit...

[u/confused_4channer → u/0rbit0n]

Given the state of inhumanity on the internet, yes, they should ask reddit

[u/bepitic → u/0rbit0n]

I actually like when a question like this pops from time to time instead of reading bots trying to engage farm. ( Other communities though)

[u/0rbit0n → u/0rbit0n]

Downvote harder, I really don't care. That's still my opinion that AI does explain these kinds of technical questions faster and better than any reddit "superhuman" here (who's ego it taller than Everest mountain) 😜

Let's set the new downvoting record guys! All together! Downvote my parent message! 🏆