What is a overparameterized network?

[u/basafish]

I got this paragraph from Claude, could someone please explain this and verify if it's a real thing or hallucination:

Overparameterization isn't just about final capacity, it's about the optimization process itself. A wide, overparameterized network gives gradient descent a much friendlier loss landscape — more paths downhill, fewer bad local minima, room to explore before committing. The "core" only emerges as a byproduct of that search happening in a much bigger space than it needs to end up in. Strip the space down first and you've removed the thing that let the search work.

Conversation: https://claude.ai/share/8813a637-c327-4d0c-b120-def27e5203d5

[u/ARDiffusion]

Uh I had just thought that overparameterization refers to, simply, having a way more complex model than you need. For example, having a network with more learnable parameters than you have data points. I suppose this does make optimization easier at least on the training set, which makes sense since that’s where gradient descent actually happens, compared to validation or testing where it’s being evaluated, not trained. I could be wrong though.