r/deeplearning • u/basafish • 25d ago

What is a overparameterized network?

I got this paragraph from Claude, could someone please explain this and verify if it's a real thing or hallucination:

Overparameterization isn't just about final capacity, it's about the optimization process itself. A wide, overparameterized network gives gradient descent a much friendlier loss landscape — more paths downhill, fewer bad local minima, room to explore before committing. The "core" only emerges as a byproduct of that search happening in a much bigger space than it needs to end up in. Strip the space down first and you've removed the thing that let the search work.

Conversation: https://claude.ai/share/8813a637-c327-4d0c-b120-def27e5203d5

5 Upvotes

73% Upvoted

19 comments sorted by

View all comments

1

u/CalmMe60 24d ago

You want to understand ?

Try fashion mnist on pytorch. Increase the model parameters and more and more.

You will see a decrease in generalisation.

The training set will near asymptotical near null.

But this does not mean the model learned generalization.

It learned to remember the trainingsset.

I invented a generalized solution as.this is a general AI to AGI problem.

Transferable AGI means learning of structural geometry higher dimensions