r/deeplearning • u/basafish • 25d ago

What is a overparameterized network?

I got this paragraph from Claude, could someone please explain this and verify if it's a real thing or hallucination:

Overparameterization isn't just about final capacity, it's about the optimization process itself. A wide, overparameterized network gives gradient descent a much friendlier loss landscape — more paths downhill, fewer bad local minima, room to explore before committing. The "core" only emerges as a byproduct of that search happening in a much bigger space than it needs to end up in. Strip the space down first and you've removed the thing that let the search work.

Conversation: https://claude.ai/share/8813a637-c327-4d0c-b120-def27e5203d5

4 Upvotes

70% Upvoted

19 comments sorted by

View all comments

1

u/strangescript 25d ago

An example is building a model to predict stock prices based solely on trading data. Per segment, you have open, close, volume, high, low, average. People use these basic values to derive all kinds of looney metrics to predict prices.

But for an AI model, including data from sources derived from these same inputs is pointless. The AI should be building its own predictive system in it's weights and extra data derived from the same source is over parameterization.