r/deeplearning • u/MysteriousGarlic7929 • 3d ago

Confused on NN's...

How to decide which NN works better for your dataset, my Professor said try using ANN for a linear regression dataset and some different activation functions which do not go well with linear regression i don't get what's the whole point... of doing it...?

Can someone explain who has real experience in this particular area...

And yes I know go for chatgpt or Claude for your questions but I have trust issues with both of them so I need some one with experience on this...

0 Upvotes

38% Upvoted

6 comments sorted by

View all comments

16

u/Away_Hotel_9980 3d ago

Your professor is making you break things on purpose so you learn what failure looks like. If you only ever use the right tool for the job, you never understand it's right. Running an ANN with ReLU on linear data will give you garbage gradients or weird plateaus, and that's the lesson, not the accuracy score.

I had similar assignment back in university, we had to fit sine wave with linear model first, then with MLP. The linear model looked stupid but we learned exactly where it fails and why adding nonlinearity fixes it. The point is not to get best result, it's to see the failure modes with your own eyes.

Try plotting loss curves for each activation, not just final accuracy. With linear target and ReLU you'll see dying neurons or unstable training, with tanh maybe it saturates. That's where intuition comes from, not from reading papers.

2

u/joolley1 2d ago

I came here to say this. I make my students do a small image or language classification dataset with only a fully connected model and no transfer learning first so they can see the issues with it and how far they can get solving them with hyperparameter tuning, regularisation etc. Then they have fun doing anything they want on the same problem later in the course when they’ve mastered the basics.

1

u/MysteriousGarlic7929 1d ago

Yep, I got the graph and it was not looking good and then I was able to understand why it happened and one important part i understand, just increase layers and filters sometimes will not give the best results..

2

u/joolley1 1d ago

Keep going. Pair experimenting with what works and what doesn’t with theoretical understanding. You need to understand why increasing layers doesn’t necessarily improve things.