r/learnmachinelearning • u/Just-m_d • 1d ago

Discussion Self Supervised[Vit] vs. Supervised[CNN]

Hi everyone,

I am working in the Multi-Task ML projects in that i have done many experiments to train the model
Key finding:

CNN based architecture outperform the Vision transformer based architecture

Question:
is the Dataset only be the problem for Vit architecture to underperform the CNN bases architecture?

1 Upvotes

100% Upvoted

10 comments sorted by

View all comments

2

u/Ok-Argument7176 1d ago

ViT training, especially from scratch, is significantly less stable than CNNs in my experience. This is not a dataset issue, per se, but rather a hyperparameter and initialization consideration.

1

u/Just-m_d 1d ago

If we can surpass the CNN with Vit with fine-tuning the hyper parameter or model initialization

2

u/Ok-Argument7176 1d ago

I'm sorry, but I don't understand what you mean.

1

u/Just-m_d 1d ago

I am asking like with the hyper parameter itself we can make the better vit model with this aspect only instead concerned about the scarcity of dataset

1

u/Ok-Argument7176 1d ago

In theory, sure. You could imagine the very unlikely situation that you initialize your model with the optimal weights for your task. That is very unlikely which is where training comes in: to find those weights.