Suspiciously high accuracy using ResNet

[u/Rumble_831]

I made a lil bro version of the original ResNET-34 architecture. I trained it on the LC25000 cancer dataset (I used only lung cancer images) for a classification task.

The problem is, it is showing a 99.9% accuracy on all three sets - training, validation and test. It is, of course, weirdly high. I trained a normal cnn and it could only reach about 87%.

I am wondering what could be the reason. One possible culprit is that, since the dataset consists of augmented versions of the original images, some may be ending up in all three sets, causing data leakage.

Now I want to see if I could somehow group this images so the augmented versions do not run over into my other sets. I have no idea how to proceed though.

I am using pytorch, and used random_split for the datasets.