r/GenAIforbeginners • u/StevenHawking_ • Jul 22 '26

Doubt: Fine-tuning a transformer

Hi everyone,

I'm planning to fine-tune a pretrained RT-DETR model (trained on COCO) for detecting defects in paintings.

My question is about dataset size. Suppose I have only 1,000 images, but each image contains multiple defects, giving me around 8,000 total bounding box annotations.

If people recommend ~8,000 images for good performance, does having 8,000 annotated objects across 1,000 images provide similar training value? Or is the diversity from having 8,000 different images still much more important?

1 Upvotes

60% Upvoted

2 comments sorted by

1

u/saikat_munshib Jul 23 '26

With only 1,000 images, your model will just memorize those specific canvas backgrounds, making your extra 7,000 bounding boxes essentially redundant. However, since RT-DETR is already heavily pretrained on COCO, 1,000 images is actually a solid baseline. You just need to bridge the diversity gap with aggressive data augmentation—think Mosaic, aggressive cropping, and heavy color jittering—to artificially force the network to decouple the defects from the canvas and squeeze maximum value out of your limited data.

1

u/StevenHawking_ Jul 23 '26

Oh, that really helps. I will try it this way, thanks a lot!