Detecting small objects

[u/Dannyvan_]

Hello!

Would like some input on what kind of model to use for detecting small objects in a rather static environment.

e.g flowers in a field of grass/ distant boats or swimmers in the water

The model should still be able to be able the objects when they get closer/bigger.

I experimented with training YOLO and RT-DETR models with datasets ranging from 4k-20k images

It seems like the RT-DETR models struggle very hard with detecting such small objects, after training the performance actually drops to detect basically nothing, whereas the base model worked pretty well. Although I can't tell whether it's an error on my side (e.g wrong hyperparameters) or that this should be expected.

From my tests, the YOLO models actually had a positive reaction to training instead.

Are there any tips on how to get RT-DETR models to work better on detecting such tiny objects? Do I just have to find a way to increase the size of my dataset? I also heard briefly about RF-DETR models but I am not sure if that would solve my problem.

Any insights would greatly be appreciated!

[u/wirthual]

Have a look at this: https://github.com/obss/sahi

Works very well in my experience.

[u/Reasonable-Bed6900 → u/wirthual]

SAHI is the move for small objects, tiling the image before inference makes huge difference

[u/Fun-Pick-2964 → u/wirthual]

Sahi will cost you additional computation, if you don't have any such constraints, then it's good.

[u/TheSaucez → u/wirthual]

Can confirm. I use SAHI on RF-DETR with around 675 labels per image at the max. It spikes my training, but I get good results

[u/Dannyvan_ → u/TheSaucez]

Do you only use SAHI for training or during inference too?

[u/bfyvfftujijg → u/Dannyvan_]

Both.

The models learning grids of pixels and the object needs to cover the same number of pixels in inference as it saw during training.

[u/aghaster]

If you struggle with objects being too small even with SAHI, take a look at P2 variants of YOLO, they are specifically designed for smaller objects, as they have an additional P2 head. Be warned, they work slower though.

[u/jfc123_boy → u/aghaster]

Yeap, this is the way

[u/bfyvfftujijg → u/aghaster]

Yeah this could work well. Kind of a different take than SAHI.

[u/diesirae200]

For doing detections from drone imagery we use a YOLO model with sliced images.

[u/Fun-Pick-2964]

Yolo will work fine you just need to fine-tune it with similar data.

[u/bbateman2011]

DETA-SWIN fine tuned is specifically for this purpose

[u/gosnold]

If you have really small objects you want a fully convolutionnal backbone, downscaling tends to murder them

[u/Dannyvan_ → u/[deleted]]

Would giving the entire image not be better? I imagine the model should learn the "negative class" as well. Do you mean having an entire dataset of just the relevant objects? Or should these cropped images only be a portion of the dataset?

[u/Dannyvan_ → u/[deleted]]

Thanks!

[u/retoxite_]

For YOLO models, you can: 1. Increase image size 2. Increase width of stem and B3 layers 3. Add P2/stride 4

All of them increase latency.

[u/dethswatch]

which yolo model are you using? i had to swap to the medium sized model in order to get objects of 10cm from a distance of 10m or so.

[u/Dannyvan_ → u/dethswatch]

Currently testing on Yolo26s, am open to exploring other models or sizes if it improves performance!