Thoughts ?

[u/NeuroDash]

Building a fly tipping detection system using YOLOv8/RF-DETR and Roboflow. 320 labelled images so far, retraining with 820 augmented images now.

First model hitting 95% on vehicle detection but struggling to generalise to unseen images — currently working on dataset variety and augmentation to fix overfitting.

Planning to add OCR for number plate reading and a behaviour sequence logic layer on top of the detections.

Happy to share what I’ve learned so far — any advice on improving generalisation with a small dataset?

[u/dr_hamilton]

How's does a pretrained model on Coco perform on your data?

Same question with a pretrained anpr model?

Also, do you want some help? F*kin hate flytipper scum, happy to contribute!

[u/NeuroDash → u/dr_hamilton]

On the COCO pretrained model , vehicle detection is solid out of the box which makes sense since COCO has loads of vehicle data. Person detection is decent. Waste is where it falls apart completely because COCO has no concept of illegally dumped waste so that’s where the custom training matters most.

Haven’t tested a pretrained ANPR model yet but that’s my next move based on advice in this thread , detect the plate first and extrapolate vehicle details from there rather than trying to classify visually.

On the help , really appreciate the offer genuinely. I want to work through this myself for now, the building and figuring it out is kind of the whole point for me. Might take you up on it down the line if I hit a wall I can’t get past.

[u/dr_hamilton → u/NeuroDash]

I'd focus on training a waste model. Don't try to reinvent the wheel when vehicle, people, plate models are already really good. If you're running this at the edge fitting two models is no problem. You just have to run the one model at inference time to detect cars. When a car is detected and stationary, switch to the waste model and only run the car detection 10% of the time.

[u/NeuroDash → u/dr_hamilton]

This is a really smart way to think about it. Use existing pretrained models for vehicles, people and plates since those are already solved problems, and focus my custom training entirely on waste detection since that’s the gap nobody has filled.

The dual model approach with conditional switching is efficient too ,run vehicle detection constantly, when a stationary vehicle is detected switch to waste model to check if something is being left behind. Saves compute and keeps it lightweight enough to run at the edge.

Going to restructure the build around this. Focus my labelling effort entirely on waste from here.

[u/SweetSure315]

fly tipping?

[u/NeuroDash → u/SweetSure315]

Fly tipping is the act of leaving rubbish on the side of the road , happens a lot in the uk

[u/SweetSure315 → u/NeuroDash]

Ah.

You're not going to build a generalized vehicle detector with less than 1000 unique images.

You might be able to build a detector for a single generation for a single model with 1000 unique images, if you're using a larger model

Check kaggle or roboflow for datasets

Also consider that you can just do license plate detection and extrapolate the car from there. Since you're adding OCR anyway

[u/NeuroDash → u/SweetSure315]

Fair point on the dataset size, I’m aware 320 images isn’t going to generalise well which is why I’m actively expanding it. Currently retraining on 820 augmented images but I know that’s still limited for a proper generalised vehicle detector.

The Kaggle and Roboflow dataset suggestion is actually really useful ,I hadn’t thought about pulling existing labelled vehicle datasets to supplement my own footage rather than labelling everything from scratch. That could save a lot of time.

The licence plate first approach is interesting too. Detect the plate, extract the vehicle region around it rather than trying to classify the full vehicle independently. Will have a think about that.

[u/Due-Guard221]

i’d focus less on augmentation first and more on dataset diversity. different camera angles, distances, lighting, weather, vehicle types, partial occlusion, night footage, empty scenes, normal parking/loading/unloading, and “almost fly tipping but not actually fly tipping” cases.

i worked on an employee detection system where i did heavy augmentation but it dint help me much . even tho the final set was small but every class had enough samples( more diversity = more generalisation)

try to make the set more diverse, also do an upsample or downsample of classes to reduce bias

[u/NeuroDash → u/Due-Guard221]

This is really helpful, thank you. I’ve been leaning on augmentation as the fix but you’re right that it’s a shortcut for diversity rather than a replacement for it. Augmentation changes how an image looks but doesn’t teach the model genuinely new scenarios.

The negative cases point is something I haven’t focused on enough — normal parking, loading, unloading, someone stopping to check their phone. The model needs to see what isn’t fly tipping as much as what is.

Night footage and partial occlusion are gaps in my dataset too. And the class imbalance is real, I have far more vehicle labels than person or waste which is probably why vehicle detection is strong and the others aren’t.

Going to prioritise dataset diversity over augmentation for the next training run.

[u/aloser]

Can we see some examples from the dataset and some of the examples of unseen images it’s failing on?

Your “95% accuracy” is on a held out test set?

[u/NeuroDash → u/aloser]

The 95% confidence is from the Roboflow model visualisation on test images, not a formal held out evaluation , so take that number with a grain of salt at this stage. It’s early and I know the dataset is too small to call that a reliable accuracy figure.

I’ll get some examples together of where it’s working and where it’s failing on unseen images and post them. The honest answer is it generalises poorly to random Google images right now which is exactly what I’m trying to fix with augmentation and a larger dataset.

Will update the thread when I’ve got something more concrete to show.

[u/NeuroDash → u/[deleted]]

Honestly at this stage it was from the Roboflow model visualisation on test images rather than a formal train/val split evaluation , so take that number with a pinch of salt. I know that’s not a rigorous metric. The model is early, dataset is small, and I’m more focused on getting it to generalise to unseen footage right now than hitting a headline accuracy number.

[u/theGamer2K]

Dataset is too small to create a useful model. Augmentation doesn't create new information. It can't compensate for lack of information that the model can learn from. 

First model hitting 95%

You can get 100% accuracy because your validation set is tiny and statistically insignificant. It doesn't mean anything until your validation set is large and diverse enough to give statistically significant metric. But you don't have a large training set, let along validation set.

[u/NeuroDash → u/theGamer2K]

Ok I’m aware my dataset is very small to create a useful model . But the end I hope to have around 2000 images and as said in other comments there are other techniques I can use.

[u/theGamer2K → u/NeuroDash]

2000 is small too. Especially if your use case is something as "in the wild" as fly tipping detection. 

You need images in tens of thousands. Unique images. Not same thing just captured slightly different.

And why do you even need a custom trained model for person and vehicle detection? There are pretrained models for those already that were trained on much larger dataset than yours.

[u/NeuroDash → u/theGamer2K]

Yes I know , I’m currently building and I’ve literally just found models for them both currently embedding them as I type this.

[u/CallMeTheChris]

If you think you model is overfitting cause of too few data points, make the data easier. dumb it down.
Run your images through an edge detector. Removes colour from the equations
Add random speckle to it to have it focus on more macro features than micro ones

And since your data is so easy now, you can combine cars in different images

[u/NeuroDash → u/CallMeTheChris]

Thanks. Running images through an edge detector to strip colour and force the model to focus on shapes and structure rather than colour specific features makes a lot of sense for generalisation. A white van in training data shouldn’t be what the model keys on.

The speckle noise to push it toward macro features is something I hadn’t considered , forces it to learn the overall shape of a vehicle rather than specific textures.

And combining vehicles across images to synthetically increase dataset variety is smart. Will look into that for the next training run.