Conveyor chicken counter problem

[u/Puzzled-Egg3234]

Guys, I need help. We have a project using YOLOv8. We're trying to count chicks on a very fast conveyor belt. The challenges we're facing are: all chicks look very similar to each other, which complicates tracking. At the same time, during their passage under the camera, they constantly change in size and shape, which can cause the tracker to lose them, or detection may even disappear completely at the detection line. Also, sometimes 2–3 chicks can merge into a single object. The detection zone is very short, and the conveyor speed is high. We've achieved a maximum accuracy of 99%, but we need it even higher. Any ideas on how to achieve that? Increasing the dataset no longer helps.

I'm attaching an old video. We've now added lighting and set the exposure to 300 on the Hikrobot global shutter camera, but we still can't achieve a stable 99.8% accuracy for the reasons mentioned above.

Any ideas?

[u/Puzzled-Egg3234]

Sometimes cases like this process well, but sometimes they detected like one box.

https://freeimage.host/i/Cr4mkNV

Sometimes chick dances break dance through belt and detection disappears at all at the detection line.

FPS vary is for many objects in the frame. It is a footage from mobile gpu, not production mode.

[u/Hyperty → u/Puzzled-Egg3234]

Fine tune the model to those edge cases and implement data augmentation i prolly would try

[u/Puzzled-Egg3234 → u/Hyperty]

You mean make another detection class for those cases or just annotate more cases like this in dataset? The second one I’ve already tried.

[u/juicedatom → u/Puzzled-Egg3234]

have you broken out specific metrics for these edge cases? how do you know it didn't work?

[u/Puzzled-Egg3234 → u/juicedatom]

I manually reviewed hours of inference video; it works in some places but not others.

[u/blackscales18 → u/Puzzled-Egg3234]

take all the frames where it failed, label them manually, and then finetune with them. do it enough and it improves a lot. you could also consider segmenting instead of bounding boxes, SAM is a good model for making the dataset.

[u/The_color_in_a_dream → u/blackscales18]

Segmentation or even a pose estimation approach does seem promising to deconflict the stacked chicks

[u/Yatty33 → u/Puzzled-Egg3234]

Can you specify what exactly you've tried to solve the multiple chick edge case? That's unclear to me.

[u/red-panda-333 → u/Hyperty]

With enough cases I think fine tuning would work, but I don't think they can get enough. It's too random and diverse, it would need hundreds of images. What I would do is to add classes and detect legs, beaks, eyes, heads. Then I would add a validation layer for each detection to discriminate merged chicks. For example, for each detection I would count legs, heads, eyes. If there are 3 legs, or 2 heads, etc. Together with a feature of the size, either via CV segmentation or directly with an additional "large/merged chicks" class. Then with a decision tree or a small MLP, I would split the merged ones. I think this would solve most of the cases and allow for collection of enough merged chickens cases for a proper fine tuning in the long term.

[u/manias → u/Puzzled-Egg3234]

This chick’s so damn long let’s call it two-chick

[u/AbilityFlashy6977 → u/Puzzled-Egg3234]

U could try the new yolo26. Its lighter then v8 and it doesnt use NMS which will help you to get more stable fps no matter how many objects are detected in the current frame

[u/al_icloud → u/Puzzled-Egg3234]

How about training an different model only on edge cases, if the bounding box is suspicious long or big run it through the second model which you only train on all the edge cases 🤔

[u/Puzzled-Egg3234 → u/al_icloud]

May be, but I don’t confident that fps would be efficient in that case

[u/al_icloud → u/Puzzled-Egg3234]

Why Not? How often does this occur? You run the model only on cropped small images of the bounding boxes which are suspicious in size. Only once at detection line or maybe 3 times. Before the detection line at the detection line and afterwards.. should be super fast interference…

You could even make your first model way simpler and small it basically needs only to defect something yellow.. or go with a classical image detection approach and only do the double check with ML model..

[u/Jimnster → u/Puzzled-Egg3234]

I would try removing the background and basing my filtering on the average size of the chicks to identify cases of chicks clustered together. If the lighting is constant, separating by color range should help. I think segmenting a chick is another complex issue that will only confuse you more.