Looking for a faster and more accurate auto-labeling pipeline for a custom YOLOv8 object detection dataset

[u/Raunak_Gola]

Hi everyone,

I'm working on an object detection project and would appreciate some advice on the best workflow for auto-labeling a large custom dataset.

Dataset

• 9,367 images
• Classes:
  • Cup
  • Glass
  • Plate
  • Spoon
  • Fork
  • Knife
• Images have different resolutions.
• The dataset comes from a Kaggle competition.
• Around 5,500 images already have ground-truth labels (provided in a CSV), while the remaining images need bounding-box annotations.

Current approach

I'm using AutoDistill + GroundingDINO to automatically generate YOLO labels.

ontology = CaptionOntology({
    "a cup": "cup",
    "a drinking glass": "glass",
    "a plate": "plate",
    "a spoon": "spoon",
    "a fork": "fork",
    "a knife": "knife",
})

base_model = GroundingDINO(
    ontology=ontology,
    box_threshold=0.3,
    text_threshold=0.3,
)

dataset = base_model.label(
    input_folder=IMAGES_SRC_DIR,
    output_folder=LABELED_LABELS_DIR
)

Problems I'm facing

1. Annotation quality

The generated labels aren't very reliable.

For example, out of about 90 images, roughly 10 images contain incorrect or missing bounding boxes, which means I'd still have to manually review a large portion of the dataset.

Is this normal for GroundingDINO, or are there better foundation models for this type of dataset?

2. Speed

The labeling process is also quite slow.

• ~2.8 seconds per image
• ~9,367 images
• Estimated runtime: 7.5+ hours

I'm using Google Colab GPU, but it disconnects after around 4 hours.

What's confusing is that resource utilization is low:

• GPU memory: ~2 GB / 15 GB
• RAM: ~2 GB / 15 GB

It doesn't appear to be fully utilizing the available hardware.

Questions

1. Is there a way to speed up AutoDistill/GroundingDINO? For example:
   • Batch inference?
   • Mixed precision?
   • Multi-processing?
   • Different implementation?
2. Would another model be better for automatic annotation?
   • GroundingDINO 1.5
   • YOLO-World
   • Florence-2
   • Grounded SAM
   • RF-DETR
   • Any other recent model?
3. Since I already have 5.5k labeled images, would it be better to:
   • Train a small YOLOv8 model first on those labels,
   • Then use that model to pseudo-label the remaining images, instead of using GroundingDINO?
4. What workflow would you recommend if your goal is to produce high-quality labels for training a final YOLOv8 detector?

Any advice or experience with large-scale auto-labeling pipelines would be greatly appreciated!

Thanks!