OD-10Advanced 120 min
Pseudo Annotation Using SAM 3
Use SAM 3 as an annotation assistant for object-detection datasets: generate high-quality masks from prompts, convert masks into bounding boxes, filter weak pseudo labels, and export reviewed annotations for YOLO or COCO training.
SAM 3PyTorchOpenCVpycocotoolsUltralytics YOLOv8Roboflow
Code Structure Note
These notes are provided only to help students understand the code structure. Students should review, optimize, and adapt the code before using it in assignments, projects, or experiments.
Topics Covered
- 1Pseudo annotation workflow: unlabeled images, prompt generation, SAM 3 masks, quality filtering, and human review
- 2Prompt strategies for SAM 3: points, boxes, detector proposals, and grid-based automatic mask generation
- 3Converting segmentation masks into object-detection boxes and COCO-style annotations
- 4Quality gates: confidence, mask area, box aspect ratio, duplicate removal, and IoU agreement
- 5Training with pseudo labels: accepted labels, reviewed labels, and final detector evaluation
Learning Objectives
- Explain when pseudo annotations reduce labeling cost and when they introduce harmful label noise
- Run a SAM 3 assisted annotation pipeline on unlabeled object-detection images
- Convert generated masks into bounding boxes and export them in YOLO and COCO formats
- Design filtering rules that reject low-quality pseudo boxes before detector training
- Compare detector performance using manual labels, pseudo labels, and human-reviewed pseudo labels
Key Takeaways
- SAM 3 can accelerate annotation, but pseudo labels still need validation before they become training data
- A mask can be converted to a detection box by taking the tight bounding rectangle around foreground pixels
- Filtering pseudo annotations is as important as generating them because noisy boxes can reduce mAP
- The strongest workflow combines automatic SAM 3 proposals with quick human review and detector-based evaluation
Hands-On Exercises
- 1Run SAM 3 on 50 unlabeled images using box prompts from a pretrained detector and save predicted masks
- 2Convert each accepted mask into a bounding box and export both YOLO txt labels and COCO JSON annotations
- 3Implement duplicate removal using IoU between generated boxes and keep the highest-confidence candidate
- 4Train YOLOv8 on pseudo labels only, then compare mAP@50 against a small manually labeled baseline
- 5Create a review table showing accepted, rejected, and manually corrected pseudo annotations with reasons
Tools & Libraries
SAM 3PyTorchOpenCVpycocotoolsUltralytics YOLOv8Roboflow