Tutorials/Object Tracking and Multi-Object Tracking/Detection-to-Track Association: From SORT to ByteTrack
Previous tutorialFoundations of Object Tracking and Motion PredictionNext tutorialAppearance, Occlusion, and Robust Identity Tracking
TR-02Intermediate 2.5 Hours
Detection-to-Track Association: From SORT to ByteTrack
Teach how detections from the current frame are assigned to existing tracks. Students construct similarity and cost matrices, apply gating, solve assignment problems, manage track lifecycles, and study the progression from SORT to ByteTrack.
PythonNumPySciPyFilterPyOpenCVYOLO26 detector
Code Structure Note
These notes are provided only to help students understand the code structure. Students should review, optimize, and adapt the code before using it in assignments, projects, or experiments.
Topics Covered
- 1Tracking by detection
- 2Existing tracks and current detections
- 3Predicted bounding boxes
- 4Intersection over Union
- 5Center and Mahalanobis distance
- 6Similarity and cost matrices
- 7Gating
- 8Matched and unmatched states
- 9Track confirmation, deletion, and buffer
Learning Objectives
- Build IoU, distance, and motion-gated association costs
- Compare greedy matching with Hungarian assignment
- Compose detector, Kalman filter, assignment, and lifecycle logic into SORT
- Explain how ByteTrack uses low-confidence observations for recovery
Key Takeaways
- Assignment connects frame-level observations to persistent track states
- Gating removes implausible pairs before optimization
- SORT combines motion prediction with IoU and Hungarian assignment
- ByteTrack separates high-confidence matching from low-confidence recovery
Tutorial Infographic
Hands-On Exercises
- 1Change the association threshold and track buffer
- 2Compare greedy and Hungarian assignment on the same matrix
- 3Inspect ID switches during partial occlusion
- 4Explain the role of persist=True in a manual video frame loop
Tools & Libraries
PythonNumPySciPyFilterPyOpenCVYOLO26 detector