CSE445Computer Vision
ModulesDetection & TrackingObject Tracking
Previous lectureOptical FlowNext lectureClassical Recognition

Object Tracking

The Kalman filter, assignment with gating, and the SORT family, scored by MOT metrics.

L19CO3M05 · Detection & TrackingFinal Exam

Key Terms

Kalman FilterData AssociationGatingHungarian AlgorithmSORT / DeepSORT / ByteTrackMOTA / IDF1 / HOTA

Python Tools

OpenCVPyTorchUltralyticsNumPy

Textbook Reference

Szeliski Ch.9

Topics Covered

  1. 1Tracking-by-detection loop and track lifecycle management
  2. 2Kalman filter: constant-velocity model, predict and update equations
  3. 3Assignment as bipartite matching with IoU or Mahalanobis gating
  4. 4SORT, DeepSORT appearance embeddings, and ByteTrack low-score recovery
  5. 5MOT evaluation: MOTA, MOTP, IDF1, and HOTA

Learning Outcomes

  • Write the predict and update steps of a constant-velocity Kalman filter
  • Formulate track-to-detection assignment and apply gating
  • Compare SORT-family trackers and interpret MOT metrics

Lecture Infographics

2 image files

Tracking walkthrough

From detections to identities: SORT, DeepSORT, ByteTrack and the MOT metrics

Five stages, 21 steps over one 12-frame sequence: why detection alone cannot give identity, what the Kalman filter does when the detector sees nothing, how the assignment is solved exactly, what SORT, DeepSORT and ByteTrack each add, and how MOTA, MOTP, IDF1 and HOTA are computed. Every detection, gain, cost, switch and score on screen is simulated live, not hard-coded.

1 / 21

SEQUENCE · DETECTIONSoccludercar 0.91van 0.89person 0.87person 0.85frame 0 / 114 detectedTHE LOOP, ONE FRAME1DetectRun the detector on frame k. Boxes only — no identity yet.42PredictAdvance every track's Kalman filter one frame to get its prio…03AssociateMatch predictions to detections by overlap, optimally.04UpdateCorrect each matched track with its detection.05ManageBirth the unmatched detections, age the unmatched tracks, del…458 boxes over 12 frames, 4 of them spuriousSEQUENCEframes12detections58objects5steps per frame5predicted IoU0.68confirm after3 hitsThe loop · Boxes without names5 objects · 2 frames with no detection · 4 spurious boxes
carA
vanB
personC
personD
cyclistE

Stage 1 · Step 1 of 4

Boxes without names

Tracking by detection: the detector gives boxes, the tracker gives identity

Run the detector on all 12 frames and you get 58 boxes — and not one of them says which object it belongs to. Linking each frame to the next without any motion model admits 1.5e22 consistent histories, so identity has to be inferred, not read off.

∏k=111P(nk,nk−1)≈1.5×1022\prod_{k=1}^{11} P(n_k, n_{k-1}) \approx 1.5 \times 10^{22}

Key idea

A detector answers "what is in this frame". A tracker answers "which of these is the same thing as before". The whole difference is the association step, and everything else in this lecture exists to make that step decidable.

Playing. Selecting any stage or step pauses playback so you can inspect it.

Lecture PDFs

Object Tracking

The embedded PDF preview is disabled to reduce Vercel bandwidth usage. Open the lecture file directly in Google Drive for viewing, downloading, or presenting.

Open PDF
Previous lectureOptical FlowNext lectureClassical Recognition