CSE445Computer Vision
ModulesDetection & TrackingLearning-Based Vision
Previous lectureClassical RecognitionNext lectureDeep Detection

Learning-Based Vision Foundations

Convolution arithmetic, backpropagation, losses and optimization, regularization, and transfer learning.

L21CO3M05 · Detection & TrackingFinal Exam

Key Terms

Conv ArithmeticBackpropagationCross-EntropySGD / AdamRegularizationTransfer Learning

Python Tools

PyTorchNumPyMatplotlib

Textbook Reference

Szeliski Ch.5

Topics Covered

  1. 1Convolution arithmetic: kernel size, stride, padding, dilation, and receptive field
  2. 2Forward and backward passes and the chain rule on computation graphs
  3. 3Classification and regression losses and their gradients
  4. 4Optimization: SGD, momentum, Adam, learning-rate schedules, and batch normalization
  5. 5Regularization and transfer learning: weight decay, dropout, augmentation, and fine-tuning

Learning Outcomes

  • Compute output shapes and receptive fields for convolutional stacks
  • Explain backpropagation through convolution, pooling, and normalization layers
  • Select losses, optimizers, and regularizers and set up transfer learning

Architecture walkthrough

From hand-crafted features to a learned detector

Seven stages, 23 steps: what changes when features are learned, how one convolution is computed, how depth buys receptive field, transfer learning, the backbone–neck–head split, and what the head predicts. The convolutions, receptive fields, parameter counts, IoU, NMS and average precision on screen are all computed live, not hard-coded.

1 / 23

INPUT IMAGE20 × 20 intensity gridTWO PIPELINESHAND-CRAFTEDSIFT / HOGdesignedk-means codebooklearnedtf-idf poolingdesignedlinear SVMlearnedLEARNEDconv stackdesignedfeature pyramiddesigneddetection headdesignedloss + SGDdesignedWHAT IS LEARNEDhand-crafted2 of 4 blocks learnedconvolutional4 of 4 blocks learnedWHERE THE LEARNING HAPPENS2hand-designed blocks4learned blocks6.62 Mtotal parametersone losssupervisionTHE FILTERS ARE PARAMETERS, SO THE FEATURE EXTRACTOR ITSELF IS TRAINED RATHER THAN CHOSEN

Stage 1 · Step 1 of 3

What was fixed

From designed features to learned features

In the classical pipeline the detector, the descriptor and the pooling were all designed by hand. Only the codebook and the classifier saw the data.

y=gθ(ϕ( f(I) )),f,ϕ fixed by handy = g_{\theta}\big( \phi(\, f(\mathcal{I}) \,) \big), \qquad f, \phi \ \text{fixed by hand}

Key idea

Bag-of-Features learned two blocks out of four. A convolutional detector learns all of them, so the features are optimised for the task instead of chosen in advance.

Playing. Selecting any stage or step pauses playback so you can inspect it.

Lecture PDFs

Learning-Based Vision Foundations

The embedded PDF preview is disabled to reduce Vercel bandwidth usage. Open the lecture file directly in Google Drive for viewing, downloading, or presenting.

Open PDF
Previous lectureClassical RecognitionNext lectureDeep Detection