CSE445Computer Vision
Tutorials/Object Detection/Sliding Window, Selective Search & R-CNN
Previous tutorialBounding Boxes, IoU, NMS & mAPNext tutorialFaster R-CNN: Region Proposal Network
OD-03Intermediate 90 min

Sliding Window, Selective Search & R-CNN

Trace the path from brute-force sliding window detectors through selective search region proposals to the original R-CNN, and understand the bottlenecks that each approach introduced.

OpenCVscikit-learnNumPyMatplotlib

Code Structure Note

These notes are provided only to help students understand the code structure. Students should review, optimize, and adapt the code before using it in assignments, projects, or experiments.

Topics Covered
  1. 1Multi-scale sliding window: image pyramid, stride, aspect ratios
  2. 2Selective search: colour, texture, size, and fill grouping hierarchies
  3. 3R-CNN pipeline: selective search → warp → CNN → SVM + box regressor
  4. 4Feature caching and the 47-second-per-image wall on VGG16
  5. 5Fast R-CNN: RoI pooling, single-pass shared convolutions, multi-task loss

Learning Objectives

  • Implement a multi-scale sliding window and measure its computational cost on a 1080p image
  • Use OpenCV's selective search to generate ~2 000 region proposals for an image
  • Explain the R-CNN training pipeline: CNN pretraining, SVM fine-tuning, box regression
  • Describe RoI pooling and explain how it enables a single shared forward pass in Fast R-CNN

Key Takeaways

  • Sliding window is O(W × H × scales × aspect_ratios) — infeasible without pruning
  • Selective search reduces candidates from millions to ~2 000 with high recall (>96% on VOC)
  • R-CNN's bottleneck: 2 000 forward passes per image, each on a warped 227×227 crop
  • Fast R-CNN eliminated per-region CNN calls but still depends on CPU selective search at test time

Tutorial Infographic

Notebook Resources

Jupyter Notebook

Student action area

Use this section to practice the tutorial code. Kaggle opens the prepared notebook, Colab lets you experiment in your own runtime, and the .ipynb download is best for local editing or submission.

Open in Kaggle Open in Colab Download .ipynb
Kaggle: follow the official tutorial notebook.
Colab: modify and rerun quickly.
IPYNB: keep a local copy.

Hands-On Exercises

  1. 1Run a sliding window detector with HOG+SVM on INRIA pedestrians and measure FPS
  2. 2Apply OpenCV selective search to 5 COCO images and visualise the top-200 proposals
  3. 3Load torchvision Fast R-CNN and compare inference time vs. repeated crop-then-classify
  4. 4Count the unique bottleneck in each pipeline stage (proposal, feature, classify, regress)

Tools & Libraries

OpenCVscikit-learnNumPyMatplotlib
Previous tutorialBounding Boxes, IoU, NMS & mAPNext tutorialFaster R-CNN: Region Proposal Network