SSL, Transformers, and Foundation Models
InfoNCE contrastive learning, the attention mechanism, and CLIP and SAM as multimodal vision foundations.
Key Terms
Self-Supervised LearningInfoNCEAttentionVision TransformerCLIPSAM
Python Tools
PyTorchHugging Face TransformersOpenCV
Textbook Reference
Szeliski Ch.5-6
Topics Covered
- 1Pretext tasks and contrastive learning with the InfoNCE objective
- 2Scaled dot-product attention, multi-head attention, and positional encoding
- 3Vision Transformers and patch embeddings
- 4CLIP: image-text contrastive pretraining and zero-shot transfer
- 5SAM and promptable segmentation; multimodal and open-vocabulary vision
Learning Outcomes
- Explain the InfoNCE objective and what makes a good positive and negative pair
- Describe the attention mechanism and the Vision Transformer architecture
- Explain how CLIP and SAM are trained and used for downstream tasks
Lecture PDFs
SSL, Transformers, and Foundation Models
The embedded PDF preview is disabled to reduce Vercel bandwidth usage. Open the lecture file directly in Google Drive for viewing, downloading, or presenting.