Tutorials/Self-Supervised Learning

CSE445 · Computer vision tutorial series

Self-Supervised Learning for Computer Vision

Learn how unlabeled images can build transferable visual representations through contrastive learning, negative-free learning, latent-space prediction, and dense teacher guidance, then adapt those representations to a range of computer-vision tasks.

5 TutorialsIntermediate to AdvancedApproximately 12–14 hrsPyTorchComputer VisionSelf-Supervised LearningTransferable Representations

Unlabeled Images → SSL Representation Learning → Task Adaptation → Label-Efficient Computer Vision

Unlabeled image dataset

SSL representation learning

Pretrained visual encoder

Task-specific adaptation

Computer-vision predictions

Transfer evaluation

Organized topic guide

Topics Covered

Follow the concepts from unlabeled representation learning through backbone transfer and controlled label-efficiency evaluation. Each group identifies the ideas and implementation decisions used later in the tutorials.

Topic 01

SSL Foundations

Build supervision from the structure of unlabeled images before introducing task annotations.

Pretext signals and representation learning
Unlabeled pretraining and labeled fine-tuning phases
Encoder, projection head, and downstream task head roles
Controlled comparison with random and supervised initialization

Topic 02

SimCLR Contrastive Learning

Learn invariance by pulling together two augmented views of one image while separating other samples.

Two-view augmentation and positive-pair construction
Projection heads, cosine similarity, and normalized embeddings
NT-Xent loss, temperature, and in-batch negatives
Checkpoint transfer from pretraining into YOLO26

Topic 03

BYOL Negative-Free Learning

Train without explicit negative pairs through asymmetric online and slowly updated target networks.

Online encoder, predictor, and EMA target encoder
Stop-gradient and exponential moving-average updates
Representation-collapse risks and diagnostic statistics
Negative-free backbone transfer and fine-tuning

Topic 04

I-JEPA Latent Prediction

Predict masked target-region representations from visible context without reconstructing image pixels.

Context and target block sampling
Context encoder, EMA target encoder, and predictor roles
Latent-space prediction with stop-gradient targets
I-JEPA-pretrained YOLO26 backbone transfer

Topic 05

DINOv3 Feature Distillation

Transfer dense semantic knowledge from a frozen vision-foundation teacher into a deployable detector.

Patch tokens and dense teacher representations
Spatial resizing, channel projection, and feature adapters
Multi-scale YOLO26 feature alignment
Joint distillation and object-detection objectives

Topic 06

Backbone Transfer to YOLO26

Move the reusable encoder into an object-detection workflow while keeping the experiment comparable.

Projection-head removal and backbone checkpoint loading
Detection-head attachment and staged unfreezing
Fixed data splits, augmentation, and optimization settings
Random, supervised, and SSL initialization baselines

Topic 07

Label-Efficient Evaluation

Measure whether SSL improves transfer quality when bounding-box annotations are limited.

10%, 25%, 50%, and 100% annotation budgets
Multi-seed runs and fixed validation/test sets
mAP, precision, recall, convergence, and latency
Per-class results, robustness checks, and failure analysis

Companion library

SSL Detection Lab Library

Every notebook in this series is powered by ssl-detection-lab, an open-source PyTorch package built for CSE445. Training loops live in the library; the notebooks stay focused on configuration, method calls, plots, and interpretation. The reusable ssldet package covers SSL objectives, YOLO/DINO feature backbones, detection adapters, labelled evaluation, video analysis, downstream checkpoint transfer, and workflow helpers.

Open source · MIT

rifat963/ssl-detection-lab

Seven SSL objectives behind one interface: SimCLR, BYOL, MoCo, DINOv2-style, DINOv3-guided distillation, MAE, and I-JEPA
Pretrains Ultralytics YOLO backbones on unlabeled images, with single-GPU and distributed T4 x2 launch helpers
Verified SSL-checkpoint transfer into YOLO26 detectors with backbone-key coverage checks
Labelled object-detection evaluation with full metric export (mAP, per-class AP, confusion matrix)
Video analysis and tracking with frame-level detection, confidence, and latency reports
Custom SSL objectives and detection backends can be registered without modifying the package

Reusable package structure

ssldet.ssl: Reusable objectives, losses, interfaces, and custom-module registry
ssldet.backbones: YOLO, DINOv2, and DINOv3 feature encoders
ssldet.detection: Backend-neutral detector protocol and Ultralytics adapter
ssldet.evaluation: Labelled object-detection evaluation and metric export
ssldet.video: Streaming detection/tracking and video metrics
ssldet.downstream: Verified SSL-checkpoint transfer into YOLO detectors
ssldet.workflow: Student-friendly dry-run and distributed-launch helpers

Recommended learning path

#	Tutorial	Kaggle notebook	Local copy
1	SimCLR pretraining (contrastive SSL basics, t-SNE, nearest neighbors)	Open on Kaggle	simclr_football_t4x2_tutorial.ipynb
2	SimCLR → YOLO26 downstream (transfer, fine-tune, evaluate, track)	Open on Kaggle	simclr_yolo26_football_downstream_tutorial.ipynb
3	BYOL pretraining (online/EMA target branches, AMP, distributed T4 x2)	Open on Kaggle	byol_football_t4x2_tutorial.ipynb
4	I-JEPA pretraining (masked latent-block prediction on YOLO26)	Open on Kaggle	ijepa_yolo26_football_ssl_tutorial.ipynb
5	I-JEPA → YOLO26 downstream	Open on Kaggle	ijepa_yolo26_downstream_tutorial.ipynb
6	DINOv3-guided distillation (frozen-teacher feature distillation into YOLO26)	Open on Kaggle	dinov3_yolo26_football_ssl_tutorial.ipynb
7	DINOv3 → YOLO26 downstream	Open on Kaggle	dinov3_yolo26_downstream_tutorial.ipynb
8	Data association and multi-object tracking with Ultralytics trackers	Open on Kaggle	— (Kaggle only)

Tutorials 1–2, 4–5, and 6–7 form pretrain → downstream pairs. Tutorial 3 (BYOL) is a standalone pretraining walkthrough, and tutorial 8 covers tracking after you have a fine-tuned detector.

View on GitHub pip install "git+https://github.com/rifat963/ssl-detection-lab.git@main"

Recommended sequence

Recommended Learning Path

Each tutorial adds one level of representational or experimental sophistication.

Contrastive SSL

Contrastive learning with SimCLR

Negative-Free SSL

Negative-free learning with BYOL

Predictive SSL

Latent prediction with I-JEPA

Dense Teacher Distillation

Foundation-model distillation with DINOv3

Comparative Research Evaluation

Controlled label-efficiency evaluation

Method selection

Architecture Comparison

Compare how the supervision signal, encoder update, and feature granularity differ across four reusable SSL strategies.

Property	SimCLR	BYOL	I-JEPA	DINOv3 Distillation
Uses negative pairs	Yes	No	No	No
Directly pretrains YOLO26 backbone	Yes	Yes	Yes	Yes, through teacher guidance
Uses momentum encoder	No	Yes	Yes	No
Uses frozen external teacher	No	No	No	Yes
Primary representation	Global and optional multi-scale	Global and optional multi-scale	Masked block-level	Dense patch-level
Complexity	Moderate	Moderate	High	High
Best role	SSL foundation	Negative-free backbone learning	Spatial predictive learning	Strong detection transfer

Reusable protocol

Common Experimental Workflow

Use the same dataset organization and checkpoint handoff across all five tutorials.

1Collect unlabeled domain images.
2Pretrain or guide a visual encoder.
3Save the SSL encoder checkpoint.
4Attach a task-specific prediction head.
5Fine-tune using the available task annotations.
6Evaluate on a fixed test set.
7Compare accuracy, label efficiency, and latency.

Applied YOLO26 project structuretext

datasets/
├── unlabeled/
├── labeled/
│   ├── images/
│   └── labels/
checkpoints/
├── simclr_yolo26_backbone.pt
├── byol_yolo26_backbone.pt
├── ijepa_yolo26_backbone.pt
└── dinov3_distilled_yolo26.pt
notebooks/
├── SSL-OD-01-SimCLR-YOLO26.ipynb
├── SSL-OD-02-BYOL-YOLO26.ipynb
├── SSL-OD-03-IJEPA-YOLO26.ipynb
├── SSL-OD-04-DINOv3-YOLO26.ipynb
└── SSL-OD-05-Label-Efficient-YOLO26.ipynb

Before you begin

Prerequisites

Convolutional neural networks
Object-detection fundamentals
YOLO training workflow
PyTorch tensors and training loops
Image augmentation
Transfer learning
Precision, recall, IoU, and mAP

Required deliverables

Series Assessment

One SimCLR-pretrained YOLO26 backbone
One BYOL-pretrained YOLO26 backbone
One I-JEPA-pretrained YOLO26 backbone
One DINOv3-guided YOLO26 detector
One label-efficiency comparison
One feature visualization
One detection failure analysis
One research-style result table

Five-part notebook sequence

Notebooks and Follow-Up Activities

SimCLR pretraining → BYOL training → I-JEPA predictive pretraining → DINOv3 distillation → controlled label-efficiency comparison.

01SimCLR + YOLO26Intermediate 2 Hours

SimCLR Pretraining for the YOLO26 Backbone

Introduce the foundations of self-supervised visual representation learning by pretraining the YOLO26 backbone with SimCLR. Students construct positive image pairs, apply strong augmentations, optimize the NT-Xent contrastive objective, and transfer the learned backbone to an object-detection task.

Key concepts

Unlabeled image pretrainingPositive and negative pairsTwo-view augmentationProjection headCosine similarityTemperature parameterNT-Xent lossLinear feature evaluationYOLO26 backbone transferDetection fine-tuning

Compact algorithm flow

ImageTwo augmented viewsShared YOLO26 backboneProjection headContrastive lossDetection fine-tuning

Practical outcome

Students produce a SimCLR-pretrained YOLO26 backbone and compare it with a randomly initialized YOLO26 detector.

Open Tutorial

02BYOL + YOLO26Intermediate–Advanced 2.5 Hours

Negative-Free SSL with BYOL and YOLO26

Teach negative-free representation learning using online and target copies of the YOLO26 backbone. Students learn how momentum updates, predictor asymmetry, stop-gradient, and exponential moving averages prevent representation collapse.

Key concepts

Online encoderTarget encoderExponential moving averageProjection headPrediction headStop-gradientRepresentation collapseNegative-free learningBackbone checkpoint transferDetection fine-tuning

Compact algorithm flow

Two viewsOnline YOLO26 + predictorEMA target YOLO26Stop-gradient alignmentPretrained backboneDetection fine-tuning

Practical outcome

Students pretrain the YOLO26 backbone using BYOL, monitor feature collapse, and fine-tune the resulting backbone for object detection.

Open Tutorial

03I-JEPA + YOLO26Advanced 2.5 Hours

I-JEPA Predictive Pretraining for YOLO26

Pretrain the YOLO26 backbone with an I-JEPA-style latent prediction objective. Students sample context and target blocks from the same image, use a context encoder to predict masked target representations, update a target encoder with an exponential moving average, and transfer the learned backbone to football-player detection.

Key concepts

Context and target blocksMulti-block mask samplingContext encoderEMA target encoderLatent-space predictionPredictor networkStop-gradient targetsRepresentation collapse monitoringYOLO26 backbone checkpoint transferDetection fine-tuning

Compact algorithm flow

ImageContext and target blocksYOLO26 context encoderEMA target encoderLatent prediction lossDetection fine-tuning

Practical outcome

Students produce an I-JEPA-pretrained YOLO26 backbone, visualize its block-prediction setup, and evaluate its transfer to football-player detection.

Open Tutorial

04DINOv3 → YOLO26Advanced 3 Hours

DINOv3-Guided Feature Distillation into YOLO26

Use a frozen DINOv3 vision foundation model as a semantic teacher for the YOLO26 backbone. Students extract dense patch-level teacher representations, align them with multi-scale YOLO26 feature maps, and jointly optimize feature-distillation and object-detection losses.

Key concepts

Frozen teacher modelDINOv3 patch tokensDense visual representationsTeacher–student learningMulti-scale YOLO26 featuresFeature adaptersSpatial resizing and projectionCosine feature alignmentDistillation lossJoint detection training

Compact algorithm flow

Input imageFrozen DINOv3 teacherDense patch featuresFeature adaptersYOLO26 multi-scale featuresDetection training

Practical outcome

Students create a DINOv3-guided YOLO26 detector and evaluate whether dense semantic supervision improves object localization and classification.

Open Tutorial

05Comparative EvaluationAdvanced / Research 3 Hours

Label-Efficient SSL-YOLO26 Comparative Study

Perform a controlled comparison between supervised and self-supervised YOLO26 detectors. Students evaluate how SimCLR, BYOL, I-JEPA, and DINOv3-guided pretraining affect object-detection performance when only a limited percentage of bounding-box annotations is available.

Key concepts

Label efficiencyControlled experimentsFixed train, validation, and test splitsMulti-seed evaluationmAP@0.5 and mAP@0.5:0.95Precision and recallPer-class average precisionConvergence speedInference latencyRobustness analysisAnnotation-cost reduction10%, 25%, 50%, and 100% label budgets

Compact algorithm flow

Six YOLO26 initializations10%, 25%, 50%, 100% labelsControlled trainingmAP and efficiency metricsFailure analysisEvidence-based recommendation

Practical outcome

Students produce a research-style comparison table, label-efficiency curves, per-class results, failure-case analysis, and a recommendation for the best YOLO26 initialization strategy.

Open Tutorial

Previous seriesObject Detection Next seriesObject Tracking and Multi-Object Tracking