CSE445Computer Vision
Tutorials/Self-Supervised Learning for Computer Vision/Negative-Free SSL with BYOL and YOLO26
Previous tutorialSimCLR Pretraining for the YOLO26 BackboneNext tutorialI-JEPA Predictive Pretraining for YOLO26
SSL-OD-02Intermediate–Advanced 2.5 Hours

Negative-Free SSL with BYOL and YOLO26

Teach negative-free representation learning using online and target copies of the YOLO26 backbone. Students learn how momentum updates, predictor asymmetry, stop-gradient, and exponential moving averages prevent representation collapse.

PyTorchUltralyticsTensorBoardscikit-learn

Code Structure Note

These notes are provided only to help students understand the code structure. Students should review, optimize, and adapt the code before using it in assignments, projects, or experiments.

Topics CoveredStudy each concept as part of the complete pipeline: understand its role, implement it, and verify its effect during transfer to YOLO26.
  1. 1Online encoderUpdate the main BYOL branch by gradient descent using the prediction error against the target branch.
  2. 2Target encoderProvide slowly changing representation targets that stabilize negative-free self-supervised training.
  3. 3Exponential moving averageUpdate target-network parameters as a momentum-weighted average of the online network instead of backpropagating through them.
  4. 4Projection headMap backbone features into the SSL objective space, then remove this temporary head before downstream transfer.
  5. 5Prediction headAdd BYOL's asymmetric predictor to the online branch so matching the target representation does not collapse to a trivial solution.
  6. 6Stop-gradientBlock gradient flow through the target branch so only the online encoder and predictor receive direct optimization updates.
  7. 7Representation collapseMonitor embedding variance and similarity statistics to detect whether all images are being mapped to nearly identical features.
  8. 8Negative-free learningLearn useful invariances without explicitly constructing or mining negative examples.
  9. 9Backbone checkpoint transferLoad only compatible encoder weights into YOLO26 while initializing the task-specific detection layers separately.
  10. 10Detection fine-tuningAdapt the transferred representation with labeled bounding boxes while tracking whether SSL accelerates convergence or improves accuracy.

Learning Objectives

  • Implement online and EMA target copies of the YOLO26 backbone
  • Explain the roles of predictor asymmetry and stop-gradient
  • Monitor representation statistics for signs of collapse
  • Transfer the BYOL-pretrained backbone into YOLO26 detection training

Key Takeaways

  • BYOL learns without explicit negative pairs
  • The target network changes slowly through an exponential moving average
  • Collapse checks are part of the training protocol, not an optional visualization

Notebook Resources

Jupyter Notebook

Student action area

Use this section to practice the tutorial code. Kaggle opens the prepared notebook, Colab lets you experiment in your own runtime, and the .ipynb download is best for local editing or submission.

Open in Kaggle
Kaggle: follow the official tutorial notebook.
Colab: modify and rerun quickly.
IPYNB: keep a local copy.

Hands-On Exercises

  1. 1Build the online and target YOLO26 backbone branches
  2. 2Implement the EMA update and stop-gradient alignment loss
  3. 3Plot embedding variance and pairwise cosine similarity during pretraining
  4. 4Fine-tune the transferred backbone for object detection

Tools & Libraries

PyTorchUltralyticsTensorBoardscikit-learn
Previous tutorialSimCLR Pretraining for the YOLO26 BackboneNext tutorialI-JEPA Predictive Pretraining for YOLO26