Contrastive Representation Distillation via Multi-Scale Feature Decoupling

Knowledge distillation is a technique aimed at enhancing the performance of a smaller student network without increasing its parameter size by transferring knowledge from a larger, pre-trained…