Noisy Student
{, }
Embedded Files
1) Motivation, Objectives and Related Works:
Motivation:
Objectives:
Proposed a semi-supervised method inspired by Knowledge Distillation called “Noisy Student” in 2019.
Related Works:
Contribution:
The key idea is to train two separate models called “Teacher” and “Student”.
The teacher model is first trained on the labeled images and then it is used to infer the pseudo-labels for the unlabeled images. These pseudo-labels can either be soft-label or converted to hard-label by taking the most confident class.
Then, the labeled and unlabeled images are combined together and a student model is trained on this combined data. The images are augmented using RandAugment as a form of input noise. Also, model noise such as Dropout and Stochastic Depth are incorporated in the student model architecture.
Once a student model is trained, it becomes the new teacher and this process is repeated for three iterations.
2) Methodology:
Method 1:
Method 2:
3) Experimental Results:
Experimental Results:
Ablations:
References:
n2 n0
θ
Page updated
Google Sites
Report abuse