Pseudo-label

Motivation:

Objectives:

• Proposed a very simple and efficient formulation called “Pseudo-label” in 2013. 

Related Works:

Contribution:

• The idea is to train a model simultaneously on a batch of both labeled and unlabeled images. 

  1. The model is trained on labeled images in a usual supervised manner with a cross-entropy loss. The same model is used to get predictions for a batch of unlabeled images and the maximum confidence class is used as the pseudo-label. 

  2. Then, the cross-entropy loss is calculated by comparing model predictions and the pseudo-label for the unlabeled images.

Method 1:

Loss Function:

•  The total loss is a weighted sum of the labeled and unlabeled loss terms.

L = Llabeled + at * Lunlabeled

• To make sure the model has learned enough from the labeled data, the at term is set to 0 during the initial 100 training steps. It is then gradually increased up to 600 training steps and then kept constant.

Experimental Results:

Ablations:

References:

1. • n2  n0

   • θ