A pretext task is a task that is used for model pretraining. By solving and studying objective functions of pretext tasks, the networks can learn visual features/representations or model weights which are useful for downstream tasks. Some examples of pretext tasks such as colorization, placing image patches in the right place, predicting rotation of images, placing frames in the right order, inpainting, recovering the input under some corruptions.
s
Pretext tasks that correspond to each model type:
By solving the color inpainting task, a model can grasp the “common sense” of what the color of dog/ grass should be.
Ashish Jaiswal et al., A survey on contrastive self-supervised learning, Technologies 9.1, 2021.
Color Transformation
(a) Original (b) Gaussian noise (c) Gaussian blur (d) Color distortion (Jitter)
Predicting Rotation
By solving this task, a model can grasp the “common sense” of what is the top and what is the bottom of a tree.
Aniket Bhatnagar, Dhruv Goyal, Cole Smith and Nikhil Supekar, Self-supervised learning - Pretext tasks, atcold repo, 2020.
Predicting Rotation
Solving Jigsaw Puzzle
Solving jigsaw puzzle (context-based) as a pretext task (Image from [4]). (a) Original image (b) Reshuffled image. The anchor is the original image while the positive sample is the reshuffled image.
By solving this task, a model can grasp the “common sense” of which location each part of a tiger should locate. Through this, a model can know the overall shape of a tiger and the correlative positions of different parts, not just its detailed information of each separated part.
Ashish Jaiswal et al., A survey on contrastive self-supervised learning, Technologies 9.1, 2021.
Solving Jigsaw Puzzle
Relative Position Prediction
Like solving jigsaw puzzles, a model can grasp the knowledge about the overall shape and the relative positions of different parts of a cat.
Jeremy Howard, Self-supervised learning and computer vision, fast.ai, 2020.
Relative Position Prediction
Predicting the Corrupted Region
By solving this task, a model can grasp the “common sense” of how to get the semantic meaning of objects derived from the information of context around.
Longlong Jing and Yingli Tian, Self-supervised visual feature learning with deep neural networks: A survey, IEEE TPAMI, 2020.
Predicting the Corrupted Region
