Video Contrastive Learning with Global Context

0) Motivation, Object and Related works:

Motivation:

• Contrastive learning has revolutionized self-supervised image representation learning field, and recently been adapted to video domain. 

• One of the greatest advantages of contrastive learning is that it allows us to flexibly define powerful loss objectives as long as we can find a reasonable way to formulate positive and negative samples to contrast. However, existing approaches rely heavily on the short-range spatio-temporal salience to form clip-level contrastive signals, thus limit themselves from using global context. 

Objectives:

1. In this paper, we propose a new video-level contrastive learning method based on segments to formulate positive pairs. 

   • Our formulation is able to capture global context in a video, thus robust to temporal content change. 

2. We also incorporate a temporal order regularization term to enforce the inherent sequential structure of videos. 

3. Extensive experiments show that our video-level contrastive learning framework (VCLR) is able to outperform previous state-of-the-arts on five video datasets for downstream action classification, action localization and video retrieval.