Interactive Video Object Segmentation

Motivation:

• Interactive Video Object Segmentation (iVOS) iVOS aims at extracting high-quality segmentation masks of a target video object through two modules: a 2D interactive segmentation module and a segmentation propagation module

Objectives:

• MiVOS [28] decouples the two modules and train them independently. During inference, MiVOS first interactively segments one or several frames in a video, followed by propagating the segmented frames to the unsegmented ones. 

• STCN [14] further improves the segmentation propagation module in MiVOS by directly encoding the query and memory frames without re-encoding the mask features for every object.

Related Works:

Contribution:

Drift: [Ref]

• A well-known challenge in tracking is “drift”, where small errors accumulate over time. Our approach counters this pitfall by requesting more labeled frames where flow errors appear to accumulate, and by using an appearance model for uniform regions with few key points.

Label propagation in video: [Ref]

• Using probabilistic models [14, 6, 2, 18]. The methods typically assume the label field in the first (and/or last) frame of the sequence is given, and then automatically track through the remaining frames based on the objects’ color and motion properties.

Keyframe selection: [Ref]

• Feyframe selection finds representative frames using clustering (e.g., [25]) or by maximizing the dissimilarity between keyframes [7, 13]

Method 1:

Method 2:

References:

1. • n2  n0

   • θ