Interactive image segmentation aims to obtain an accurate binary mask of the target object with the least interaction (strokes) cost. It has developed into an indispensable tool in serving pixel-level data annotation and image editing.
ADE20k: 22k images with 434k instances (total)
OpenImages: 944k images with 2.6M instances (total)
MS COCO: 118k images with 1.2M instances (train)
HVACv1.0: 100k images with 1.2M instances (total)
COCO+LVIS* Custom: 99k images with 1.2M instances (train)
SBD: 8498 images with 20172 instances for (train); 2857 images with 6671 instances for (test)
Grab Cut: 50 images with one object each (test)
Berkeley: 96 images with 100 instances (test)
DAVIS-585: 345 images with one object each (test)
Pascal VOC: 1449 images with 3417 instances (validation)
COCO_MVal: 800 images with 800 instances (test)
DAVIS: A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation (DAVIS)
A comparative evaluation of interactive segmentation algorithms.
J. M. Gauch, and C. W. Hsia, "Comparison of three-color image segmentation algorithms in four color spaces", Visual Communications and Image Processing'92, 1992.
N. Ikonomakis, K. N. Plataniotis, and A. N. Venetsanopoulos, "Color image segmentation for multimedia applications", Journal of Intelligent and Robotic Systems, 2000.
S. D. Olabarriaga, and A. W. M. Smeulders, "Interaction in the segmentation of medical images: A survey", Medical image analysis, 2001.
Y. Boykov, and V. Kolmogorov, "An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision", IEEE Transactions on Pattern Analysis and Machine Intelligence (tPAMI), Vol. 26, I. 9, pp. 1124 - 1137, 2004.
P. Gupta, S. Saxena, S. Singh, S. Dhami, and V. Singh, "Color image segmentation: a state of the art survey", Proceedings-Indian National Science Academy Part A, 67(2), pp.207-222, 2012.
F. Zhao, and X. Xie, "An Overview on Interactive Medical Image Segmentation", Annals of the BMVA, Vol. 2013, No. 7, pp 1–22, 2013.
H. A. Jalab, and A. M. Hasan, "Magnetic resonance imaging segmentation techniques of brain tumors: A review", Archives of Neuroscience, 2019
H.amadan, C. Lachqar, and H. Tairi, "A survey of recent interactive image segmentation methods", Computational visual media, 2020.
K. H. Höhne, and W. A. Hanson, "Interactive 3D segmentation of MRI and CT volumes using morphological operations", Journal of Computer-assisted Tomography, 1992.
[MSRM] J. Ning, L. Zhang, D. Zhang, and C. Wu, “Interactive image segmentation by maximal similarity based region merging”, Pattern Recognition, vol. 43, no. 2, pp. 445–456, 2010.
[Lazy Snapping] Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum, “Lazy snapping,” ACM Trans. Graph., vol. 23, no. 3, pp. 303–308, 2004. (Object marking followed by boundary editing).
X. Bai and G. Sapiro, “Distance cut: interactive segmentation and matting of images and videos,” in IEEE Intl. Conf. on Image Processing (ICIP), 2007, vol. 2, pp. 249–252.
Y. Boykov and M.P. Jolly, “Interactive graph cuts for optimal boundary and region segmentation of objects in ND images,” in International Conference on Computer Vision. Citeseer, 2001, vol. 1, pp. 105–112.
A. Blake, C. Rother, M. Brown, P. Perez, and P. Torr, “Interactive image segmentation using an adaptive GMMRF model,” Lecture Notes in Computer Science, pp. 428–441, 2004.
E.N. Mortensen and W.A. Barrett, “Intelligent scissors for image composition,” in Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. ACM New York, NY, USA, 1995, pp. 191–198.
[Geodesic] X. Bai and G. Sapiro, “A geodesic framework for fast interactive image and video segmentation and matting,” in Proc. IEEE Int. Conf. Comput. Vis., pp. 1–8, 2007
[Intelligent Scissors] "Intelligent scissors for image composition", SIGGRAPH, 1995.
[GraphCut]
"Interactive graph cuts for optimal boundary and region segmentation of objects in nd images"
"Graph cuts in vision and graphics: theories and applications"
[GrabCut] C. Rother, V. Kolmogorov, and A. Blake, "GrabCut: interactive foreground extraction using iterated graph cuts", ACM transactions on graphics (TOG), 2004. [Code]
N. Xu, B. Price, S. Cohen, J. Yang, and T. Huang, “Deep Grabcut for Object Selection,” arXiv preprint arXiv:1707.00243, 2017. (bounding box-based)
[Random Walks]
[RW] L. Grady, “Random walks for image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 11, pp. 1768–1783, Nov. 2006.
"Generative image segmentation using random walks with restart"
[The Shortest Paths]
"Interactive segmentation with intelligent scissors"
"Adaptive boundary detection using ‘livewire’ two-dimensional dynamic programming"
iFCN [42] guides a CNN from positive and negative points acquired from the ground-truth masks.
RIS-Net [10] build on iFCN to improve the result by adding local context.
Our method DEXTR significantly improves the results by using 4 points as the supervisory signal: the extreme points.
Adams R, Bischof L (1994) Seeded region growing. IEEE Trans Pattern Anal Mach Intel 16(6):641–647
Mehnert A, Jackway P (1997) An improved seeded region growing algorithm. Pattern Recogn Lett 18(10):1065–1071
Ning J, Zhang L, Zhang D, Wu C (2010) Interactive image segmentation by maximal similarity based region merging. Pattern Recogn 43(2):445–456
Interactive image segmentation using adaptive constraint propagation. IEEE TIP, 2016.
Diffusive likelihood for interactive image segmentation. PR, 2018.
Probabilistic diffusion for interactive image segmentation
[ABR*] D Liu, Y Xiong, L Shapiro, and K Pulli, "Robust interactive image segmentation with automatic boundary refinement", Proceedings of 2010 IEEE 17th International Conference on Image Processing (ICIP), 2010. (Boundary)
[CxAC*] T. N. A. Nguyen, J. Cai, J. Zhang, and J. Zheng, "Robust interactive image segmentation using convex active contours", IEEE Transactions on Image Processing, 2012.
[CdAC*] NTN Anh, J Cai, J. Zhang, and J. Zheng, "Constrained active contours for boundary refinement in interactive image segmentation", IEEE International Symposium on Circuits and Systems (ISCAS), 2012.
[SAL*] C. Oh, B. Ham, and K. Sohn, "Robust interactive image segmentation using structure-aware labeling", Expert Systems with Applications, Vol. 79, pp. 90-100, 2017.
[MST] L. Xu, S. Li, Y. Chen, J. Luo, "MST: Adaptive Multi-Scale Tokens Guided Interactive Segmentation". in arXiv preprint arXiv:2401.04403, 2024.
[CFR-ICL] S. Sun, M. Xian, F. Xu, L. Capriotti, T. Yao, "CFR-ICL: Cascade-Forward Refinement with Iterative Click Loss for Interactive Image Segmentation", Proceedings of the AAAI Conference on Artificial Intelligence, pp. 5017-5024., 2024.
[AdaptiveClick] J. Lin, J. Chen, K. Yang, A. Roitberg, S. Li, Z. Li, and S. Li, “AdaptiveClick: Clicks-aware Transformer with Adaptive Focal Loss for Interactive Image Segmentation”, arXiv preprint arXiv:2305.04276, 2023.
[SimpleClick] Q Liu, Z Xu, G Bertasius, M Niethammer, "SimpleClick: Interactive Image Segmentation with Simple Vision Transformers", arXiv preprint arXiv:2210.11006, 2022.
Z Lin, Z Zhang, ZY Zhu, DP Fan, XL Liu, "Sequential Interactive Image Segmentation", CVMJ, 2022
J. Bragantini, A. X. Falcão, and L. Najman, "Rethinking Interactive Image Segmentation: Feature Space Annotation", Pattern Recognition, 2022 [Code]
[PseudoClick] Q. Liu, M. Zheng, B. Planche, S. Karanam, and T. Chen, "PseudoClick: Interactive Image Segmentation with Click Imitation", ECCV, 2022
[FocusCut] Z. Lin, Z. P. Duan, Z. Zhang, C. L. Guo, and M. M. Cheng, "FocusCut: Diving Into a Focus View in Interactive Segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2637-2646, 2022.
[FocalClick] X. Chen, Z. Zhao, Y. Zhang, M. Duan, D. Qi, and H. Zhao, "FocalClick: Towards Practical Interactive Image Segmentation", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1300-1309, 2022. [Code]
[RITM] K. Sofiiuk, I. A. Petrov, and A. Konushin, "Reviving Iterative Training with Mask Guidance for Interactive Segmentation", arXiv preprint arXiv:2102.06583, 2021. [Code] (random perturbations for the simulated clicks)
[EdgeFlow] Y. Hao, Y. Liu, Z. Wu, L. Han, Y. Chen, G. Chen, L. Chu, S. Tang, Z. Yu, Z. Chen, and B. Lai, "EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flow", Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1551-1560, 2021. [Code]
[UCP-Net] C. Dupont, Y. Ouakrim, Q.C. Pham, "UCP-Net: Unstructured Contour Points for Instance Segmentation", IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021.
J. H. Liew, S. Cohen, B. Price, L .Mai, and J. Feng, "Deep Interactive Thin Object Selection", Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 305-314, 2021. [Code] [Supplement] (click-based + extreme points)
[FBA-Matting] M. Forte, B. Price, S. Cohen, N. Xu, and F. Pitié, "Getting to 99% Accuracy in Interactive Segmentation", arXiv preprint arXiv:2003.07932, 2020. [Code]
S Majumder, A Rai, A Khurana, A Yao, "Two-in-One Refinement for Interactive Segmentation", BMVC, 2020 (combination)
[IOG] S Zhang, JH Liew, Y Wei, S Wei, Y Zhao, "Interactive Object Segmentation With Inside-Outside Guidance", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12234-12244, 2020. (combination)
[FCA-Net ] Z Lin, Z Zhang, LZ Chen, MM Cheng, SP Lu, "Interactive Image Segmentation With First Click Attention", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13339-13348, 2020. (take first click as special guidance).
[f-BRS] K Sofiiuk, I Petrov, O Barinova, A Konushin, "f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10-11. 8623-8, 2020. [Code]
R. Benenson, S. Popov, and V. Ferrari, “Large-scale interactive object segmentation with human annotators,” in CVPR, pp. 11700–11709, 2019. (combination/ encode positive and negative clicks to a 2-channel feature map with the same spatial size as the input image)
[Click Carving] Click carving: Interactive object segmentation in images and videos with point clicks. IJCV, 2019. (boundary clicks)
Multiseg: Semantically meaningful, scale-diverse segmentations from minimal user input. In ICCV, 2019. (scale diversity).
[ITIS] S. Mahadevan, P. Voigtlaender, B. Leibe, "Iteratively Trained Interactive Segmentation”, in British Machine Vision Conference (BMVC), 2018.
[DEXTR] K. K. Maninis, S. Caelles, J. Pont-Tuset, and L. V. Gool, "Deep Extreme Cut: From Extreme Points to Object Segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1-11. 616-625, 2018. (click-based + extreme points)
[IA+SA] Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections. ECCV, 2020. (user corrections as training samples)
A fully convolutional two-stream fusion network for interactive image segmentation. NN, 2019. (two-branch architecture)
[CM guidance] "Content-Aware Multi-Level Guidance for Interactive Instance Segmentation", In CVPR, 2019. (content-aware guidance maps)
[BRS] "Interactive Image Segmentation via Back-propagating Refinement Scheme", 2019. [Code] (BRS - improved f-BRS)
Interactive boundary prediction for object selection. In ECCV, 2018. (boundary clicks)
[Latent Diversity] "Interactive Image Segmentation With Latent Diversity", 2018. [Code] (couple two convolutional networks to train and select the proper result)
[RIS-Net] "Regional Interactive Image Segmentation Networks", 2017. [Code] (exploit the local region from click pairs)
[DIOS] N. Xu, B. Price, S. Cohen, J. Yang, T.S. Huang, "Deep Interactive Object Selection", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 373-381, 2016. (Click map transformation + random sampling technique)
Error-tolerant scribbles-based interactive image segmentation. In CVPR, 2014. (scribbles-based)
Milcut: A sweeping line multiple instance learning paradigm for interactive image segmentation. In CVPR, 2014. (scribbles-based)
P Gabbur, M Bilkhu, J Movellan, "Probabilistic Attention for Interactive Segmentation", Advances in Neural Information Processing Systems, 2021.
CT Lin, WC Tu, CT Liu, SY Chien, "Interactive Object Segmentation With Dynamic Click Transform", IEEE International Conference on Image Processing (ICIP), 2021.
J. Y. Oh, J. M. Park, "Interactive Part Segmentation Using Edge Images", Applied Sciences, 2021
C. Zhang, C.. Hu, Y Liu, X. He, "Intention-aware Feature Propagation Network for Interactive Segmentation", arXiv preprint arXiv:2203.05145, 2022.
[ECONet] M. Asad, L. Fidon, and T. Vercauteren, "ECONet: Efficient Convolutional Online Likelihood Network for Scribble-based Interactive Segmentation", Proceedings of Machine Learning Research (MIDL), 2022.
"Content-based Propagation of User Markings for Interactive Segmentation of Patterned Images"
M. Amrehn, R. Kortekaas, M. Strumia, M. Weingarten, M. Kowarschik, A. Maier "A Semi-Automated Usability Evaluation Framework for Interactive Image Segmentation Systems" [Code]
"DISIR: Deep Image Segmentation with Interactive Refinement"
[ClickSEG] X. Chen, Z. Zhao, F. Yu, Y. Zhang, and M. Duan, “Conditional Diffusion for Interactive Segmentation” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7345–7354, 2021. [Code] (Non-local method)
[EISeg] M. Xian, F. Xu, H. D. Cheng, Y. Zhang, and J. Ding, "EISeg: Effective interactive segmentation", 2016.
M. Forte, B. Price, S. Cohen, N. Xu, and F. Pitié, "Interactive Training and Architecture for Deep Object Selection", IEEE International Conference on Multimedia and Expo (ICME), 2020. [Code]
T. V. Spina, P. A. V. de Miranda, and A. X. Falcão, "Hybrid Approaches for Interactive Image Segmentation Using the Live Markers Paradigm", IEEE Transactions on Image Processing, Vol. 23, I. 12, pp. 5756-5769, 2014.
Interactive full image segmentation by considering all regions jointly. In CVPR, 2019 (click-based + extreme points)
D. Batra, A. Kowdle, D. Parikh, J. Luo, and T. Chen. Interactive Co-segmentation with Intelligent Scribble Guidance. In CVPR, 2010. (recommend where a user should scribble)
A. Friebel, T. Johann, D. Drasdo, S. Hoehme, "Guided interactive image segmentation using machine learning and color-based image set clustering", Bioinformatics, 2022
A number of interactive segmentation approaches and software packages that utilize machine learning have been published in recent years.
A supervoxel-based approach for segmentation of mitochondria in volumetric electron microscopy images was developed by Lucchi et al. (2012), though the published executable software and code encompassed only the supervoxel algorithm SLIC.
SuRVos (Luengo et al., 2017) is a software for interactive segmentation of 3D images using a supervoxel-based hierarchy, which lays a focus on the segmentation of noisy, low-contrast electron microscopy datasets.
The extendable software package Microscopy Image Browser (Belevich et al., 2016) allows for the processing of multidimensional datasets and features a selection of conventional processing algorithms, several region selection methods aiding manual segmentation, and methods for semi-automatic segmentation, including a supervoxel-based classification approach. FastER (Hilsenbeck et al., 2017) is designed specifically for cell segmentation in grayscale images, using features that are very fast to compute. The Fiji plugin Trainable Weka Segmentation (ArgandaCarreras et al., 2017) is a tool for pixel classification in 2D and 3D images using a broad range of features. Ilastik (Berg et al., 2019) provides a streamlined user interface with workflows for i.e. image segmentation, object classification and tracking as well as a sophisticated on-demand back end enabling processing of images with up to five dimensions that are larger than available RAM. Its segmentation workflow employs a random forest classifier for pixel classification from local features such as color, edgeness and texture at different scales.
[SAM] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, P. Dollar, and R. Girshick, "Segment Anything", Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4015-4026, 2023.
[MedSAM] J. Ma, Y. He, F. Li, L. Han, C. You, and B. Wang, Segment Anything in Medical Images, Nature Communications, 2024. (fine-tune only the mask decoder with bounding boxes as the prompts).
14 (fine-tune the entire network).
[SAMed] 15 (fine-tune only a small fraction of the parameters by using LoRA; modify the mask decoder to provide semantic information).
12 (introduce an adaptation technique).
[SAM-Adapter] 10 (address underperformance scenarios such as camouflage and shadow).
13 (present a visual prompt tuning method for ophthalmology).
16 (adding additional prompt).
[FastGeodis] M. Asad, R. Dorent, and T. Vercauteren, "FastGeodis: Fast Generalised Geodesic Distance Transform", arXiv preprint arXiv:2208.00001, 2022.
K Gotkowski, C Gonzalez, IJ Kaltenborn, R Fischbach, A. Bucher , A. Mukhopadhyay, "i3Deep: Efficient 3D interactive segmentation with the nnU-Net", Medical Imaging with Deep Learning (MIDL), 2022. [Code]
[iSegFormer] Q. Liu, Z. Xu, Y. Jiao, and M. Niethammer, "iSegFormer: Interactive Segmentation via Transformers with Application to 3D Knee MR Images", International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2022.
J. Deng, and X. Xie, "3D Interactive Segmentation With Semi-Implicit Representation and Active Learning", IEEE Transactions on Image Processing, 2021.
C. Ma, Q. Xu, X. Wang, B. Jin, X. Zhang, Y. Wang, and Y. Zhang, "Boundary-Aware Supervoxel-Level Iteratively Refined Interactive 3D Image Segmentation With Multi-Agent Reinforcement Learning", IEEE Transactions on Medical Imaging, Vol. 40, I. 10, pp. 2563 - 2574, 2020.
S. Cagnoni, A. B. Dobrzeniecki, R. Poli, and J. C. Yanch, "Genetic algorithm-based interactive segmentation of 3D medical images", Image and Vision Computing, 1999.
T Zhou, L Li, G Bredell, J Li, E Konukoglu, "Quality-Aware Memory Network for Interactive Image Segmentation", Medical Image Computing and Computer Assisted Intervention (MICCAI), pp 560–570, 2021.
[MiVOS] H. K. Cheng, Y. W. Tai, and C. K. Tang, "Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1-11. 5559-5, 2021. [Code] [Project Page] [Demo] [Papers with Code] (Scribble To Mask)
[STCN] H. K. Cheng, Y.-W. Tai, and C.-K. Tang, “Rethinking space-time networks with improved memory coverage for efficient video object segmentation,” Advances in Neural Information Processing Systems, vol. 34, 2021.
[GIS] Y Heo, YJ Koh, CS Kim, "Guided Interactive Video Object Segmentation Using Reliability-Based Attention Maps", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7322-7330, 2021.
"Active Frame Selection for Label Propagation in Videos" (actively select frames for label propagation)
S. W. Oh, J.-Y. Lee, N. Xu, and S. J. Kim, “Fast user-guided video object segmentation by interaction-and-propagation networks,” in CVPR, pp. 5247–5256, 2019.
A. Agarwala, A. Hertzmann, D. Salesin, and S. Seitz. Keyframe-Based Tracking for Rotoscoping and Animation. In SIGGRAPH, 2004. (the user corrects any mistakes along the way)
B. L. Price, B. S. Morse, and S. Cohen. Live-cut: Learning-based Interactive Video Segmentation by Evaluation of Multiple Propagated Cues. In ICCV, 2009. (the user corrects any mistakes along the way)
J. Wang, P. Bhat, R. A. Colburn, M. Agrawala, and M. F. Cohen. Interactive Video Cutout. In SIGGRAPH, 2005. (indicate foreground objects in a space-time volume)
X. Bai, J. Wang, D. Simons, and G. Sapiro. Video SnapCut: Robust Video Object Cutout using Localized Classifiers. In SIGGRAPH, 2009. (indicate foreground objects in a space-time volume)
M. Grundmann, V. Kwatra, M. Han, and I. Essa. Efficient Hierarchical Graph-based Video Segmentation. In CVPR, 2010. (graph-based clustering)
J. Fauqueur, G. Brostow, and R. Cipolla. Assisted Video Object Labeling by Joint Tracking of Regions and Keypoints. In Proc. Int. Workshop on Interactive Computer Vision, 2007. (tracking)
X. Ren and J. Malik. Tracking as Repeated Figure/Ground Segmentation. In CVPR, 2007. (random field models)
A. Vazquez-Reina, S. Avidan, H. Pfister, and E. Miller. Multiple Hypothesis Video Segmentation from Superpixel Flows. In ECCV, 2010. (random field models)
I. Patras, E. Hendriks, and R. Lagendijk. Semi-automatic Object-based Video Segmentation with Labeling of Color Segments. Signal Processing: Image Communication, 2003.
I. Budvytis, V. Badrinarayanan, and R. Cipolla. Label Propagation in Complex Video Sequences using Semi-supervised Learning. In BMVC, 2010.
V. Badrinarayanan, F. Galasso, and R. Cipolla. Label Propagation in Video Sequences. In CVPR, 2010.
D. Tsai, M. Flagg, and J. M.Rehg. Motion Coherent Tracking with Multi-label MRF Optimization. BMVC, 2010.
W. Wolf. Key Frame Selection by Motion Analysis. In ICASSP, 1996. (clustering)
M. Cooper and J. Foote. Discriminative Techniques for Keyframe Selection. ICME, 2005. (maximizing the dissimilarity between keyframes)
T. Liu and J. Kender. Optimization Algorithms for the Selection of Key Frame Sequences of Variable Length. In ECCV, 2002. (maximizing the dissimilarity between keyframes)
[MIDeepSeg] X. Luo, G. Wang, T Song, J. Zhang, M. Aertsen, J. Deprest, S. Ourselind, T. Vercauterend, and S. Zhangab , "MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning", Medical Image Analysis, vol. 72, pp. 102102, 2021. [Code] (click-based/ Gaussian-smoothed maps)
Interactive segmentation of medical images through fully convolutional neural networks
[MONAI Label] A. Diaz-Pinto, S. Alle, A. Ihsani, M. Asad, V. Nath, F. Perez-Garcıa, P. Mehta, W. Li, H. R. Roth, T. Vercauteren, et al., “Monai label: A framework for ai-assisted interactive labeling of 3d medical images,” arXiv preprint arXiv:2203.12362, 2022. (click-based and scribbles-based)
C. H. Chao, H. T. Cheng, T. Y. Ho, L. Lu, and M. Sun, “Interactive radiotherapy target delineation with 3d-fused context propagation,” arXiv preprint arXiv:2012.06873, 2020.
T. Sakinis, F. Milletari, H. Roth, P. Korfiatis, P. Kostandy, K. Philbrick, Z. Akkus, Z. Xu, D. Xu, and B. J. Erickson, “Interactive segmentation of medical images through fully convolutional neural networks,” arXiv preprint arXiv:1903.08205, 2019.
[DeepIGeoS] G. Wang, M. A. Zuluaga, W. Li, R. Pratt, P. A. Patel, M. Aertsen, T. Goal, A. L. David, J. Desprest, S. Ourselin, and T. Vercauteren, "DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 41, I. 7, 2019.
[WDTISeg] X. Li, M. Qiao, Y Guo, J. Zhou, S. Zhou, C. Chang, and Y. Wang, "WDTISeg: One-Stage Interactive Segmentation for Breast Ultrasound Image Using Weighted Distance Transform and Shape-Aware Compound Loss", Applied Sciences, 2021.
[TIS] W. Liu, C. Ma, Y. Yang, W. Xie, and Y. Zhang, "Transforming the Interactive Segmentation for Medical Imaging", International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2022.
[P-Net] G. Wang, W. Li, M. A. Zuluaga, R. Pratt, P. A. Patel, M. Aertsen, T. Doel, A. L. David, J. Deprest, S. Ourselin, and T. Vercauteren, "Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning", IEEE Transactions on Medical Imaging, Vol. 37, I. 7, 2018.
T Zhou, L Li, G Bredell, J Li, E Konukoglu, "Quality-Aware Memory Network for Interactive Volumetric Image Segmentation", Medical Image Computing and Computer Assisted Intervention, MICCAI, 2021.
"NuClick: A Deep Learning Framework for Interactive Segmentation of Microscopy Images"
"RootPainter: Deep Learning Segmentation of Biological Images with Corrective Annotation" [Code]
Balancing regional and global information: An interactive segmentation framework for ultrasound breast lesion
"iW-Net: an automatic and minimalistic interactive lung nodule segmentation deep network"
W Du, H Shen, G Zhang, X Yao, J Fu, "Interactive defect segmentation in X-Ray images based on deep learning", Expert Systems with Applications, 2022.
Interactive Whole-Heart Segmentation in Congenital Heart Disease
"Transforming the Interactive Segmentation for Medical Imaging"
PyMIC: A deep learning toolkit for annotation-efficient medical image segmentation
PA-Seg: Learning from Point Annotations for 3D Medical Image Segmentation using Contextual Regularization and Cross Knowledge Distillation
Semi-supervised medical image segmentation via uncertainty rectified pyramid consistency
Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer
HMRNet: High and Multi-Resolution Network With Bidirectional Feature Calibration for Brain Structure Segmentation in Radiotherapy
System and computer-implemented method for segmenting an image
[C3Det] C. Lee, S. Park, H. Song, J. Ryu, S. Kim, H. Kim, S. Pereira, and D. Yoo, "Interactive Multi-Class Tiny-Object Detection", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14136-14145, 2022.
[DARos] L. Wang, R. Benmokhtar, and X. Perrotton, "Interactive Deep Annotation as DARos: Object Detection Supervision for Efficient Instance Segmentation", International Conference on Image Analysis and Processing (ICIAP), 2022.
[FI-BRS] Q. Wang, Y. Pang, M. Wang, M. Cui, "FI-BRS: A Few-Interaction Backpropagating Refinement Scheme for Video Object Segmentation", 3rd International Academic Exchange Conference on Science and Technology Innovation (IAECST), 2021.
"SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting"
"A Baseline Statistical Method For Robust User-Assisted Multiple Segmentation"
C Xu, B Dong, N Stier, C McCully, DA Howell, P Sen, T. Höllerer , "Interactive Segmentation and Visualization for Tiny Objects in Multi-megapixel Images", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 21447-21452, 2022.
Efficient interactive annotation of segmentation datasets with polygon-rnn++. In CVPR, 2018. (poligon-based)
Annotating object instances with a polygon-rnn. In CVPR, 2017. (poligon-based)
Fast interactive object annotation with curve-gcn. In CVPR, 2019. (poligon-based)
Iteratively refined interactive 3d medical image segmentation with multi-agent reinforcement learning. In CVPR, 2020.
Automatic seed generation with deep reinforcement learning for robust interactive segmentation. In CVPR, 2018.
HAZN [49] - Zoom better to see clearer: Human and object parsing with hierarchical auto-zoom net. In ECCV, 2016. (can adaptively adjust the scale of view to the object or the part to refine the segmentation)
GLNet [8] - Collaborative global-local networks for memory-efficient segmentation of ultra-high resolution images. In CVPR, 2019. (aggregates feature maps captured by local and global branches).
AWMF-CNN [44] - Adaptive weighting multi-field-of-view cnn for semantic segmentation in pathology. In CVPR, 2019. (semantic segmentation - assigns weights to different magnifications of local patches separately).
CascadePSP [11] - Cascadepsp: toward class-agnostic and very highresolution segmentation via global and local refinement. In CVPR, 2020. (feeds image patches from the original image through the refinement module).
MagNet [18] - Progressive semantic segmentation. In CVPR, 2021. (refines segmentation results of local patches with different scales in a progressive way).
J. He, C. S. Kim, C. C. J. Kuo, "Interactive Segmentation Techniques" Algorithms and Performance Evaluation", 2014.
https://github.com/PaddlePaddle/PaddleSeg: ElSeg, Pace, EdgeFlow
[iSegformer] Although these CNN-based methods have achieved excellent performance, they suffer from limited receptive fields and cannot learn global and long-range semantic information well due to the inductive bias of locality and weight sharing [N. Cohen and A. Shashua, “Inductive bias of deep convolutional networks through pooling geometry,” arXiv preprint arXiv:1605.06743, 2016.].
==> Several techniques have been proposed to address these problems, such as atrous convolutional layers [9] and non-local blocks [10]. A recent research direction is to replace CNN with vision Transformer (ViT), which can naturally capture long-range dependencies through the self-attention mechanism [11, 12]
Combine context: SIFT
Join Interactive with Boundary Detection
Combine Deep Clustering:
[DIC] L. Zhou, and W. Wei, "DIC: Deep Image Clustering for Unsupervised Image Segmentation, IEEE Access, 2020.
A Friebel, T Johann, D Drasdo, S Hoehme, "Guided interactive image segmentation using machine learning and color-based image set clustering", Bioinformatics, 2022
Combine Optical Flow:
[RAFT] Z. Teed, and J. Deng, "RAFT: Recurrent All-Pairs Field Transforms", European conference on computer vision (ECCV), 2020. [Fast-read] (Transformer-based Optical Flow)
Supervoxel:
SD Jain, K Grauman, "Supervoxel-consistent foreground propagation in video", in European conference on computer vision, (ECCV) 2014
[SuRVoS] https://diamondlightsource.github.io/SuRVoS/docs/tutorials/2016_11_SuRVoS_Workshop_Final_comp.pdf
Active Contours (Snakes): Curves or shapes that iteratively evolve toward object boundaries guided by image features and user input.
Conditional Random Fields (CRFs): Refining pixel-level classifications considering relationships between neighboring pixels.
https://scholar.google.com/citations?user=Z2sFN4EAAAAJ&hl=en (Professor doing research IS with Medical Images)
https://github.com/michaelowenliu/awesome-interactive-segmentation
https://github.com/michaelowenliu/awesome-image-matting Mating ???
Done
A Web Interface for 3D Visualization and Interactive Segmentation of Medical Images: https://www.youtube.com/watch?v=KMRvHGKmhBk
Interactive 3D segmentation of medical images: https://www.youtube.com/watch?v=eIOy4BGOl1c
Iteratively-Refined Interactive 3D Medical Image Segmentation with Multi-Agent Reinforcement Learning: https://www.youtube.com/watch?v=z9ouWEwyTPg
Introduction to CT Chest - Anatomy and Approach: https://www.youtube.com/watch?v=tJPfcmA-WWs
