0. Motivation, Objective and Related Works:

Motivation:

• Defects in the textile manufacturing process lead to a great waste of resources and further affect the quality of textile products. 

• Automated quality guarantee of textile fabric materials is one of the most important and demanding computer vision tasks in textile smart manufacturing.

Objectives:

• This paper presents a systematic literature review on automatic fabric defect detection methods of the textile industry smart manufacturing. 

  1. Introduces the importance and inevitability of fabric defect detection towards the era of manufacturing of artificial intelligence. 

  2. Two main categories: (1) traditional algorithms and (2) learning-based algorithms. Traditional algorithms are further categorized into statistical, structural, spectral, and model-based algorithms. The learning-based algorithms are further divided into conventional machine learning algorithms and deep learning algorithms. 

  3. The deployments of fabric defect detection algorithms are discussed in this study.

1.1.3. Structural Approach: 

1. Ngan et al.: wavelet preprocessed golden image subtraction (WGIS) [34].

   • (Previous work) The golden image subtraction (GIS) method is used for segmenting defective areas on the patterned textile fabric image.

2. Jia and Liang [35]: segment the fabric images into non-overlapping regions named lattices and then the similarity of these lattices are calculated in the feature space. 

   • The proposed Isotropic Lattice Segmentation (ILS) method shows satisfying results on the box and star pattern image database. 

   • Another approach [36] is on the basis of lattice segmentation and lattice templates. In this study, the lattices are segmented according to different placement rules of texture primitives that belong to different classes. The distances of undetermined lattice and lattice templates are calculated, and the lattices are regarded as the defective area when the distances are larger than a certain threshold. 

   • The algorithm is further improved by adding template statistics which are learned from defect-free images in [37]. 

3. Chang’s work [38]: Template-based correction approach for fabric images with periodical structure.

   • A fabric image is divided into lattices due to variation regularity and correction is then made to reduce the lattice mis-alignment. 

   • The defective lattices are first located and defect regions are segmented at a later step. 

4. Shi et al. [39]: using low-rank decomposition of gradient information combined with structured graph. 

   • The fabric image is first divided into defect-free regions and defect regions based on the structured graph information. 

   • Adaptive thresholding is utilized during the lattice merging step. 

   • Finally, the matrix decomposition is calculated under the prior information from the segmentation results; thus the defect regions are emphasized. 

5. Abouelela et al. [40]: employing simple statistical features such as median, mean, and variance. 

   • The author holds that time efficiency is crucial to any industrial procedure. 

1.1.4. Model-Based Methods

1. Ngan et al.: propose motif-based methods for detecting defects in 2D patterned texture. 

   • Patterned images can be divided into lattices and motifs. 

   • And further energy of moving subtraction is calculated to differentiate defective and defect-free regions [41].

   • In order to reduce the detection rate of false positives and false negatives, the Gaussian mixture model is used to represent the energy variance value [42].

   • K-means clustering is applied to the data and the convex hull of each cluster is calculated in that fitting ellipsoidal region.

2. Lucia et al.: detecting the fabric defects in uniformly structured textile fabric images. 

   • The algorithm includes two stages: the feature extraction stage and the defect recognition stage. 

   • In the first stage, the symmetric Gabor filter bank and principal component analysis are utilized for feature extraction

   • In the second stage, the Euclidean norm of features is calculated and compared for defect recognition. 

3. Shu et al. [44]: Using principal component analysis and nonlocal average filtering to enhance the fabric texture and reduce the noise. 

   • A texture-based defect measurement method is used for calculating the similarity; thus defect and non-defect areas can be distinguished. 

4. Some researchers treat the fabric defect detection problem as a one-class classification. 

   • Bu et al. [45, 46]: based on the support vector data description (SVDD) model.

   • Bu et al. [47]: the features extracted in this method are based on autoregressive (AR) spectral estimation model combined with Burg algorithm. 

5. Campbell et al. [48]: propose two model-based methods in their study. 

   • The first method states the maximum likelihood of image binarization. 

   • Another is mainly for defect detection in repeated weaving patterns; thus the discrete Fourier transformation is utilized for texture analysis. 

   • In the end, a model-based clustering method is applied to delineate the defective regions. 

6. Tsang et al. [49]: propose a method named Elo rating (ER).

   • The fabric images are divided into standardsize partitions. 

   • Matches are calculated between partitions and revised through an Elo point matrix. 

   • The defect area (partition) will win the competition as a powerful player. This presented method was tested on dot-, star-, and box-patterned fabrics database.

1.2.1. Classical Machine Learning Algorithms:

(1) Dictionary Learning-Based Algorithms:

1. [50, 51]: 

   • A dictionary is learned from the training or test image.

   • Then a non-defective image is reconstructed using the learned dictionary.

   • Thenceforth the detection is implemented by subtracting the reconstructed image from the test image. 

2. Li et al.: propose an algorithm on the basis of biological vision modeling. 

   • The biological visual saliency is modeled by low-rank representation (LRR); 

   • The fabric image is decomposed into salient defect regions and defect-free backgrounds [52]. 

3. Li et al. [53]: model a defect-free region as a low-rank structure and the defect region as a sparse structure. 

   • Thus a fabric image can be regarded as the sum of a low-rank matrix and a sparse matrix. 

   • For dimensionality reduction, uses eigenvalue decomposition on blocked image matrix. 

4. Shi et al. [39]: point out two shortcomings of the low-rank decomposition. 

   • One is that existing low-rank decomposition models barely detect the defect regions with high gradients. 

   • Another shortcoming is that small defect area or complex area will be incorrectly segmented given the inaccuracy of prior information. 

   • Propose a low-rank decomposition method utilizing gradient information combined with a structured graph algorithm. 

5. Table 3: other dictionary learning-based algorithms using low-rank decomposition. 

(2) Traditional machine learning algorithms:

1. KNN [63] and neural network [64]: feature engineering is one of the major processes in the machine learning life cycle. 

2. Mak et al.: extracted [65] four novel fractal features and employed support vector data description (SVDD), which is a support vector machine learning algorithm used for one-class classification. 

3. Zhang et al.: employing the radial basis function (RBF) network. 

   • Gaussian mixture model (GMM) is utilized to improve the accuracy of Gaussian RBF parameter estimation. 

   • The validity of the proposed method has been proved on multiple class datasets [66].

4. Tian and Li [67]: propose an autoencoder-based method 

   • By exploring similarities between image patches. 

   • Utilizing the repeated texture pattern, similar non-defective patches were found for each candidate defect patch and the corresponding latent variables were weighted and combined according to which the original latent variable can be modified. 

5. Yapi et al.: [68] consider this problem as a binary problem. 

   • A compact and accurate feature set was extracted by statistical modeling of multiscale contourlet decomposition.

   • Then a Bayesian classifier (BC) is used to classify the defect and non-defect classes. 

6. Some other traditional machine learning algorithms are shown in Table 4. 

1.2.2. Deep Learning Algorithms [72] [73] [74] [75] [76] [77]

• Classical deep learning algorithms for object detection are listed in Table 5. 

(1) One-Stage Detection Algorithms: Does not have a separate proposal generation phase. These algorithms treat all locations on the image as potential objects and manage to categorize each interest region into a target object or background. 

1. SSD: for the fabrics defect scenario by Liu et al. [78]

2. Ouyang et al.: [79] introduces a dynamic activation layer utilizing the defect probability information with a pairwise potential function to a CNN. 

3. Liu et al.: [80] limited resources. 

4. Zhou et al.: Efficient Defect Detectors (EDDs) [81]. 

   • To extract more low-level features, EDDs adjust the input resolution, depth, and width using a scaling strategy. 

5. Xu et al.: [82] de-deformation defect detection network (D4Net). 

   • This model is composed of reference generation, de-deformation network, and marginal loss. 

   • The most suitable reference is selected and paired with the input image and then is sent into the de-deformation network. 

   • The dissimilarities are calculated and enhanced by the marginal loss. 

6. Peng et al.: Priori Anchor Convolutional Neural Network (PRAN-Net). 

   • Feature Pyramid Network (FPN) is utilized to selected multi-scale feature maps and then sparse priori anchors are generated based on ground truth boxes [83]. 

7. Li et al.: [84] using several microarchitectures. 

   • The microarchitecture is constructed of multiscale analysis, filter factorization, multilocation pooling, and parameters reduction, thus making the network a compact one. . 

(2) Two-Stage Detection Algorithms: A sparse set of proposals is generated in the first stage and in the second stage, and the features of generated proposals are sent into DCNN for prediction results. 

1. [85–87]: utilize the modified Faster R-CNN model.

2. Jun et al.: [88] utilized the Inception-V1 model and LeNet-5 model. 

   • This approach includes local defect prediction in the first stage and global defect recognition in the second stage. 

3. Table 6 lists some other deep learning algorithms utilized in textile fabric defect applications. 

(3) Generative Adversarial Networks (GANs): GAN-based algorithms can automatically adapt to different fabric textures by learning existing fabric defect samples [99]. 

1. Liu et al.: design a deep semantic segmentation network to detect fabric defects. 

   • They train a multistage GAN model to synthesize reasonable defect samples from non-defect samples. 

2. Le et al.: [100] utilize Wasserstein generative adversarial nets (WGANs) combined with transfer learning techniques and multimodel ensembling framework.. 

(4) Long-short-term memory: 

1. Zhao et al.: [101] describe a CNN model based on visual long-short-term memory (VLSTM). 

   • Three types of features, visual perception features, visual short-term memory (VSTM) features, and visual long-term memory (VLTM) features, are extracted by stacked convolutional autoencoders, a shallow CNN, and nonlocal neural networks, respectively. 

(5) Attention mechanism: 

1. Wang et al.: [102] propose a deep saliency detection model that incorporated self-attention mechanism into a CNN.

   • Multiscale feature maps are generated from a fully convolutional network, and a self-attention module is used to coordinate the dependence between the features of different layers. 

   • The self-attention mechanism in this algorithm proved to be very effective with complex or blurred defects. 

(6) Combination: 

1. Wang et al.: [103] extracted global deep features using CNN in combination with handcrafted low-level features, and nonconvex robust PCA regularized by nonconvex total variation are employed to data processing and noise reduction. 

2. Application and Deployment:

• Engineering implementation problems [104, 105]. 

• Realizing an intelligent textile system in the real textile manufacturing process covers many aspects, involving the Internet of Things (IoT) [106], cyber-physical systems (CPS) [107], and more [108, 109]. 

2.1. Hardware Selection of Detection System.

• Hardware such as cameras, lens, lights, and frame grabber is an important factor. 

• Different hardware corresponds to different subsequent algorithms. 

• Yildiz et al. [111] using a thermal camera.

• Fang et al. [112] introduce a tactile inspection system, mainly based on a visual tactile sensor, which consists of several LEDs, a camera, and an elastic sensing layer. 

• The conveyor belt used for conveying cloth on the production line will also affect the image taking speed [113]. 

2.2. Dataset. 

• It is very difficult to collect a fair amount of fabric defect image data in industrial scenes [114]. 

• Employ semisupervised and unsupervised learning algorithms for the detection [115]. 

• In addition, some studies utilize non-defect image data and synthetic defective image data generated by using defect characteristics based on expert knowledge [91]. 

  1. Chen et al. [116] propose a data augmentation method based on automatic image acquisition. Different image acquisition angles, various acquisition scenes, and random illumination conditions are designed for image collection as a simulation under the actual textile production scene. 

• Unbalanced categories bring actual challenges to fabric defect detection [86]. 

2.3. Real Time of the Algorithm. 

• Limited computing power. 

• Shunji et al. [119] design a detection method for tubular knitted fabric which is produced by a circular knitting machine. 

• The embedded system proposed by Schmitt et al. [120] ensures that all steps of image acquisition, processing, and evaluation can be executed in real time.

• Currently, the accuracy of existing detection models is low. We must consider ways to reduce the size of the model. Inspired by the successful use of deep convolutional neural networks (DCNN) for target detection, we propose a wide-and-light network structure called WALNet. 

3. Discussion:

• Textile manufacturing companies need to upgrade equipment and technology to maintain growth and competitiveness. 

• The sensing, storage, and computing capabilities of automated fabric detection systems based on computer vision will continue to improve. 

• There is a lot of work that needs to be done during the whole textile manufacturing process. 

• In the future, more work needs to be done in the process of moving towards Industry 4.0 [122, 123]. 

  1. Smart manufacturing integrates various technologies, covering robotics, CPS, IoT, big data analytics [124], and cloud computing. 

  2. CPS is an engineering system that seamlessly integrates physical and computational components [125]. 

  3. Adding artificial intelligence, big data analysis, and cloud services to the IoT ecosystem is the key development direction of CPS in the future.