Convolution is an important component of neural networks. Different convolution structures have different functions but are essentially used for feature extraction. In traditional image processing, different operators are designed to extract fixed features such as edge, horizontal, and vertical.
In CNN, a fixed-size convolution kernel is randomly initialized, and parameters are automatically updated through BP.
0) Convolutional Layer Types:
The original convolution (Vanilla Convolution)
Group Convolution (Group Convolution)
Transposed Convolution
1x1 convolution
Hollow convolution
Depthwise separable convolution
Deformable convolution
Spatially separable convolution
Graph convolution
Inception block
Asymmetric convolution
Octave Convolution
Heterogeneous convolution HeConv
Conditional convolution
Dynamic convolution
Ghost Block
Adaptive convolution SCConv
DO-Conv
ResNeSt Block
Internal volume
DSConv
BSConv
1) One-dimensional Convolution:
From a signal perspective, convolution can be defined as the weighted output of a function (blue unit response square wave) on an input signal (red square wave) (the black triangle is the convolution result).
There are two definitions for dividing convolution from the type of signal:
2) Two-dimensional Convolution:
The original convolution (Vanilla Convolution)
Unlike traditional hand-designed filter operators, the convolution kernel in CNN can update parameters based on, for example, the gradient descent method to better adapt to the distribution of the input image. By continuously stacking convolutional layers through the network, the network can be gradient to a rich Hierarchical feature representation (Hierarchical feature representation).
Group Convolution (Group Convolution)
AlexNet: The earliest grouped convolution is due to the limitation of the computing power of a single GPU, and the network segmentation can be calculated in parallel on multiple GPUs.
References:
Convolution variants 1: https://zhuanlan.zhihu.com/p/393200454
Convolution variants 2: https://zhuanlan.zhihu.com/p/393496515
4 common convolutional layers: https://zhuanlan.zhihu.com/p/267249291
Pooling: https://zhuanlan.zhihu.com/p/393761765
