andy
7bc4dffd8b
Affected files: STEM/AI/Neural Networks/CNN/Examples.md STEM/AI/Neural Networks/CNN/FCN/FCN.md STEM/AI/Neural Networks/CNN/FCN/ResNet.md STEM/AI/Neural Networks/CNN/FCN/Skip Connections.md STEM/AI/Neural Networks/CNN/GAN/DC-GAN.md STEM/AI/Neural Networks/CNN/GAN/GAN.md STEM/AI/Neural Networks/CNN/Interpretation.md STEM/AI/Neural Networks/CNN/UpConv.md STEM/AI/Neural Networks/Deep Learning.md STEM/AI/Neural Networks/MLP/MLP.md STEM/AI/Neural Networks/Properties+Capabilities.md STEM/AI/Neural Networks/SLP/Least Mean Square.md STEM/AI/Neural Networks/SLP/SLP.md STEM/AI/Neural Networks/Transformers/Transformers.md STEM/AI/Properties.md STEM/CS/Language Binding.md STEM/CS/Languages/dotNet.md STEM/Signal Proc/Image/Image Processing.md
30 lines
897 B
Markdown
30 lines
897 B
Markdown
- Residual networks
|
|
- 152 layers
|
|
- Skips every two layers
|
|
- Residual block
|
|
- Later layers learning the identity function
|
|
- Skips help
|
|
- Deep network should be at least as good as shallower one by allowing some layers to do very little
|
|
- Vanishing gradient
|
|
- Allows shortcut paths for gradients
|
|
- Accuracy saturation
|
|
- Adding more layers to suitably deep network increases training error
|
|
|
|
# Design
|
|
|
|
- Skips across pairs of [conv layers](../Convolutional%20Layer.md)
|
|
- Elementwise addition
|
|
- All layer 3x3 kernel
|
|
- Spatial size halves each layer
|
|
- Filters doubles each layer
|
|
- [Fully convolutional](FCN.md)
|
|
- No fc layer
|
|
- No [pooling](../Max%20Pooling.md)
|
|
- Except at end
|
|
- No dropout
|
|
|
|
[ImageNet](../../CV/Datasets.md#ImageNet) Error:
|
|
|
|
|
|
|
|
 |