2023-05-27 22:17:56 +01:00
|
|
|
- Residual networks
|
|
|
|
- 152 layers
|
|
|
|
- Skips every two layers
|
|
|
|
- Residual block
|
|
|
|
- Later layers learning the identity function
|
|
|
|
- Skips help
|
|
|
|
- Deep network should be at least as good as shallower one by allowing some layers to do very little
|
|
|
|
- Vanishing gradient
|
|
|
|
- Allows shortcut paths for gradients
|
|
|
|
- Accuracy saturation
|
|
|
|
- Adding more layers to suitably deep network increases training error
|
|
|
|
|
|
|
|
# Design
|
|
|
|
|
2023-05-27 23:02:51 +01:00
|
|
|
- Skips across pairs of [[Convolutional Layer|conv layers]]
|
2023-05-27 22:17:56 +01:00
|
|
|
- Elementwise addition
|
|
|
|
- All layer 3x3 kernel
|
|
|
|
- Spatial size halves each layer
|
|
|
|
- Filters doubles each layer
|
2023-05-27 23:02:51 +01:00
|
|
|
- [[FCN|Fully convolutional]]
|
2023-05-27 22:17:56 +01:00
|
|
|
- No fc layer
|
2023-05-27 23:02:51 +01:00
|
|
|
- No [[Max Pooling|pooling]]
|
2023-05-27 22:17:56 +01:00
|
|
|
- Except at end
|
|
|
|
- No dropout
|
|
|
|
|
2023-05-31 17:33:05 +01:00
|
|
|
[[Datasets#ImageNet|ImageNet]] Error:
|
2023-05-27 23:02:51 +01:00
|
|
|
![[imagenet-error.png]]
|
|
|
|
|
|
|
|
![[resnet-arch.png]]
|
|
|
|
![[resnet-arch2.png]]
|