9 lines
215 B
Markdown
9 lines
215 B
Markdown
|
- [[Skip connections]] across individual layers
|
||
|
|
- Conditionally
|
||
|
|
- Soft gates
|
||
|
|
- Learn vs carry
|
||
|
|
- Gradients propagate further
|
||
|
|
- Inspired by [[LSTM]] [[RNN]]s
|
||
|
|
|
||
|
|
![[highway-vs-residual.png]]
|
||
|
|
![[skip-connections 1.png]]
|