---
tags:
  - ai
---
- [Skip Connections](Skip%20Connections.md) across individual layers
	- Conditionally
- Soft gates
	- Learn vs carry
- Gradients propagate further
- Inspired by [LSTM](../../RNN/LSTM.md) [RNN](../../RNN/RNN.md)s

![highway-vs-residual](../../../../img/highway-vs-residual.png)
![skip-connections 1](../../../../img/skip-connections%201.png)