andy
23991f92c9
Affected files: .obsidian/app.json .obsidian/workspace-mobile.json .obsidian/workspace.json Events/🪣🪣🪣.md STEM/AI/Neural Networks/CNN/FCN/README.md STEM/AI/Neural Networks/CNN/GAN/README.md STEM/AI/Neural Networks/CNN/README.md STEM/AI/Neural Networks/MLP/README.md STEM/AI/Neural Networks/README.md STEM/AI/Neural Networks/RNN/README.md STEM/AI/Neural Networks/SLP/README.md STEM/AI/Neural Networks/Transformers/README.md Untitled.canvas |
||
|---|---|---|
| .. | ||
| LSTM.md | ||
| README.md | ||
| RNN.md | ||
| VQA.md | ||
Recurrent Neural Network
- Hard to train on long sequences
- Weights hold memory
- Implicit
- Lots to remember
- Weights hold memory
Text Analysis
- Train sequences of text character-by-character
- Maintains state vector representing data up to current token
- Combines state vector with next token to create new vector
- In theory, info from one token can propagate arbitrarily far down the sequence
- In practice suffers from vanishing gradient
- Can't extract precise information about previous tokens
- In practice suffers from vanishing gradient
!
!