andy
3606944190
Affected files: .obsidian/graph.json .obsidian/workspace-mobile.json .obsidian/workspace.json Food/Meal Plan.md Lab/Linux/Alpine.md Lab/Linux/KDE.md Lab/Scratch Domain.md Lab/Windows/Active Directory.md Languages/Spanish/README.md Languages/Spanish/Spanish.md Money/Accounts.md Money/Monthly/23-04.md Money/Monthly/23-05.md Money/Monthly/23-06.md Projects/Mixonomer.md Projects/NoteCrawler.md Projects/Projects.md Projects/README.md Projects/Selector.md Projects/To Do App.md Projects/img/selector-arch.png STEM/AI/Classification/Supervised/SVM.md STEM/AI/Neural Networks/CNN/FCN/Super-Resolution.md STEM/AI/Neural Networks/CNN/GAN/GAN.md STEM/AI/Neural Networks/CNN/GAN/cGAN.md STEM/AI/Neural Networks/CNN/Interpretation.md STEM/AI/Neural Networks/Deep Learning.md STEM/AI/Neural Networks/MLP/MLP.md STEM/AI/Neural Networks/Properties+Capabilities.md STEM/AI/Neural Networks/RNN/Representation Learning.md STEM/AI/Pattern Matching/Markov/Markov.md STEM/AI/Searching/Informed.md STEM/AI/Searching/README.md STEM/AI/Searching/Searching.md STEM/AI/Searching/Uninformed.md STEM/CS/Languages/Javascript.md STEM/CS/Languages/Python.md STEM/CS/Languages/dotNet.md STEM/CS/Resources.md STEM/IOT/Cyber-Physical Systems.md STEM/IOT/Networking/Networking.md STEM/IOT/Networking/README.md STEM/IOT/Software Services.md STEM/img/cyberphysical-social-data.png STEM/img/cyberphysical-system-types.png STEM/img/cyberphysical-systems.png STEM/img/depth-first-cons.png STEM/img/depth-first.png STEM/img/iot-mesh-network.png STEM/img/iot-network-radar.png STEM/img/iot-network-types 1.png STEM/img/iot-network-types.png STEM/img/markov-start-end-matrix.png STEM/img/markov-start-end-probs.png STEM/img/markov-start-end.png STEM/img/markov-state-duration.png STEM/img/markov-state.png STEM/img/markov-weather.png STEM/img/search-bidirectional.png STEM/img/search-breadth-first.png STEM/img/search-lim-goal.png STEM/img/search-lim1.png STEM/img/search-lim2.png STEM/img/search-lim3-2.png STEM/img/search-lim3.png STEM/img/search-lim4.png STEM/img/searching-graph-tree.png STEM/img/searching-graph.png Work/Freelancing.md |
||
|---|---|---|
| .. | ||
| Autoencoder.md | ||
| Deep Image Prior.md | ||
| LSTM.md | ||
| MoCo.md | ||
| README.md | ||
| Representation Learning.md | ||
| RNN.md | ||
| SimCLR.md | ||
| VQA.md | ||
Recurrent Neural Network
- Hard to train on long sequences
- Weights hold memory
- Implicit
- Lots to remember
- Weights hold memory
Text Analysis
- Train sequences of text character-by-character
- Maintains state vector representing data up to current token
- Combines state vector with next token to create new vector
- In theory, info from one token can propagate arbitrarily far down the sequence
- In practice suffers from vanishing gradient
- Can't extract precise information about previous tokens
- In practice suffers from vanishing gradient
