stem/AI/Neural Networks/RNN
andy 3606944190 vault backup: 2023-06-08 17:52:08
Affected files:
.obsidian/graph.json
.obsidian/workspace-mobile.json
.obsidian/workspace.json
Food/Meal Plan.md
Lab/Linux/Alpine.md
Lab/Linux/KDE.md
Lab/Scratch Domain.md
Lab/Windows/Active Directory.md
Languages/Spanish/README.md
Languages/Spanish/Spanish.md
Money/Accounts.md
Money/Monthly/23-04.md
Money/Monthly/23-05.md
Money/Monthly/23-06.md
Projects/Mixonomer.md
Projects/NoteCrawler.md
Projects/Projects.md
Projects/README.md
Projects/Selector.md
Projects/To Do App.md
Projects/img/selector-arch.png
STEM/AI/Classification/Supervised/SVM.md
STEM/AI/Neural Networks/CNN/FCN/Super-Resolution.md
STEM/AI/Neural Networks/CNN/GAN/GAN.md
STEM/AI/Neural Networks/CNN/GAN/cGAN.md
STEM/AI/Neural Networks/CNN/Interpretation.md
STEM/AI/Neural Networks/Deep Learning.md
STEM/AI/Neural Networks/MLP/MLP.md
STEM/AI/Neural Networks/Properties+Capabilities.md
STEM/AI/Neural Networks/RNN/Representation Learning.md
STEM/AI/Pattern Matching/Markov/Markov.md
STEM/AI/Searching/Informed.md
STEM/AI/Searching/README.md
STEM/AI/Searching/Searching.md
STEM/AI/Searching/Uninformed.md
STEM/CS/Languages/Javascript.md
STEM/CS/Languages/Python.md
STEM/CS/Languages/dotNet.md
STEM/CS/Resources.md
STEM/IOT/Cyber-Physical Systems.md
STEM/IOT/Networking/Networking.md
STEM/IOT/Networking/README.md
STEM/IOT/Software Services.md
STEM/img/cyberphysical-social-data.png
STEM/img/cyberphysical-system-types.png
STEM/img/cyberphysical-systems.png
STEM/img/depth-first-cons.png
STEM/img/depth-first.png
STEM/img/iot-mesh-network.png
STEM/img/iot-network-radar.png
STEM/img/iot-network-types 1.png
STEM/img/iot-network-types.png
STEM/img/markov-start-end-matrix.png
STEM/img/markov-start-end-probs.png
STEM/img/markov-start-end.png
STEM/img/markov-state-duration.png
STEM/img/markov-state.png
STEM/img/markov-weather.png
STEM/img/search-bidirectional.png
STEM/img/search-breadth-first.png
STEM/img/search-lim-goal.png
STEM/img/search-lim1.png
STEM/img/search-lim2.png
STEM/img/search-lim3-2.png
STEM/img/search-lim3.png
STEM/img/search-lim4.png
STEM/img/searching-graph-tree.png
STEM/img/searching-graph.png
Work/Freelancing.md
2023-06-08 17:52:09 +01:00
..
Autoencoder.md vault backup: 2023-06-07 09:02:27 2023-06-07 09:02:27 +01:00
Deep Image Prior.md vault backup: 2023-06-07 09:02:27 2023-06-07 09:02:27 +01:00
LSTM.md vault backup: 2023-06-05 17:01:29 2023-06-05 17:01:29 +01:00
MoCo.md vault backup: 2023-06-07 09:02:27 2023-06-07 09:02:27 +01:00
README.md vault backup: 2023-05-31 21:29:04 2023-05-31 21:29:04 +01:00
Representation Learning.md vault backup: 2023-06-08 17:52:08 2023-06-08 17:52:09 +01:00
RNN.md vault backup: 2023-06-05 17:01:29 2023-06-05 17:01:29 +01:00
SimCLR.md vault backup: 2023-06-07 09:02:27 2023-06-07 09:02:27 +01:00
VQA.md vault backup: 2023-06-05 17:01:29 2023-06-05 17:01:29 +01:00

Recurrent Neural Network

  • Hard to train on long sequences
    • Weights hold memory
      • Implicit
    • Lots to remember

Text Analysis

  • Train sequences of text character-by-character
    • Maintains state vector representing data up to current token
    • Combines state vector with next token to create new vector
    • In theory, info from one token can propagate arbitrarily far down the sequence
      • In practice suffers from vanishing gradient
        • Can't extract precise information about previous tokens

rnn-input rnn-recurrence