sarsoo/stem
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
f65496a79f
stem/AI/Neural Networks/MLP.md
andy f65496a79f vault backup: 2023-05-23 09:28:54 
Affected files:
.obsidian/workspace.json
STEM/AI/Neural Networks/MLP.md
STEM/AI/Neural Networks/MLP/Activation Functions.md
STEM/AI/Neural Networks/MLP/Back-Propagation.md
STEM/AI/Neural Networks/MLP/Decision Boundary.md
STEM/img/hidden-neuron-decision.png
STEM/img/mlp-non-linear-decision.png
STEM/img/mlp-xor-2.png
STEM/img/mlp-xor.png
STEM/img/sigmoid.png
STEM/img/tlu.png
2023-05-23 09:28:54 +01:00

765 B
Raw Blame History

  • Feed-forward
  • Single hidden layer can learn any function
    • Universal approximation theorem
  • Each hidden layer can operate as a different feature extraction layer
  • Lots of weights to learn
  • Back-Propagation is supervised

!mlp-arch.png

Universal Approximation Theory

A finite feed-forward MLP with 1 hidden layer can in theory approximate any mathematical function

!activation-function.png !mlp-arch-diagram.png

Weight Matrix

  • Use matrix multiplication for layer output
  • TLU is hard limiter !tlu.png
  • o_1 to o_4 must all be one to overcome -3.5 bias and force output to 1 !mlp-non-linear-decision.png
  • Can generate a non-linear decision boundary