stem/AI/Neural Networks/MLP
andy 4cc2e79866 vault backup: 2023-05-31 22:21:56
Affected files:
.obsidian/global-search.json
.obsidian/workspace.json
Health/Alexithymia.md
Health/BWS.md
STEM/AI/Neural Networks/Activation Functions.md
STEM/AI/Neural Networks/Architectures.md
STEM/AI/Neural Networks/CNN/CNN.md
STEM/AI/Neural Networks/MLP/Back-Propagation.md
STEM/AI/Neural Networks/Transformers/Attention.md
STEM/CS/Calling Conventions.md
STEM/CS/Languages/Assembly.md
2023-05-31 22:21:56 +01:00
..
Back-Propagation.md vault backup: 2023-05-31 22:21:56 2023-05-31 22:21:56 +01:00
Decision Boundary.md vault backup: 2023-05-23 09:28:54 2023-05-23 09:28:54 +01:00
MLP.md vault backup: 2023-05-27 22:17:56 2023-05-27 22:17:56 +01:00
README.md vault backup: 2023-05-31 21:29:04 2023-05-31 21:29:04 +01:00

  • Architectures
  • Single hidden layer can learn any function
    • Universal approximation theorem
  • Each hidden layer can operate as a different feature extraction layer
  • Lots of Weight Init to learn
  • Back-Propagation is supervised

!mlp-arch.png

Universal Approximation Theory

A finite Architectures MLP with 1 hidden layer can in theory approximate any mathematical function

!activation-function.png !mlp-arch-diagram.png

Weight Matrix

  • Use matrix multiplication for layer output
  • TLU is hard limiter !tlu.png
  • o_1 to o_4 must all be one to overcome -3.5 bias and force output to 1 !mlp-non-linear-decision.png
  • Can generate a non-linear Decision Boundary