stem/AI/Neural Networks/MLP
2023-06-04 22:34:31 +01:00
..
Back-Propagation.md Revert "vault backup: 2023-06-04 22:31:53" 2023-06-04 22:34:31 +01:00
Decision Boundary.md vault backup: 2023-06-01 08:11:37 2023-06-01 08:11:37 +01:00
MLP.md vault backup: 2023-05-27 22:17:56 2023-05-27 22:17:56 +01:00
README.md vault backup: 2023-05-31 21:29:04 2023-05-31 21:29:04 +01:00

  • Architectures
  • Single hidden layer can learn any function
    • Universal approximation theorem
  • Each hidden layer can operate as a different feature extraction layer
  • Lots of Weight Init to learn
  • Back-Propagation is supervised

!mlp-arch.png

Universal Approximation Theory

A finite Architectures MLP with 1 hidden layer can in theory approximate any mathematical function

!activation-function.png !mlp-arch-diagram.png

Weight Matrix

  • Use matrix multiplication for layer output
  • TLU is hard limiter !tlu.png
  • o_1 to o_4 must all be one to overcome -3.5 bias and force output to 1 !mlp-non-linear-decision.png
  • Can generate a non-linear Decision Boundary