andy
1513f2b378
Affected files: STEM/AI/Classification/Classification.md STEM/AI/Classification/Decision Trees.md STEM/AI/Classification/Gradient Boosting Machine.md STEM/AI/Classification/Logistic Regression.md STEM/AI/Classification/Random Forest.md STEM/AI/Classification/Supervised.md STEM/AI/Classification/Supervised/README.md STEM/AI/Classification/Supervised/SVM.md STEM/AI/Classification/Supervised/Supervised.md STEM/AI/Learning.md STEM/AI/Neural Networks/Learning/Boltzmann.md STEM/AI/Neural Networks/Learning/Competitive Learning.md STEM/AI/Neural Networks/Learning/Credit-Assignment Problem.md STEM/AI/Neural Networks/Learning/Hebbian.md STEM/AI/Neural Networks/Learning/Learning.md STEM/AI/Neural Networks/Learning/README.md STEM/AI/Neural Networks/RNN/Autoencoder.md STEM/AI/Neural Networks/RNN/Deep Image Prior.md STEM/AI/Neural Networks/RNN/MoCo.md STEM/AI/Neural Networks/RNN/Representation Learning.md STEM/AI/Neural Networks/RNN/SimCLR.md STEM/img/comp-learning.png STEM/img/competitive-geometric.png STEM/img/confusion-matrix.png STEM/img/decision-tree.png STEM/img/deep-image-prior-arch.png STEM/img/deep-image-prior-results.png STEM/img/hebb-learning.png STEM/img/moco.png STEM/img/receiver-operator-curve.png STEM/img/reinforcement-learning.png STEM/img/rnn+autoencoder-variational.png STEM/img/rnn+autoencoder.png STEM/img/simclr.png STEM/img/sup-representation-learning.png STEM/img/svm-c.png STEM/img/svm-non-linear-project.png STEM/img/svm-non-linear-separated.png STEM/img/svm-non-linear.png STEM/img/svm-optimal-plane.png STEM/img/svm.png STEM/img/unsup-representation-learning.png
30 lines
913 B
Markdown
30 lines
913 B
Markdown
- Stochastic
|
||
- Recurrent structure
|
||
- Binary operation (+/- 1)
|
||
- Energy function
|
||
|
||
$$E=-\frac 1 2 \sum_j\sum_k w_{kj}x_kx_j$$
|
||
- $j\neq k$
|
||
- No self-feedback
|
||
- $x$ = neuron state
|
||
- Neurons randomly flip from $x$ to $-x$
|
||
|
||
$$P(x_k \rightarrow-x_k)=\frac 1 {1+e^{\frac{-\Delta E_k}{T}}}$$
|
||
|
||
- Energy change based on pseudo-temperature
|
||
- System will reach thermal equilibrium
|
||
- Delta E is the energy change resulting from the flip
|
||
- Visible and hidden neurons
|
||
- Visible act as interface between network and environment
|
||
- Hidden always operate freely
|
||
|
||
# Operation Modes
|
||
- Clamped
|
||
- Visible neurons are clamped onto specific states determined by environment
|
||
- Free-running
|
||
- All neurons able to operate freely
|
||
- $\rho_{kj}^+$ = Correlation between states while clamped
|
||
- $\rho_{kj}^-$ = Correlation between states while free
|
||
- Both exist between +/- 1
|
||
|
||
$$\Delta w_{kj}=\eta(\rho_{kj}^+-\rho_{kj}^-), \space j\neq k$$ |