andy
d7ab8f329a
Affected files: Money/Assets/Financial Instruments.md Money/Assets/Security.md Money/Markets/Markets.md Politcs/Now.md STEM/AI/Neural Networks/CNN/Examples.md STEM/AI/Neural Networks/CNN/FCN/FCN.md STEM/AI/Neural Networks/CNN/FCN/FlowNet.md STEM/AI/Neural Networks/CNN/FCN/Highway Networks.md STEM/AI/Neural Networks/CNN/FCN/ResNet.md STEM/AI/Neural Networks/CNN/FCN/Skip Connections.md STEM/AI/Neural Networks/CNN/FCN/Super-Resolution.md STEM/AI/Neural Networks/CNN/GAN/DC-GAN.md STEM/AI/Neural Networks/CNN/GAN/GAN.md STEM/AI/Neural Networks/CNN/GAN/StackGAN.md STEM/AI/Neural Networks/CNN/Inception Layer.md STEM/AI/Neural Networks/CNN/Interpretation.md STEM/AI/Neural Networks/CNN/Max Pooling.md STEM/AI/Neural Networks/CNN/Normalisation.md STEM/AI/Neural Networks/CNN/UpConv.md STEM/AI/Neural Networks/CV/Layer Structure.md STEM/AI/Neural Networks/MLP/MLP.md STEM/AI/Neural Networks/Neural Networks.md STEM/AI/Neural Networks/RNN/LSTM.md STEM/AI/Neural Networks/RNN/RNN.md STEM/AI/Neural Networks/RNN/VQA.md STEM/AI/Neural Networks/SLP/Least Mean Square.md STEM/AI/Neural Networks/SLP/Perceptron Convergence.md STEM/AI/Neural Networks/SLP/SLP.md STEM/AI/Neural Networks/Transformers/LLM.md STEM/AI/Neural Networks/Transformers/Transformers.md STEM/AI/Properties.md STEM/CS/Language Binding.md STEM/Light.md STEM/Maths/Tensor.md STEM/Quantum/Orbitals.md STEM/Quantum/Schrödinger.md STEM/Quantum/Standard Model.md STEM/Quantum/Wave Function.md Tattoo/Music.md Tattoo/Plans.md Tattoo/Sources.md
42 lines
2.1 KiB
Markdown
42 lines
2.1 KiB
Markdown
Error-Correcting Perceptron Learning
|
|
|
|
- Uses a McCulloch-Pitt neuron
|
|
- One with a hard limiter
|
|
- Unity increment
|
|
- Learning rate of 1
|
|
|
|
If the $n$-th member of the training set, $x(n)$, is correctly classified by the weight vector $w(n)$ computed at the $n$-th iteration of the algorithm, no correction is made to the weight vector of the perceptron in accordance with the rule:
|
|
$$w(n + 1) = w(n) \text{ if $w^Tx(n) > 0$ and $x(n)$ belongs to class $\mathfrak{c}_1$}$$
|
|
$$w(n + 1) = w(n) \text{ if $w^Tx(n) \leq 0$ and $x(n)$ belongs to class $\mathfrak{c}_2$}$$
|
|
Otherwise, the weight vector of the perceptron is updated in accordance with the rule
|
|
$$w(n + 1) = w(n) - \eta(n)x(n) \text{ if } w^Tx(n) > 0 \text{ and } x(n) \text{ belongs to class }\mathfrak{c}_2$$
|
|
$$w(n + 1) = w(n) + \eta(n)x(n) \text{ if } w^Tx(n) \leq 0 \text{ and } x(n) \text{ belongs to class }\mathfrak{c}_1$$
|
|
|
|
1. _Initialisation_. Set $w(0)=0$. perform the following computations for
|
|
time step $n = 1, 2,...$
|
|
2. _Activation_. At time step $n$, activate the perceptron by applying continuous-valued input vector $x(n)$ and desired response $d(n)$.
|
|
3. _Computation of Actual Response_. Compute the actual response of the perceptron:
|
|
$$y(n) = sgn[w^T(n)x(n)]$$
|
|
where $sgn(\cdot)$ is the signum function.
|
|
4. _Adaptation of Weight Vector_. Update the weight vector of the perceptron:
|
|
$$w(n+1)=w(n)+\eta[d(n)-y(n)]x(n)$$ where
|
|
$$
|
|
d(n) = \begin{cases}
|
|
+1 &\text{if $x(n)$ belongs to class $\mathfrak{c_1}$}\\
|
|
-1 &\text{if $x(n)$ belongs to class $\mathfrak{c_2}$}
|
|
\end{cases}
|
|
$$
|
|
5. _Continuation_. Increment time step $n$ by one and go back to step 2.
|
|
|
|
- Guarantees convergence provided
|
|
- Patterns are linearly separable
|
|
- Non-overlapping classes
|
|
- Linear separation boundary
|
|
- Learning rate not too high
|
|
- Two conflicting requirements
|
|
1. Averaging of past inputs to provide stable weight estimates
|
|
- Small eta
|
|
2. Fast adaptation with respect to real changes in the underlying distribution of process responsible for $x$
|
|
- Large eta
|
|
|
|
 |