From 4cc2e798669bba0bb5b76ce50eb89d7759fc30db Mon Sep 17 00:00:00 2001 From: andy Date: Wed, 31 May 2023 22:21:56 +0100 Subject: [PATCH] vault backup: 2023-05-31 22:21:56 Affected files: .obsidian/global-search.json .obsidian/workspace.json Health/Alexithymia.md Health/BWS.md STEM/AI/Neural Networks/Activation Functions.md STEM/AI/Neural Networks/Architectures.md STEM/AI/Neural Networks/CNN/CNN.md STEM/AI/Neural Networks/MLP/Back-Propagation.md STEM/AI/Neural Networks/Transformers/Attention.md STEM/CS/Calling Conventions.md STEM/CS/Languages/Assembly.md --- AI/Neural Networks/Activation Functions.md | 7 ++++--- AI/Neural Networks/Architectures.md | 8 ++++---- AI/Neural Networks/CNN/CNN.md | 4 ++-- AI/Neural Networks/MLP/Back-Propagation.md | 2 +- AI/Neural Networks/Transformers/Attention.md | 10 +++++----- CS/Calling Conventions.md | 8 ++++---- CS/Languages/Assembly.md | 4 ++-- 7 files changed, 22 insertions(+), 21 deletions(-) diff --git a/AI/Neural Networks/Activation Functions.md b/AI/Neural Networks/Activation Functions.md index 5ffe7aa..fa510ff 100644 --- a/AI/Neural Networks/Activation Functions.md +++ b/AI/Neural Networks/Activation Functions.md @@ -11,7 +11,7 @@ - Bipolar - -1 <-> +1 -![[threshold-activation.png]] +![threshold-activation](../../img/threshold-activation.png) # Sigmoid - Logistic function @@ -26,7 +26,8 @@ $$\frac d {dx} \sigma(x)= \right] =\sigma(x)\cdot(1-\sigma(x))$$ -![[sigmoid.png]] +![sigmoid](../../img/sigmoid.png) + ### Derivative $$y_j(n)=\varphi_j(v_j(n))= @@ -58,7 +59,7 @@ Rectilinear - Sometimes small scalar for negative - Leaky ReLu -![[relu.png]] +![relu](../../img/relu.png) # SoftMax - Output is per-class vector of likelihoods diff --git a/AI/Neural Networks/Architectures.md b/AI/Neural Networks/Architectures.md index 8703826..6f2417a 100644 --- a/AI/Neural Networks/Architectures.md +++ b/AI/Neural Networks/Architectures.md @@ -2,7 +2,7 @@ - *Acyclic* - Count output layer, no computation at input -![[feedforward.png]] +![feedforward](../../img/feedforward.png) # Multilayer Feedforward - Hidden layers @@ -12,12 +12,12 @@ - Fully connected - Every neuron is connected to every neuron adjacent layers - Below is a 10-4-2 network -![[multilayerfeedforward.png]] +![multilayerfeedforward](../../img/multilayerfeedforward.png) # Recurrent - At least one feedback loop - Below has no self-feedback -![[recurrent.png]] -![[recurrentwithhn.png]] +![recurrent](../../img/recurrent.png) +![recurrentwithhn](../../img/recurrentwithhn.png) - Above has hidden neurons \ No newline at end of file diff --git a/AI/Neural Networks/CNN/CNN.md b/AI/Neural Networks/CNN/CNN.md index 05ecc1c..3b0af13 100644 --- a/AI/Neural Networks/CNN/CNN.md +++ b/AI/Neural Networks/CNN/CNN.md @@ -5,13 +5,13 @@ - Niche - No-one cared/knew about CNNs ## After -- [[Datasets#ImageNet|ImageNet]] +- [ImageNet](../CV/Datasets.md#ImageNet) - 16m images, 1000 classes - GPUs - General processing GPUs - CUDA - NIPS/ECCV 2012 - - Double digit % gain on [[Datasets#ImageNet|ImageNet]] accuracy + - Double digit % gain on [ImageNet](../CV/Datasets.md#ImageNet) accuracy # Full Connected [[MLP|Dense]] diff --git a/AI/Neural Networks/MLP/Back-Propagation.md b/AI/Neural Networks/MLP/Back-Propagation.md index 5d3b833..b283877 100644 --- a/AI/Neural Networks/MLP/Back-Propagation.md +++ b/AI/Neural Networks/MLP/Back-Propagation.md @@ -79,7 +79,7 @@ $$\Delta w_{ji}(n)=\eta\cdot\delta_j(n)\cdot y_i(n)$$ 2. Error WRT output $y$ 3. Output $y$ WRT Pre-activation function sum 4. Pre-activation function sum WRT weight - - Other [[Weight Init|weights]] constant, goes to zero + - Other [weights](../Weight%20Init.md) constant, goes to zero - Leaves just $y_i$ - Collect 3 boxed terms as delta $j$ - Local gradient diff --git a/AI/Neural Networks/Transformers/Attention.md b/AI/Neural Networks/Transformers/Attention.md index 851c730..31893ea 100644 --- a/AI/Neural Networks/Transformers/Attention.md +++ b/AI/Neural Networks/Transformers/Attention.md @@ -10,16 +10,16 @@ - [LSTM](../RNN/LSTM.md) tends to poorly preserve far back [knowledge](../Neural%20Networks.md#Knowledge) - Attention layer access all previous states and weighs according to learned measure of relevance - Allows referring arbitrarily far back to relevant tokens -- Can be addd to [[RNN]]s -- In 2016, a new type of highly parallelisable _decomposable attention_ was successfully combined with a [[Architectures|feedforward]] network - - Attention useful in of itself, not just with [[RNN]]s -- [[Transformers]] use attention without recurrent connections +- Can be addd to [RNNs](../RNN/RNN.md) +- In 2016, a new type of highly parallelisable _decomposable attention_ was successfully combined with a [feedforward](../Architectures.md) network + - Attention useful in of itself, not just with [RNNs](../RNN/RNN.md) +- [Transformers](Transformers.md) use attention without recurrent connections - Process all tokens simultaneously - Calculate attention weights in successive layers # Scaled Dot-Product - Calculate attention weights between all tokens at once -- Learn 3 [[Weight Init|weight]] matrices +- Learn 3 [weight](../Weight%20Init.md) matrices - Query - $W_Q$ - Key diff --git a/CS/Calling Conventions.md b/CS/Calling Conventions.md index cf1cad8..994bfe9 100644 --- a/CS/Calling Conventions.md +++ b/CS/Calling Conventions.md @@ -5,15 +5,15 @@ - Also known as: callee-saved registers or non-volatile registers - How the task of preparing the stack for, and restoring after, a function call is divided between the caller and the callee -Subtle differences between [[compilers]], can be difficult to interface codes from different [[compilers]] +Subtle differences between [Compilers](Compilers.md), can be difficult to interface codes from different [compilers](Compilers.md) -Calling conventions, type representations, and name mangling are all part of what is known as an [application binary interface](https://en.wikipedia.org/wiki/Application_binary_interface) ([[ABI]]) +Calling conventions, type representations, and name mangling are all part of what is known as an [application binary interface](https://en.wikipedia.org/wiki/Application_binary_interface) ([ABI](ABI.md)) # cdecl C declaration -- Originally from Microsoft's C [[compilers|compiler]] - - Used by many C [[compilers]] for x86 +- Originally from Microsoft's C [compiler](Compilers.md) + - Used by many C [compilers](Compilers.md) for x86 - Subroutine arguments passed on the stack - Function arguments pushed right-to-left - Last pushed first diff --git a/CS/Languages/Assembly.md b/CS/Languages/Assembly.md index 5a6019d..55380c0 100644 --- a/CS/Languages/Assembly.md +++ b/CS/Languages/Assembly.md @@ -1,11 +1,11 @@ [Uni of Virginia - x86 Assembly Guide](https://www.cs.virginia.edu/~evans/cs216/guides/x86.html) ## x86 32-bit -![[x86registers.png]] +![x86registers](../../img/x86registers.png) ## Stack - push, pop, call, ret -![[stack.png]] +![stack](../../img/stack.png) - Growing upwards