vault backup: 2023-05-26 18:52:08

Affected files:
.obsidian/graph.json
.obsidian/workspace.json
STEM/AI/Neural Networks/CNN/FCN/FCN.md
STEM/AI/Neural Networks/CNN/FCN/FlowNet.md
STEM/AI/Neural Networks/CNN/FCN/Super-Resolution.md
STEM/AI/Neural Networks/CNN/GAN/CycleGAN.md
STEM/AI/Neural Networks/CNN/GAN/DC-GAN.md
STEM/AI/Neural Networks/CNN/GAN/GAN.md
STEM/AI/Neural Networks/CNN/GAN/cGAN.md
STEM/AI/Neural Networks/CNN/Interpretation.md
STEM/AI/Neural Networks/CNN/UpConv.md
STEM/img/am-process.png
STEM/img/am.png
STEM/img/fcn-arch.png
STEM/img/fcn-eval.png
STEM/img/fcn-uses.png
STEM/img/flownet-encode.png
STEM/img/flownet-training.png
STEM/img/flownet-upconv.png
STEM/img/flownet.png
STEM/img/super-res.png
STEM/img/superres-results.png

AI/Neural Networks/CNN/FCN/FCN.md

@@ -0,0 +1,42 @@
+Fully [[Convolution]]al Network
+
+Convolutional and up-convolutional layers with [[Activation Functions#ReLu|ReLu]] but no others (pooling)
+- All some sort of Encoder-Decoder
+
+Contractive → [[UpConv]]
+
+# Image Sementation
+- For visual output
+	- Previously image $\rightarrow$ vector
+- Additional layers to up-sample representation to an image
+	- Up-[[convolution]]al
+	- De-[[convolution]]al
+
+![[fcn-uses.png]]
+
+![[fcn-arch.png]]
+
+# Training
+- Rarely from scratch
+- Pre-trained weights
+- Replace final layers
+	- FC layers
+	- White-noise initialised
+- Add [[upconv]] layer(s)
+	- Fine-tune train
+	- Freeze others
+	- Annotated GT images
+- Can use summed per-pixel log loss
+
+# Evaluation
+![[fcn-eval.png]]
+- SDS
+	- Classical method
+	- 52% mAP
+- FCN
+	- 62% mAP
+- Intersection over Union
+	- IOU
+	- Jaccard
+	- Averaged over all images
+	- $J(A,B)=\frac{|A\cap B|}{|A\cup B|}$

AI/Neural Networks/CNN/FCN/FlowNet.md

@@ -0,0 +1,26 @@
+Optical Flow
+
+- 2-Channel optical flow
+	- $dx,dy$
+- Two consecutive frames
+	- 6-channel tensor
+
+![[flownet.png]]
+
+# Skip Connections
+- Further through the network information is condensed
+	- Less high frequency information
+- Link encoder layers to [[upconv]] layers
+	- Append activation maps from encoder to decoder
+
+# Encode
+![[flownet-encode.png]]
+
+# [[Upconv]]
+![[flownet-upconv.png]]
+
+# Training
+- Synthetic rendered objects
+- Real background images
+
+![[flownet-training.png]]

AI/Neural Networks/CNN/FCN/Super-Resolution.md

@@ -0,0 +1,12 @@
+- Auto-encoders
+	- Get same image back
+- Up-sample blurry small image classically
+	- Bi-cubic
+	- Encode-decode to deep sharpen
+- No ground truth
+	- Unsupervised?
+- Decoder stage
+	- Identical architecture to encoder
+![[super-res.png]]
+- Is actually contractive/up sampling
+![[superres-results.png]]

AI/Neural Networks/CNN/GAN/CycleGAN.md

@@ -1,4 +1,4 @@
-Cycle Consistent GAN
+Cycle Consistent [[GAN]]
 
 - G
 	- $x \rightarrow y$

AI/Neural Networks/CNN/GAN/DC-GAN.md

@@ -1,14 +1,14 @@
-Deep Convolutional GAN
+Deep Convolutional [[GAN]]
 ![[dc-gan.png]]
 
 - Generator
-	- FCN
+	- [[FCN]]
 	- Decoder
 	- Generate image from code
 		- Low-dimensional
 			- ~100-D
 	- Reshape to tensor
-		- Upconv to image
+		- [[Upconv]] to image
 	- Train using Gaussian random noise for code
 - Discriminator
 	- Contractive
@@ -57,12 +57,12 @@ $$J^{(G)}=-J^{(D)}$$
 
 # What is Learnt?
 - Encoding texture/patch detail from training set
-	- Similar to FCN
+	- Similar to [[FCN]]
 	- Reproducing texture at high level
 	- Cues triggered by code vector
 		- Input random noise
 - Iteratively improves visual feasibility
-	- Different to FCN
+	- Different to [[FCN]]
 - Discriminator is a task specific classifier
 - Difficult to train over diverse footage
 	- Mixing concepts doesn't work

AI/Neural Networks/CNN/GAN/GAN.md

@@ -1,12 +1,12 @@
 # Fully Convolutional
-- Remove max-pooling
+- Remove [[Max Pooling]]
-	- Use strided upconv
+	- Use strided [[upconv]]
 - Remove FC layers
 	- Hurts convergence in non-classification
 - Normalisation tricks
 	- Batch normalisation
 		- Batches of 0 mean and variance 1
-	- Leaky ReLu
+	- Leaky [[Activation Functions#ReLu|ReLu]]
 
 # Stages
 ## Generator, G

AI/Neural Networks/CNN/GAN/cGAN.md

@@ -1,7 +1,7 @@
-Conditional GAN
+Conditional [[GAN]]
 
-- Hard to control with AM
+- Hard to control with [[Interpretation#Activation Maximisation|AM]]
-	- Unconditional GAN
+	- Unconditional [[GAN]]
 - Condition synthesis on a class label
 - Concatenate unconditional code with conditioning vector
 	- Label

AI/Neural Networks/CNN/Interpretation.md

@@ -0,0 +1,20 @@
+# Activation Maximisation
+- Synthesise an ideal image for a class
+	- Maximise 1-hot output
+	- Maximise [[Activation Functions#SoftMax|SoftMax]]
+
+![[am.png]]
+- **Use trained network**
+	- Don't update weights
+- Feedforward noise
+	- [[Back-Propagation|Back-propagate]] loss
+		- Don't update weights
+		- Update image
+
+![[am-process.png]]
+## Regulariser
+- Fit to natural image statistics
+- Prone to high frequency noise
+	- Minimise
+- Total variation
+	- $x^*$ is the best solution to minimise loss