diff --git a/AI/Neural Networks/CNN/FCN/FCN.md b/AI/Neural Networks/CNN/FCN/FCN.md new file mode 100644 index 0000000..797cf4f --- /dev/null +++ b/AI/Neural Networks/CNN/FCN/FCN.md @@ -0,0 +1,42 @@ +Fully [[Convolution]]al Network + +Convolutional and up-convolutional layers with [[Activation Functions#ReLu|ReLu]] but no others (pooling) +- All some sort of Encoder-Decoder + +Contractive → [[UpConv]] + +# Image Sementation +- For visual output + - Previously image $\rightarrow$ vector +- Additional layers to up-sample representation to an image + - Up-[[convolution]]al + - De-[[convolution]]al + +![[fcn-uses.png]] + +![[fcn-arch.png]] + +# Training +- Rarely from scratch +- Pre-trained weights +- Replace final layers + - FC layers + - White-noise initialised +- Add [[upconv]] layer(s) + - Fine-tune train + - Freeze others + - Annotated GT images +- Can use summed per-pixel log loss + +# Evaluation +![[fcn-eval.png]] +- SDS + - Classical method + - 52% mAP +- FCN + - 62% mAP +- Intersection over Union + - IOU + - Jaccard + - Averaged over all images + - $J(A,B)=\frac{|A\cap B|}{|A\cup B|}$ \ No newline at end of file diff --git a/AI/Neural Networks/CNN/FCN/FlowNet.md b/AI/Neural Networks/CNN/FCN/FlowNet.md new file mode 100644 index 0000000..b9ef983 --- /dev/null +++ b/AI/Neural Networks/CNN/FCN/FlowNet.md @@ -0,0 +1,26 @@ +Optical Flow + +- 2-Channel optical flow + - $dx,dy$ +- Two consecutive frames + - 6-channel tensor + +![[flownet.png]] + +# Skip Connections +- Further through the network information is condensed + - Less high frequency information +- Link encoder layers to [[upconv]] layers + - Append activation maps from encoder to decoder + +# Encode +![[flownet-encode.png]] + +# [[Upconv]] +![[flownet-upconv.png]] + +# Training +- Synthetic rendered objects +- Real background images + +![[flownet-training.png]] \ No newline at end of file diff --git a/AI/Neural Networks/CNN/FCN/Super-Resolution.md b/AI/Neural Networks/CNN/FCN/Super-Resolution.md new file mode 100644 index 0000000..c624141 --- /dev/null +++ b/AI/Neural Networks/CNN/FCN/Super-Resolution.md @@ -0,0 +1,12 @@ +- Auto-encoders + - Get same image back +- Up-sample blurry small image classically + - Bi-cubic + - Encode-decode to deep sharpen +- No ground truth + - Unsupervised? +- Decoder stage + - Identical architecture to encoder +![[super-res.png]] +- Is actually contractive/up sampling +![[superres-results.png]] \ No newline at end of file diff --git a/AI/Neural Networks/CNN/GAN/CycleGAN.md b/AI/Neural Networks/CNN/GAN/CycleGAN.md index 1c8f3dc..2c69782 100644 --- a/AI/Neural Networks/CNN/GAN/CycleGAN.md +++ b/AI/Neural Networks/CNN/GAN/CycleGAN.md @@ -1,4 +1,4 @@ -Cycle Consistent GAN +Cycle Consistent [[GAN]] - G - $x \rightarrow y$ diff --git a/AI/Neural Networks/CNN/GAN/DC-GAN.md b/AI/Neural Networks/CNN/GAN/DC-GAN.md index f5a06d5..c0c4b07 100644 --- a/AI/Neural Networks/CNN/GAN/DC-GAN.md +++ b/AI/Neural Networks/CNN/GAN/DC-GAN.md @@ -1,14 +1,14 @@ -Deep Convolutional GAN +Deep Convolutional [[GAN]] ![[dc-gan.png]] - Generator - - FCN + - [[FCN]] - Decoder - Generate image from code - Low-dimensional - ~100-D - Reshape to tensor - - Upconv to image + - [[Upconv]] to image - Train using Gaussian random noise for code - Discriminator - Contractive @@ -57,12 +57,12 @@ $$J^{(G)}=-J^{(D)}$$ # What is Learnt? - Encoding texture/patch detail from training set - - Similar to FCN + - Similar to [[FCN]] - Reproducing texture at high level - Cues triggered by code vector - Input random noise - Iteratively improves visual feasibility - - Different to FCN + - Different to [[FCN]] - Discriminator is a task specific classifier - Difficult to train over diverse footage - Mixing concepts doesn't work diff --git a/AI/Neural Networks/CNN/GAN/GAN.md b/AI/Neural Networks/CNN/GAN/GAN.md index d16014a..ac77d59 100644 --- a/AI/Neural Networks/CNN/GAN/GAN.md +++ b/AI/Neural Networks/CNN/GAN/GAN.md @@ -1,12 +1,12 @@ # Fully Convolutional -- Remove max-pooling - - Use strided upconv +- Remove [[Max Pooling]] + - Use strided [[upconv]] - Remove FC layers - Hurts convergence in non-classification - Normalisation tricks - Batch normalisation - Batches of 0 mean and variance 1 - - Leaky ReLu + - Leaky [[Activation Functions#ReLu|ReLu]] # Stages ## Generator, G diff --git a/AI/Neural Networks/CNN/GAN/cGAN.md b/AI/Neural Networks/CNN/GAN/cGAN.md index 929cc63..06950b3 100644 --- a/AI/Neural Networks/CNN/GAN/cGAN.md +++ b/AI/Neural Networks/CNN/GAN/cGAN.md @@ -1,7 +1,7 @@ -Conditional GAN +Conditional [[GAN]] -- Hard to control with AM - - Unconditional GAN +- Hard to control with [[Interpretation#Activation Maximisation|AM]] + - Unconditional [[GAN]] - Condition synthesis on a class label - Concatenate unconditional code with conditioning vector - Label diff --git a/AI/Neural Networks/CNN/Interpretation.md b/AI/Neural Networks/CNN/Interpretation.md new file mode 100644 index 0000000..515dd34 --- /dev/null +++ b/AI/Neural Networks/CNN/Interpretation.md @@ -0,0 +1,20 @@ +# Activation Maximisation +- Synthesise an ideal image for a class + - Maximise 1-hot output + - Maximise [[Activation Functions#SoftMax|SoftMax]] + +![[am.png]] +- **Use trained network** + - Don't update weights +- Feedforward noise + - [[Back-Propagation|Back-propagate]] loss + - Don't update weights + - Update image + +![[am-process.png]] +## Regulariser +- Fit to natural image statistics +- Prone to high frequency noise + - Minimise +- Total variation + - $x^*$ is the best solution to minimise loss \ No newline at end of file diff --git a/AI/Neural Networks/CNN/UpConv.md b/AI/Neural Networks/CNN/UpConv.md new file mode 100644 index 0000000..e69de29 diff --git a/img/am-process.png b/img/am-process.png new file mode 100644 index 0000000..c5f1e80 Binary files /dev/null and b/img/am-process.png differ diff --git a/img/am.png b/img/am.png new file mode 100644 index 0000000..2a1b66c Binary files /dev/null and b/img/am.png differ diff --git a/img/fcn-arch.png b/img/fcn-arch.png new file mode 100644 index 0000000..feb9c4f Binary files /dev/null and b/img/fcn-arch.png differ diff --git a/img/fcn-eval.png b/img/fcn-eval.png new file mode 100644 index 0000000..8b1499f Binary files /dev/null and b/img/fcn-eval.png differ diff --git a/img/fcn-uses.png b/img/fcn-uses.png new file mode 100644 index 0000000..e0f9cf8 Binary files /dev/null and b/img/fcn-uses.png differ diff --git a/img/flownet-encode.png b/img/flownet-encode.png new file mode 100644 index 0000000..bb1be30 Binary files /dev/null and b/img/flownet-encode.png differ diff --git a/img/flownet-training.png b/img/flownet-training.png new file mode 100644 index 0000000..7393d56 Binary files /dev/null and b/img/flownet-training.png differ diff --git a/img/flownet-upconv.png b/img/flownet-upconv.png new file mode 100644 index 0000000..06e77bb Binary files /dev/null and b/img/flownet-upconv.png differ diff --git a/img/flownet.png b/img/flownet.png new file mode 100644 index 0000000..27a83c7 Binary files /dev/null and b/img/flownet.png differ diff --git a/img/super-res.png b/img/super-res.png new file mode 100644 index 0000000..9424fc3 Binary files /dev/null and b/img/super-res.png differ diff --git a/img/superres-results.png b/img/superres-results.png new file mode 100644 index 0000000..3ee403d Binary files /dev/null and b/img/superres-results.png differ