diff --git a/AI/Classification/Gradient Boosting Machine.md b/AI/Classification/Gradient Boosting Machine.md index ba03de5..3479779 100644 --- a/AI/Classification/Gradient Boosting Machine.md +++ b/AI/Classification/Gradient Boosting Machine.md @@ -4,4 +4,5 @@ - Strictly outperform random forest most of the time - Similar properties - One of the best algorithm for dealing with non perceptual data -- XGBoost \ No newline at end of file +- XGBoost + - [Confusing XGBoost Hyperparameters](https://towardsdatascience.com/10-confusing-xgboost-hyperparameters-and-how-to-tune-them-like-a-pro-in-2023-e305057f546?source=rss----7f60cf5620c9---4) \ No newline at end of file diff --git a/AI/Neural Networks/CV/Visual Search/Visual Search.md b/AI/Neural Networks/CV/Visual Search/Visual Search.md new file mode 100644 index 0000000..8b5cadd --- /dev/null +++ b/AI/Neural Networks/CV/Visual Search/Visual Search.md @@ -0,0 +1,10 @@ +- Shallow would be BOVW +- Use metric space over feature space + - Get ranked list + +![](../../../../img/visual-search-arch.png) +# Crude Method +- Doesn't enforce metric relationship +- Sample prior to final softmax +![](../../../../img/visual-search-crude.png) + diff --git a/AI/Neural Networks/Learning/Tasks.md b/AI/Neural Networks/Learning/Tasks.md new file mode 100644 index 0000000..e2011c4 --- /dev/null +++ b/AI/Neural Networks/Learning/Tasks.md @@ -0,0 +1,44 @@ +# Pattern Association +- Associative memory + - Learns by association +- Autoassociation + - Store a set of patterns by repeatedly presenting them in the network + - Then presented partial or distorted stored pattern + - Recall intended + - Input and output data spaces are same dimensionality +- Heteroassociation + - Arbitrary set of input patterns paired with another arbitrary set of output patterns + - Supervised instead of unsupervised + - No required relationship between input/output dimensionality +- Stages + - Storage + - Recall + +# Pattern Recognition +- Received pattern/signal is assigned to one of a prescribed number of classes +![](../../../img/nn-tasks-pattern.png) + +# Function Approximation +- System Identification +![](../../../img/nn-tasks-function-approx.png) +- Inverse System +![](../../../img/nn-tasks-function-approx-inverse.png) + +# Control +- Learn to control a process or critical part of a system + +# Filtering +- Filtering + - Extraction of information about a quantity of interest at discrete time $n$ by using data from time up to $n$ +- Smoothing + - Use information past time $n$ + - Expect smoother result + - Delay in processing +- Prediction + - Predict later data using current and previous + +# Beamforming +- Spatial filtering +- Distinguish spatial properties of a target signal and background noise + - Similar to bats + - Used in radar and sonar \ No newline at end of file diff --git a/AI/Pattern Matching/Dynamic Time Warping.md b/AI/Pattern Matching/Dynamic Time Warping.md index e69de29..da9345c 100644 --- a/AI/Pattern Matching/Dynamic Time Warping.md +++ b/AI/Pattern Matching/Dynamic Time Warping.md @@ -0,0 +1,73 @@ +***Deterministic +Pattern Recogniser*** +Allows timescale variations in sequences for same class + +![](../../img/dtw-graph.png) + +$$D(T,N)=\min_{t,i}\sum_{\substack{t\in1..T \\ i\in1..N}}d(t,i)$$ +- $d(t,i)$ is distance between features from $t$-th frame of test to $i$-th frame of template + +$$D(t,i)=\min[D(t,i-1),D(t-1, i-1),D(t-1,i)]+d(t,i)$$ +- Allowing transition from current and previous frame only +- Recursive + +![](../../img/dtw-graph-unit.png) + +# Problems +- How much flexibility to allow? +- How to penalise warping? +- How to determine a fair distance metric? +- How many templates to register? + - How to select best ones? + +# Basic Algorithm +1. Initialise the cumulative distances for $t=1$ +$$D(1,i)=\begin{cases}d(1,i) & \text{for }i=1, \\ D(1, i-1)+d(1,i) & \text{for }i=2,...,N\end{cases}$$ +2. Recur for $t=2,...,T$ +$$D(t,i)=\begin{cases}D(t-1,i) + d(t,i) & \text{for }i=1, \\ \min[D(t, i-1), D(t-1, i-1),D(t-1,i)] + d(t,i) & \text{for }i=2,...,N\end{cases}$$ +3. Finalise, the cumulative distance up to the final point gives the total cost of the match: $D(T,N)$ + +![](../../img/dtw-heatmap.png) +- Euclidean distances + +# Distortion Penalty +1. Initialise the cumulative distances for $t=1$ +$$D(1,i)=\begin{cases}d(1,i) & \text{for }i=1, \\ d(1,i)+D(1, i-1)+d_V & \text{for }i=2,...,N\end{cases}$$ +2. Recur for $t=2,...,T$ +$$D(t,i)=\begin{cases}d(t,i)+D(t-1,i1)+d_H & \text{for }i=1, \\ \min[d(t,i)+D(t,i-1)+d_V,2d(t,i)+D(t-1,i-1),d(t,i)+D(t-1,i)+d_H] & \text{for }i=2,...,N\end{cases}$$ +- Where $d_V$ and $d_H$ are costs associated with vertical and horizontal transitions respectively +3. Finalise, the cumulative distance up to the final point gives the total cost of the match: $D(T,N)$ + +- Allows weighting for dynamic penalties when moving horizontally or vertically + - As opposed to diagonally + +![](../../img/dtw-heatmap-distortion.png) + +# Store Best Path +1. Initialise distances and traceback indicator for $t=1$ +$$D(1,i)=\begin{cases}d(1,i) & \text{for } i=1,\\ d(1,i)+D(1,i-1) & \text{for }i = 2,...,N\end{cases}$$ +$$\phi(1,i)=\begin{cases}[0,0] & \text{for } i=1,\\ [1,i-1] & \text{for }i = 2,...,N\end{cases}$$ +2. Recur for cumulative distances at $t=2,...,T$ +$$D(1,i)=\begin{cases}d(t,i)+D(t-1,i) & \text{for } i=1,\\ d(t,i)+\min[D(t,i-1),D(t-1,i-1),D(t-1,i)] & \text{for }i = 2,...,N\end{cases}$$ +$$\phi(1,i)=\begin{cases}[t-1,i] & \text{for } i=1,\\ \arg\min[D(t,i-1),D(t-1,i-1),D(t-1,i)] & \text{for }i = 2,...,N\end{cases}$$ +3. Final point gives the total alignment cost D(T,N) and the end coordinates of the best path $z_K=[T,N]$, where $K$ is the number of nodes on the optimal path +4. Trace the path back for $k=K-1,...,1,z_k=\phi(z_{k+1}), \text{ and }Z=\{z_1,...,z_K\}$ +- Stores best path + +![](../../img/dtw-possible-movements.png) +- Vary allowable movements through grid +- Second row for blocking multiple of the same movements in succession + +# Search Pruning +- Speed up algorithm for real-time +- Kill bad options + +## Gross Partitioning +![](../../img/dtw-gross-partitioning.png) +- Too far from diagonal +- Probably wrong or bad + +## Score Pruning +![](../../img/dtw-score-pruning.png) +- Examine existing branches +- See which scores are really bad \ No newline at end of file diff --git a/AI/Problem Solving.md b/AI/Problem Solving.md new file mode 100644 index 0000000..2c749bc --- /dev/null +++ b/AI/Problem Solving.md @@ -0,0 +1,57 @@ +# Problem Types +- Toy/game problems + - Illustrative +- Real world problems + - Exact solutions + +# Intelligent Agents +*Any thing which can be viewed as perceiving its environment through sensors and acting upon that environment through its affectors* + +## Reflex +![](img/problem-solving-reflex.png) +## Goal Based +![](img/problem-solving-goal-based.png) +## Utility-based + +# Environments +- (in)accessible +- (non)-deterministic +- (non)-episodic +- Static/dynamic +- Discrete/continuous + +# Solving +1. Precise problem definition + - Start conditions and goal state +2. Problem analysis + - Appropriate representation + - Problem characteristics + - Knowledge intensive + - Decomposability +3. Choice of best technique + +# Moves +- Ignorable + - Simple control structure +- Recoverable + - Backtrack using control stack +- Irrecoverable + - Much effort needed for each decision/move + +![](img/problem-solving-arch.png) +- Control + - Cause movement + - Be systematic + - Be guided by heuristics + +# Predictability +## 8 Puzzle +- Can plan ahead with certainty — only need a +control structure with backtracking + - recoverable, certain outcome +## Chess +- Future moves are not predictable with certainty. +- May need to consider several possibilities, and decide best option based on Incomplete information +- irrecoverable — once a piece is taken cannot in general recover from this state. +- deciding on an optimal move can be difficult +- try to predict opponent's move \ No newline at end of file diff --git a/CS/Regex.md b/CS/Regex.md new file mode 100644 index 0000000..b9fee48 --- /dev/null +++ b/CS/Regex.md @@ -0,0 +1,14 @@ +# Markdown + +## All links +`\[.*?\]\(.*?\)` + +## Images +`!\[.*?\]\(.*?\.png\)` + +## Inter-Ref +`\[.*?\]\(.*?\.md(#.*?)*\)` + +## Tags +`#{1}[^\s#.]+` +`(^|[[:blank:]])#{1}[^\s#.]+` \ No newline at end of file diff --git a/img/dtw-graph-unit.png b/img/dtw-graph-unit.png new file mode 100644 index 0000000..0f3c132 Binary files /dev/null and b/img/dtw-graph-unit.png differ diff --git a/img/dtw-graph.png b/img/dtw-graph.png new file mode 100644 index 0000000..823dd19 Binary files /dev/null and b/img/dtw-graph.png differ diff --git a/img/dtw-gross-partitioning.png b/img/dtw-gross-partitioning.png new file mode 100644 index 0000000..9a299eb Binary files /dev/null and b/img/dtw-gross-partitioning.png differ diff --git a/img/dtw-heatmap-distortion.png b/img/dtw-heatmap-distortion.png new file mode 100644 index 0000000..96e83cb Binary files /dev/null and b/img/dtw-heatmap-distortion.png differ diff --git a/img/dtw-heatmap.png b/img/dtw-heatmap.png new file mode 100644 index 0000000..5ad83d5 Binary files /dev/null and b/img/dtw-heatmap.png differ diff --git a/img/dtw-possible-movements.png b/img/dtw-possible-movements.png new file mode 100644 index 0000000..5b7fa67 Binary files /dev/null and b/img/dtw-possible-movements.png differ diff --git a/img/dtw-score-pruning.png b/img/dtw-score-pruning.png new file mode 100644 index 0000000..824c280 Binary files /dev/null and b/img/dtw-score-pruning.png differ diff --git a/img/nn-tasks-function-approx-inverse.png b/img/nn-tasks-function-approx-inverse.png new file mode 100644 index 0000000..7ff7856 Binary files /dev/null and b/img/nn-tasks-function-approx-inverse.png differ diff --git a/img/nn-tasks-function-approx.png b/img/nn-tasks-function-approx.png new file mode 100644 index 0000000..41cb83a Binary files /dev/null and b/img/nn-tasks-function-approx.png differ diff --git a/img/nn-tasks-pattern.png b/img/nn-tasks-pattern.png new file mode 100644 index 0000000..de711e5 Binary files /dev/null and b/img/nn-tasks-pattern.png differ diff --git a/img/problem-solving-arch.png b/img/problem-solving-arch.png new file mode 100644 index 0000000..3780e1c Binary files /dev/null and b/img/problem-solving-arch.png differ diff --git a/img/problem-solving-goal-based.png b/img/problem-solving-goal-based.png new file mode 100644 index 0000000..677070e Binary files /dev/null and b/img/problem-solving-goal-based.png differ diff --git a/img/problem-solving-reflex.png b/img/problem-solving-reflex.png new file mode 100644 index 0000000..66fe332 Binary files /dev/null and b/img/problem-solving-reflex.png differ diff --git a/img/visual-search-arch.png b/img/visual-search-arch.png new file mode 100644 index 0000000..417f6e3 Binary files /dev/null and b/img/visual-search-arch.png differ diff --git a/img/visual-search-crude.png b/img/visual-search-crude.png new file mode 100644 index 0000000..c7a8180 Binary files /dev/null and b/img/visual-search-crude.png differ