diff --git a/AI/Literature.md b/AI/Literature.md index bf998f9..a3b98f2 100644 --- a/AI/Literature.md +++ b/AI/Literature.md @@ -1,3 +1,4 @@ +#lit [https://web.stanford.edu/~jurafsky/slp3/A.pdf](https://web.stanford.edu/~jurafsky/slp3/A.pdf) [Towards Data Science: 3 Things You Need To Know Before You Train-Test Split](https://towardsdatascience.com/3-things-you-need-to-know-before-you-train-test-split-869dfabb7e50) [https://machinelearningmastery.com/train-final-machine-learning-model/](https://machinelearningmastery.com/train-final-machine-learning-model/) diff --git a/AI/Properties.md b/AI/Properties.md new file mode 100644 index 0000000..7692d25 --- /dev/null +++ b/AI/Properties.md @@ -0,0 +1,59 @@ +# Three Key Components + +1. Representation + - Declarative & Procedural knowledge + - Typically human-readable symbols +2. Reasoning + - Ability to solve problems + - Express and solve range of problems and types + - Make explicit and implicit information known to it + - Control mechanism to decide which operations to use if and when, when a solution has been found +3. Learning + +An AI system must be able to + +1. Store knowledge +2. Apply knowledge to solve problems +3. Acquire new knowledge through experience + +![[ai-nested-subjects.png]] + +# Expert Systems +- Usually easier to obtain compiled experience from experts than duplicate experience that made them experts for network + +# Information Processing +## Inductive +- General patterns and rules determined from data and experience +- Similarity-based learning + +## Deductive +- General rules are used to determine specific facts +- Proof of a theorem + +Explanation-based learning uses both + +# Classical AI vs Neural Nets +## Level of Explanation +- Classical has emphasis on building symbolic representations + - Models cognition as sequential processing of symbolic representations +- Neural nets emphasis on parallel distributed processing models + - Models assume information processing takes place through interactions of large numbers of neurons + +## Processing style +- Classical processing is sequential + - Von Neumann Machine +- Neural nets use parallelism everywhere + - Source of flexibility + - Robust + +## Representational Structure +- Classical emphasises language of thought + - Symbolic representation has quasi-linguistic structure + - New symbols created from compositionality +- Neural nets have problem describing nature and structure of representation + +Symbolic AI is the formal manipulation of a language of algorithms and data representations in a top-down fashion + +Neural nets bottom-up + +![[ai-io.png]] \ No newline at end of file diff --git a/CS/ABI.md b/CS/ABI.md index 4ca0cd9..a3656d2 100644 --- a/CS/ABI.md +++ b/CS/ABI.md @@ -1,8 +1,8 @@ -- How data structures & computational routines are accessed in machine code +- How data structures & computational routines are accessed in machine code ([[Code Types]]) - Machine code therefore hardware-dependent - API defines this structure in source code - Adherence usually responsibility of - - Compiler + - [[Compilers]] - OS - Library author @@ -13,7 +13,7 @@ - Stack organisation - Memory access types - Size, layouts and alignments of basic data types -- ___Calling convention___ +- [[Calling Conventions]] - How function arguments are passed - Stack or register - Which registers for which function param diff --git a/CS/Code Types.md b/CS/Code Types.md index 5c11609..962727e 100644 --- a/CS/Code Types.md +++ b/CS/Code Types.md @@ -27,9 +27,9 @@ Portable Code - Compact numeric codes, constants and references - Encode compiler output following analysis and validation - Can be further compiled - - JIT + - [[Compilers#JIT]] - Typically passed to VM - - Java, Python + - Java, [[Python]] ## Object Code - Product of compiler diff --git a/CS/Compilers.md b/CS/Compilers.md index 60cccb4..7ea0740 100644 --- a/CS/Compilers.md +++ b/CS/Compilers.md @@ -12,7 +12,8 @@ Just-in-Time - Dynamic compilation - Adaptive optimization - Dynamic recompilation - - Microarchitecture-specific speedups + - Microarchitecture-specific speedups + - [[ISA]] ## AOT Ahead-of-Time diff --git a/CS/Language Binding.md b/CS/Language Binding.md index 0d6421c..3bda512 100644 --- a/CS/Language Binding.md +++ b/CS/Language Binding.md @@ -4,22 +4,25 @@ ## Runtime Environments ### Object Models -- COM - - Component Object Model - - MS only cross-language model -- CLI - - .NET Common Language Infrastructure -- Freedesktop.org D-Bus - - Open cross-platform-language model +- COM + - [[C++]] + - Component Object Model + - MS only cross-language model +- CLI + - [[dotNet]] + - .NET Common Language Infrastructure +- Freedesktop.org D-Bus + - Open cross-platform-language model ### Virtual Machines -- CLR - - .NET Common Language Runtime -- Mono - - CLI languages - - Cross-platform -- Adobe Flash Player - - Tamarin -- JVM -- LLVM -- Silverlight \ No newline at end of file +- CLR + - [[dotNet]] + - .NET Common Language Runtime +- Mono + - CLI languages + - Cross-platform +- Adobe Flash Player + - Tamarin +- JVM +- LLVM +- Silverlight \ No newline at end of file diff --git a/CS/Languages/dotNet.md b/CS/Languages/dotNet.md index b1b8e96..564d87a 100644 --- a/CS/Languages/dotNet.md +++ b/CS/Languages/dotNet.md @@ -10,6 +10,7 @@ - JIT managed code into machine instructions - Execution engine - VM + - [[Language Binding#Virtual Machines]] - Services - Memory management - Type safety @@ -27,4 +28,6 @@ # Assemblies - Compiled CLI code - Portable executable (PE) - - DLL, EXE \ No newline at end of file + - DLL, EXE + +![[cli-infrastructure.png]] \ No newline at end of file diff --git a/CS/Quantum.md b/CS/Quantum.md index 70d3580..95ad4b3 100644 --- a/CS/Quantum.md +++ b/CS/Quantum.md @@ -1 +1,2 @@ +#lit [5 books](https://fivebooks.com/best-books/quantum-computing-chris-bernhardt/) \ No newline at end of file diff --git a/CS/Resources.md b/CS/Resources.md index 168060b..fa14fe4 100644 --- a/CS/Resources.md +++ b/CS/Resources.md @@ -1 +1,2 @@ +#lit [Wigle - wifi enumerating](http://wigle.net) diff --git a/CS/Turing Machines.md b/CS/Turing Machines.md new file mode 100644 index 0000000..3f42f61 --- /dev/null +++ b/CS/Turing Machines.md @@ -0,0 +1,16 @@ +# David Hilbert +- Wondered if there was a universal algorithmic process to decide whether any mathematical proposition was true +- Then suggested that there were no unsolvable problems + +# Incompleteness Theorem +## Kurt Godel + +You might be able to prove every conceivable statement about numbers within a system by going outside the system in order to come up with new rules and axioms, but by doing so you'll only create a larger system with its own unprovable statements + +# Turing Machine +- Model of computation + - Resolves whether or not mathematics contained problems were incomputable + - No algorithmic solution + +### Church-Turing Thesis +Any algorithm capable of being devised can be run on a Turing machine \ No newline at end of file diff --git a/Maths/Algebra.md b/Maths/Algebra.md new file mode 100644 index 0000000..e902770 --- /dev/null +++ b/Maths/Algebra.md @@ -0,0 +1,18 @@ +# Field + +- Set on which addition and multiplication defined + - Behave same as on rational and real numbers + - Subtraction, division implied +- Examples + - Rational numbers + - Real numbers + - Complex numbers +- Any field can be used as scalars for a vector space +- A commutative ring where 0 =/= 1 and all nonzero elements are invertible + +## Vector Space +- Set of vectors + - Can be added together and multiplied by scalar + - Can be scaled by complex numbers + - Part of definitions + - Must satisfy vector axioms \ No newline at end of file diff --git a/Semiconductors/Equations.md b/Semiconductors/Equations.md index b2fd25d..7f81f16 100644 --- a/Semiconductors/Equations.md +++ b/Semiconductors/Equations.md @@ -11,7 +11,7 @@ $$J=\sigma E$$ $$V_{bi} = \frac{kT}{q}ln(\frac{N_D N_A}{n_i^2})$$ - $V_{bi}$ = Built-in Potential - +[[Doping]] $$J=nev$$ - $n$ = Charge Density - $e$ = Charge diff --git a/Signal Proc/Convolution.md b/Signal Proc/Convolution.md new file mode 100644 index 0000000..1b600b8 --- /dev/null +++ b/Signal Proc/Convolution.md @@ -0,0 +1,26 @@ +Integral operator +- Satisfies mathematical properties of integral operator +- Product of two after one has been reversed and shifted + +$$x(t)=x_1(t)\circledast x_2(t)=\int_{-\infty}^\infty x_1(t-\tau)\cdot x_2(\tau)d\tau$$ + +# Properties +1. $x_1(t)\circledast x_2(t)=x_2(t)\circledast x_1(t)$ + 1. Commutativity +2. $(x_1(t)\circledast x_2(t))\circledast x_3(t)=x_1(t)\circledast (x_2(t)\circledast x_3(t))$ + 1. Associativity +3. $x_1(t)\circledast [x_2(t)+x_3(t)]=x_1(t)\circledast x_2(t)+ x_1(t)\circledast x_3(t)$ + 1. Distributivity +4. $Ax_1(t)\circledast Bx_2(t)=AB[x_1(t)\circledast x_2(t)]$ + 1. Associativity with Scalar +5. Symmetrical graph about origin + +# Applications + +1. Communications systems + - Shift signal in frequency domain (Frequency modulation) +2. System analysis + - Find system output given input and transfer function + +# Polynomial Multiplication +- Convolving coefficients of two poly gives coefficients of product \ No newline at end of file diff --git a/Signal Proc/Fourier Transform.md b/Signal Proc/Fourier Transform.md new file mode 100644 index 0000000..63a960a --- /dev/null +++ b/Signal Proc/Fourier Transform.md @@ -0,0 +1,66 @@ +$$X(\omega)=\int_{-\infty}^{\infty}x(t)e^{-j\omega t}dt$$ +$$x(t)=\frac{1}{2\pi}\int_{2\pi}X(\omega)e^{j\omega t}d\omega$$ +## Discrete-Time +$$X(\omega)=\sum_{-\infty}^{\infty}x[n]e^{-j\omega n}$$ +$$x[n]=\frac{1}{2\pi}\int_{2\pi}X(\omega)e^{j\omega n}d\omega$$ + +## Discrete Fourier Transform +Digital Signal +$$X[k]=\sum_{n=0}^{N-1}x[n]e^{-j\omega_{k}n}$$ +$$x[n]=\frac{1}{N}\sum_{k=0}^{N-1}X[k]e^{j\omega_{k}n}, n=0,1,\ldots,N-1$$ + +## Power Spectral Density +PSD +$$P[k]=|X[k]|^2$$ + +## Spectrogram +- PSD vertically +- Frequency power over time horizontally +- ___Time and frequency resolution inversely proportional___ +- Resolution + - Frequency + - $fs/N$ + - Time + - $N/fs$ +- STFT has fixed resolution depending on window size + - Wider window + - Better frequency res + - Worse time resolution + - Can't tell where stuff changes with big window + - Can't use too wide + - Frequency can change during window +- 20-30ms window of speech usually treated as quasi-stationary +- Overlapping window + - Hop size of 5ms +- Appending windows can cause discontinuities + - Use window function to smooth + - Hann + +## Fast-Fourier +FFT +- Faster version of DFT + - Three parts + - Shuffling + - Bit reversal + - Shuffle N-dimensional input into N one-dimensional signals + - N one-point DFTs + - Merge + - N one-point DFTs into one N-point DFT + - Butterfly merging equations + +## Short-Time Fourier Transform +STFT + +- Short-term +- N-point windowed DFT + - Probably use FFT +$$x[k,m]=\sum_{n=0}^{N-1}x[m\delta+n]w(n)e^{-j\omega_kn}$$ +- $\omega$ + - Discrete angular frequency +- $m$ + - Time-frame index +- $\delta$ + - Hop size +- $w(n)$ + - Window function + - Hann diff --git a/Speech/Literature.md b/Speech/Literature.md new file mode 100644 index 0000000..7fe247b --- /dev/null +++ b/Speech/Literature.md @@ -0,0 +1,15 @@ +#lit +Daniel Jurafsky +James H. Martin + +[Speech and Language Processing - 3rd Ed. Draft](https://web.stanford.edu/~jurafsky/slp3/ed3book.pdf)[Hidden Markov Models](https://web.stanford.edu/~jurafsky/slp3/A.pdf) + +# Coursework +- [Stack Overflow, Spectrogram Matlab Explanation](https://stackoverflow.com/questions/29321696/what-is-a-spectrogram-and-how-do-i-set-its-parameters) +- [Matlab - LPC Analysis and Synthesis of Speech](https://uk.mathworks.com/help/dsp/ug/lpc-analysis-and-synthesis-of-speech.html) +- [Matlab - Formant Estimation with LPC Coefficients](https://uk.mathworks.com/help/signal/ug/formant-estimation-with-lpc-coefficients.html) +- [Matlab - Linear Prediction and Autoregressive Modeling](https://uk.mathworks.com/help/signal/ug/linear-prediction-and-autoregressive-modeling.html) +- [Quefrency Paper](https://www.researchgate.net/publication/3321562_From_Frequency_to_Quefrency_A_History_of_the_Cepstrum) +- [Aalto Uni - Pre-emphasis](https://wiki.aalto.fi/display/ITSP/Pre-emphasis) +- [Preemphasis paper](https://mini.dcs.shef.ac.uk/wp-content/papercite-data/pdf/loweimi_nolisp13.pdf) +- [Quora - Preemphasis](https://www.quora.com/Why-is-pre-emphasis-i-e-passing-the-speech-signal-through-a-first-order-high-pass-filter-required-in-speech-processing-and-how-does-it-work) \ No newline at end of file diff --git a/img/ai-io.png b/img/ai-io.png new file mode 100644 index 0000000..465392a Binary files /dev/null and b/img/ai-io.png differ diff --git a/img/ai-nested-subjects.png b/img/ai-nested-subjects.png new file mode 100644 index 0000000..c589f6c Binary files /dev/null and b/img/ai-nested-subjects.png differ diff --git a/img/cli-infrastructure.png b/img/cli-infrastructure.png new file mode 100644 index 0000000..0332583 Binary files /dev/null and b/img/cli-infrastructure.png differ