diff --git a/README.md b/README.md index 831a0f3..ce48239 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ Hidden Markov Models ==================== -Speech recognition coursework focusing on training and analysing hidden markov models. +Speech recognition coursework focusing on training and analysing hidden markov models, main dev notebook [here](markov.ipynb). ![PDFs with observations marked](report/res/pdfs-w-obs.png) Probability density functions with provided observations marked diff --git a/markov.ipynb b/markov.ipynb index a8fd9fd..6ae0a04 100644 --- a/markov.ipynb +++ b/markov.ipynb @@ -24,7 +24,7 @@ "cells": [ { "cell_type": "code", - "execution_count": 38, + "execution_count": 2, "metadata": {}, "outputs": [], "source": [ @@ -113,14 +113,14 @@ "output_type": "stream", "name": "stdout", "text": [ - "3.8 -> State 1: 0.02185157424475792, State 2: {gaussian(obs, state2.mean, state2.std_dev)}\n4.2 -> State 1: 0.009496655019831194, State 2: {gaussian(obs, state2.mean, state2.std_dev)}\n3.4 -> State 1: 0.04499247209432338, State 2: {gaussian(obs, state2.mean, state2.std_dev)}\n-0.4 -> State 1: 0.16833223796171576, State 2: {gaussian(obs, state2.mean, state2.std_dev)}\n1.9 -> State 1: 0.2509478601290037, State 2: {gaussian(obs, state2.mean, state2.std_dev)}\n3.0 -> State 1: 0.08289761566062391, State 2: {gaussian(obs, state2.mean, state2.std_dev)}\n1.6 -> State 1: 0.2933877723035829, State 2: {gaussian(obs, state2.mean, state2.std_dev)}\n1.9 -> State 1: 0.2509478601290037, State 2: {gaussian(obs, state2.mean, state2.std_dev)}\n5.0 -> State 1: 0.0012852324969092556, State 2: {gaussian(obs, state2.mean, state2.std_dev)}\n" + "3.8 -> State 1: 0.02185157424475792, State 2: 0.5471239427774459\n4.2 -> State 1: 0.009496655019831194, State 2: 0.5471239427774459\n3.4 -> State 1: 0.04499247209432338, State 2: 0.3947074079064296\n-0.4 -> State 1: 0.16833223796171576, State 2: 1.500652790137751e-09\n1.9 -> State 1: 0.2509478601290037, State 2: 0.006331212017054291\n3.0 -> State 1: 0.08289761566062391, State 2: 0.2054255182126689\n1.6 -> State 1: 0.2933877723035829, State 2: 0.001596702666402633\n1.9 -> State 1: 0.2509478601290037, State 2: 0.006331212017054291\n5.0 -> State 1: 0.0012852324969092556, State 2: 0.2054255182126689\n" ] } ], "source": [ "for obs in observations:\n", " print(f'{obs} -> State 1: {gaussian(obs, state1.mean, state1.std_dev)},', \n", - " 'State 2: {gaussian(obs, state2.mean, state2.std_dev)}')" + " f'State 2: {gaussian(obs, state2.mean, state2.std_dev)}')" ] }, { diff --git a/report/references.bib b/report/references.bib index e254534..ed57ee2 100644 --- a/report/references.bib +++ b/report/references.bib @@ -48,4 +48,14 @@ urldate = {2021-1-1}, } +@Book{standford-hmm-book, + author = {Dan Jurafsky and James H. Martin}, + title = {Speech and Language Processing}, + year = {2020}, + edition = {3 (Draft)}, + chapter = {A}, + url = {https://web.stanford.edu/~jurafsky/slp3/}, + urldate = {2021-1-3}, +} + @Comment{jabref-meta: databaseType:bibtex;} diff --git a/report/report.lyx b/report/report.lyx index 8407d98..b258bb7 100644 --- a/report/report.lyx +++ b/report/report.lyx @@ -8,7 +8,7 @@ \begin_preamble \def\changemargin#1#2{\list{}{\rightmargin#2\leftmargin#1}\item[]} \let\endchangemargin=\endlist -\pagenumbering{roman} +\pagenumbering{gobble} \usepackage{pxfonts} \usepackage{color} @@ -182,6 +182,22 @@ University of Surrey \end_inset +\end_layout + +\begin_layout Standard +\begin_inset ERT +status open + +\begin_layout Plain Layout + + +\backslash +pagenumbering{roman} +\end_layout + +\end_inset + + \end_layout \begin_layout Abstract @@ -282,6 +298,16 @@ literal "false" memoryless \emph default . + The hidden Markov model extends the standard Markov chain by modeling a + system where the states are not directly observable but are hidden +\begin_inset CommandInset citation +LatexCommand cite +key "standford-hmm-book" +literal "false" + +\end_inset + +, instead the states are inferred from the visible observations. \end_layout \begin_layout Standard @@ -678,6 +704,31 @@ noprefix "false" ns. \end_layout +\begin_layout Standard +The forward and backward procedure are outlined by +\begin_inset CommandInset citation +LatexCommand cite +key "standford-hmm-book" +literal "false" + +\end_inset + +, the equations for both forward and backward likelihood can be seen below, +\end_layout + +\begin_layout Standard +\begin_inset Flex TODO Note (inline) +status open + +\begin_layout Plain Layout +Finish? +\end_layout + +\end_inset + + +\end_layout + \begin_layout Section Results \end_layout @@ -3261,6 +3312,7 @@ The development of the model behind this report was completed using Jupyter The used notebook can be seen formatted in plain text below, the relevant mark scheme elements are referenced in brackets ( \begin_inset listings +lstparams "language=Python,basicstyle={\ttfamily}" inline true status open @@ -3310,6 +3362,11 @@ noprefix "false" shows how the output probability density functions move during 5 iterations of training. + State 1's output function can be seen to spread out (increasing variance) + and shift slightly to the right (increasing mean). + State 2's mean begins by increasing before decreasing below the initial + value. + The variance also decreases, tightening the function. \end_layout \begin_layout Standard @@ -3354,6 +3411,20 @@ name "fig:5-training-iterations" \end_inset +\end_layout + +\begin_layout Standard +\begin_inset Formula +\[ +\hat{a}_{ij}=\left[\begin{array}{cc} +0.80 & 0.19\\ +0.26 & 0.50 +\end{array}\right] +\] + +\end_inset + + \end_layout \end_body