teaching/scientificComputing
Archived
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge branch 'master' of https://whale.am28.uni-tuebingen.de/git/teaching/scientificComputing

Browse Source
This commit is contained in:
Jan Grewe 2019-10-15 17:53:11 +02:00
commit 9e661f1951
2 changed files with 38 additions and 37 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
chapter.mk
View File

@ -9,8 +9,8 @@ pythonplots : $(PYPDFFILES)
$(PYPDFFILES) : %.pdf: %.py $(PYPDFFILES) : %.pdf: %.py
echo $$(which python) echo $$(which python)
#python3 $< python3 $<
python $< #python $<
cleanpythonplots : cleanpythonplots :
rm -f $(PYPDFFILES) rm -f $(PYPDFFILES)

71
likelihood/lecture/likelihood.tex
View File

@ -4,60 +4,61 @@
\chapter{\tr{Maximum likelihood estimation}{Maximum-Likelihood-Sch\"atzer}} \chapter{\tr{Maximum likelihood estimation}{Maximum-Likelihood-Sch\"atzer}}
\label{maximumlikelihoodchapter} \label{maximumlikelihoodchapter}
\selectlanguage{ngerman} \selectlanguage{english}
In vielen Situationen wollen wir einen oder mehrere Parameter $\theta$ There are situations in which we want to estimate one or more
einer Wahrscheinlichkeitsverteilung sch\"atzen, so dass die Verteilung parameters $\theta$ of a probability distribution that best describe
die Daten $x_1, x_2, \ldots x_n$ am besten beschreibt. the data $x_1, x_2, \ldots x_n$. \enterm{Maximum likelihood
\determ{Maximum-Likelihood-Sch\"atzer} (\enterm{maximum likelihood estimators} (\determ{Maximum-Likelihood-Sch\"atzer},
estimator}, \determ[mle|see{Maximum-Likelihood-Sch\"atzer}]{mle}) \determ[mle|see{Maximum-Likelihood-Sch\"atzer}]{mle}) choose the
w\"ahlen die Parameter so, dass die Wahrscheinlichkeit, dass die Daten parameters such that it maximizes the likelihood of $x_1, x_2, \ldots
aus der Verteilung stammen, am gr\"o{\ss}ten ist. x_n$ originating from the distribution.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Maximum Likelihood} \section{Maximum Likelihood}
Sei $p(x|\theta)$ (lies ``Wahrscheinlichkeit(sdichte) von $x$ gegeben
$\theta$'') die Wahrscheinlichkeits(dichte)verteilung von $x$ mit dem Let $p(x|\theta)$ (to be read as ``Probability(density) of $x$ given
Parameter(n) $\theta$. Das k\"onnte die Normalverteilung $\theta$.'') the probability (density) distribution of $x$ given the
parameters $\theta$. This could be the normal distribution
\begin{equation} \begin{equation}
\label{normpdfmean} \label{normpdfmean}
p(x|\theta) = \frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} p(x|\theta) = \frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}
\end{equation} \end{equation}
sein mit dem Mittelwert $\mu$ und der Standardabweichung $\sigma$ als defined by the mean ($\mu$) and the standard deviation $\sigma$ as
den Parametern $\theta$. parameters $\theta$. If the $n$ independent observations of $x_1,
x_2, \ldots x_n$ originate from the same probability density
Wenn nun den $n$ unabh\"angigen Beobachtungen $x_1, x_2, \ldots x_n$ distribution (\enterm{i.i.d.} independent and identically distributed)
die gleiche Wahrscheinlichkeitsverteilung $p(x|\theta)$ zugrundeliegt then is the conditional probability $p(x_1,x_2, \ldots x_n|\theta)$ of
(\enterm{i.i.d.} independent and identically distributed), dann ist die observing $x_1, x_2, \ldots x_n$ given the a specific $\theta$,
Verbundwahrscheinlichkeit $p(x_1,x_2, \ldots x_n|\theta)$ des
Auftretens der Werte $x_1, x_2, \ldots x_n$, gegeben ein bestimmtes
$\theta$,
\begin{equation} \begin{equation}
p(x_1,x_2, \ldots x_n|\theta) = p(x_1|\theta) \cdot p(x_2|\theta) p(x_1,x_2, \ldots x_n|\theta) = p(x_1|\theta) \cdot p(x_2|\theta)
\ldots p(x_n|\theta) = \prod_{i=1}^n p(x_i|\theta) \; . \ldots p(x_n|\theta) = \prod_{i=1}^n p(x_i|\theta) \; .
\end{equation} \end{equation}
Andersherum gesehen ist das die \determ{Likelihood} Vice versa, is the \enterm{likelihood} of the parameters $\theta$
(\enterm{likelihood}) den Parameter $\theta$ zu haben, gegeben die given the observed data $x_1, x_2, \ldots x_n$,
Me{\ss}werte $x_1, x_2, \ldots x_n$,
\begin{equation} \begin{equation}
{\cal L}(\theta|x_1,x_2, \ldots x_n) = p(x_1,x_2, \ldots x_n|\theta) \; . {\cal L}(\theta|x_1,x_2, \ldots x_n) = p(x_1,x_2, \ldots x_n|\theta) \; .
\end{equation} \end{equation}
Beachte, dass die Likelihood ${\cal L}$ keine Wahrscheinlichkeit im engeren Sinne ist, da sie sich nicht zu Eins aufintegriert ($\int {\cal L}(\theta|x_1,x_2, \ldots x_n) \, d\theta \ne 1$). Note: the likelihood ${\cal L}$ is not a probability in the
classic sense since it does not integrate to unity ($\int {\cal
L}(\theta|x_1,x_2, \ldots x_n) \, d\theta \ne 1$).
Wir sind nun an dem Wert des Parameters $\theta_{mle}$ interessiert, der die When applying maximum likelihood estimations we are interested in the
Likelihood maximiert (Maximum-Likelihood Estimate ``mle''): parameters $\theta$ that maximize the likelihood (``mle''):
\begin{equation} \begin{equation}
\theta_{mle} = \text{argmax}_{\theta} {\cal L}(\theta|x_1,x_2, \ldots x_n) \theta_{mle} = \text{argmax}_{\theta} {\cal L}(\theta|x_1,x_2, \ldots x_n)
\end{equation} \end{equation}
$\text{argmax}_xf(x)$ bezeichnet den Wert des Arguments $x$ der Funktion $f(x)$, bei $\text{argmax}_xf(x)$ denotes the values of the argument $x$ of the
dem $f(x)$ ihr globales Maximum annimmt. Wir suchen also den Wert von $\theta$ function $f(x)$ at which the function $f(x)$ reaches its global
bei dem die Likelihood ${\cal L}(\theta)$ ihr Maximum hat. maximum. Thus, we search the value of $\theta$ at which the
likelihood ${\cal L}(\theta)$ reaches its maximum.
The position of a function's maximum does not change when the values
of the function are transformed by a strictly monotonously rising
function such as the logarithm. For numerical and reasons that we will
discuss below, we commonly search for the maximum of the logarithm of
the likelihood (\enterm{log-likelihood}):
An der Stelle eines Maximums einer Funktion \"andert sich nichts, wenn
die Funktionswerte mit einer streng monoton steigenden Funktion
transformiert werden. Aus numerischen und gleich ersichtlichen mathematischen
Gr\"unden wird meistens das Maximum der logarithmierten Likelihood
(\determ{log-Likelihood}, \enterm{log-likelihood}) gesucht:
\begin{eqnarray} \begin{eqnarray}
\theta_{mle} & = & \text{argmax}_{\theta}\; {\cal L}(\theta|x_1,x_2, \ldots x_n) \nonumber \\ \theta_{mle} & = & \text{argmax}_{\theta}\; {\cal L}(\theta|x_1,x_2, \ldots x_n) \nonumber \\
& = & \text{argmax}_{\theta}\; \log {\cal L}(\theta|x_1,x_2, \ldots x_n) \nonumber \\ & = & \text{argmax}_{\theta}\; \log {\cal L}(\theta|x_1,x_2, \ldots x_n) \nonumber \\
@ -66,7 +67,7 @@ Gr\"unden wird meistens das Maximum der logarithmierten Likelihood
\end{eqnarray} \end{eqnarray}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Beispiel: Das arithmetische Mittel} \subsection{Example: the arithmetic mean}
Wenn die Me{\ss}daten $x_1, x_2, \ldots x_n$ der Normalverteilung Wenn die Me{\ss}daten $x_1, x_2, \ldots x_n$ der Normalverteilung
\eqnref{normpdfmean} entstammen, und wir den Mittelwert $\mu=\theta$ als \eqnref{normpdfmean} entstammen, und wir den Mittelwert $\mu=\theta$ als