teaching/scientificComputing
Archived
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge branch 'master' of https://whale.am28.uni-tuebingen.de/git/teaching/scientificComputing

Browse Source
This commit is contained in:
Jan Grewe 2019-10-15 17:53:11 +02:00
commit 9e661f1951
2 changed files with 39 additions and 38 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
chapter.mk
View File

@ -9,8 +9,8 @@ pythonplots : $(PYPDFFILES)
$(PYPDFFILES) : %.pdf: %.py $(PYPDFFILES) : %.pdf: %.py
echo $$(which python) echo $$(which python)
#python3 $< python3 $<
python $< #python $<
cleanpythonplots : cleanpythonplots :
rm -f $(PYPDFFILES) rm -f $(PYPDFFILES)

73
likelihood/lecture/likelihood.tex
View File

@ -4,60 +4,61 @@
\chapter{\tr{Maximum likelihood estimation}{Maximum-Likelihood-Sch\"atzer}} \chapter{\tr{Maximum likelihood estimation}{Maximum-Likelihood-Sch\"atzer}}
\label{maximumlikelihoodchapter} \label{maximumlikelihoodchapter}
\selectlanguage{ngerman} \selectlanguage{english}
In vielen Situationen wollen wir einen oder mehrere Parameter $\theta$ There are situations in which we want to estimate one or more
einer Wahrscheinlichkeitsverteilung sch\"atzen, so dass die Verteilung parameters $\theta$ of a probability distribution that best describe
die Daten $x_1, x_2, \ldots x_n$ am besten beschreibt. the data $x_1, x_2, \ldots x_n$. \enterm{Maximum likelihood
\determ{Maximum-Likelihood-Sch\"atzer} (\enterm{maximum likelihood estimators} (\determ{Maximum-Likelihood-Sch\"atzer},
estimator}, \determ[mle|see{Maximum-Likelihood-Sch\"atzer}]{mle}) \determ[mle|see{Maximum-Likelihood-Sch\"atzer}]{mle}) choose the
w\"ahlen die Parameter so, dass die Wahrscheinlichkeit, dass die Daten parameters such that it maximizes the likelihood of $x_1, x_2, \ldots
aus der Verteilung stammen, am gr\"o{\ss}ten ist. x_n$ originating from the distribution.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Maximum Likelihood} \section{Maximum Likelihood}
Sei $p(x|\theta)$ (lies ``Wahrscheinlichkeit(sdichte) von $x$ gegeben
$\theta$'') die Wahrscheinlichkeits(dichte)verteilung von $x$ mit dem Let $p(x|\theta)$ (to be read as ``Probability(density) of $x$ given
Parameter(n) $\theta$. Das k\"onnte die Normalverteilung $\theta$.'') the probability (density) distribution of $x$ given the
parameters $\theta$. This could be the normal distribution
\begin{equation} \begin{equation}
\label{normpdfmean} \label{normpdfmean}
p(x|\theta) = \frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} p(x|\theta) = \frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}
\end{equation} \end{equation}
sein mit dem Mittelwert $\mu$ und der Standardabweichung $\sigma$ als defined by the mean ($\mu$) and the standard deviation $\sigma$ as
den Parametern $\theta$. parameters $\theta$. If the $n$ independent observations of $x_1,
x_2, \ldots x_n$ originate from the same probability density
Wenn nun den $n$ unabh\"angigen Beobachtungen $x_1, x_2, \ldots x_n$ distribution (\enterm{i.i.d.} independent and identically distributed)
die gleiche Wahrscheinlichkeitsverteilung $p(x|\theta)$ zugrundeliegt then is the conditional probability $p(x_1,x_2, \ldots x_n|\theta)$ of
(\enterm{i.i.d.} independent and identically distributed), dann ist die observing $x_1, x_2, \ldots x_n$ given the a specific $\theta$,
Verbundwahrscheinlichkeit $p(x_1,x_2, \ldots x_n|\theta)$ des
Auftretens der Werte $x_1, x_2, \ldots x_n$, gegeben ein bestimmtes
$\theta$,
\begin{equation} \begin{equation}
p(x_1,x_2, \ldots x_n|\theta) = p(x_1|\theta) \cdot p(x_2|\theta) p(x_1,x_2, \ldots x_n|\theta) = p(x_1|\theta) \cdot p(x_2|\theta)
\ldots p(x_n|\theta) = \prod_{i=1}^n p(x_i|\theta) \; . \ldots p(x_n|\theta) = \prod_{i=1}^n p(x_i|\theta) \; .
\end{equation} \end{equation}
Andersherum gesehen ist das die \determ{Likelihood} Vice versa, is the \enterm{likelihood} of the parameters $\theta$
(\enterm{likelihood}) den Parameter $\theta$ zu haben, gegeben die given the observed data $x_1, x_2, \ldots x_n$,
Me{\ss}werte $x_1, x_2, \ldots x_n$,
\begin{equation} \begin{equation}
{\cal L}(\theta|x_1,x_2, \ldots x_n) = p(x_1,x_2, \ldots x_n|\theta) \; . {\cal L}(\theta|x_1,x_2, \ldots x_n) = p(x_1,x_2, \ldots x_n|\theta) \; .
\end{equation} \end{equation}
Beachte, dass die Likelihood ${\cal L}$ keine Wahrscheinlichkeit im engeren Sinne ist, da sie sich nicht zu Eins aufintegriert ($\int {\cal L}(\theta|x_1,x_2, \ldots x_n) \, d\theta \ne 1$). Note: the likelihood ${\cal L}$ is not a probability in the
classic sense since it does not integrate to unity ($\int {\cal
L}(\theta|x_1,x_2, \ldots x_n) \, d\theta \ne 1$).
Wir sind nun an dem Wert des Parameters $\theta_{mle}$ interessiert, der die When applying maximum likelihood estimations we are interested in the
Likelihood maximiert (Maximum-Likelihood Estimate ``mle''): parameters $\theta$ that maximize the likelihood (``mle''):
\begin{equation} \begin{equation}
\theta_{mle} = \text{argmax}_{\theta} {\cal L}(\theta|x_1,x_2, \ldots x_n) \theta_{mle} = \text{argmax}_{\theta} {\cal L}(\theta|x_1,x_2, \ldots x_n)
\end{equation} \end{equation}
$\text{argmax}_xf(x)$ bezeichnet den Wert des Arguments $x$ der Funktion $f(x)$, bei $\text{argmax}_xf(x)$ denotes the values of the argument $x$ of the
dem $f(x)$ ihr globales Maximum annimmt. Wir suchen also den Wert von $\theta$ function $f(x)$ at which the function $f(x)$ reaches its global
bei dem die Likelihood ${\cal L}(\theta)$ ihr Maximum hat. maximum. Thus, we search the value of $\theta$ at which the
likelihood ${\cal L}(\theta)$ reaches its maximum.
An der Stelle eines Maximums einer Funktion \"andert sich nichts, wenn
die Funktionswerte mit einer streng monoton steigenden Funktion The position of a function's maximum does not change when the values
transformiert werden. Aus numerischen und gleich ersichtlichen mathematischen of the function are transformed by a strictly monotonously rising
Gr\"unden wird meistens das Maximum der logarithmierten Likelihood function such as the logarithm. For numerical and reasons that we will
(\determ{log-Likelihood}, \enterm{log-likelihood}) gesucht: discuss below, we commonly search for the maximum of the logarithm of
the likelihood (\enterm{log-likelihood}):
\begin{eqnarray} \begin{eqnarray}
\theta_{mle} & = & \text{argmax}_{\theta}\; {\cal L}(\theta|x_1,x_2, \ldots x_n) \nonumber \\ \theta_{mle} & = & \text{argmax}_{\theta}\; {\cal L}(\theta|x_1,x_2, \ldots x_n) \nonumber \\
& = & \text{argmax}_{\theta}\; \log {\cal L}(\theta|x_1,x_2, \ldots x_n) \nonumber \\ & = & \text{argmax}_{\theta}\; \log {\cal L}(\theta|x_1,x_2, \ldots x_n) \nonumber \\
@ -66,7 +67,7 @@ Gr\"unden wird meistens das Maximum der logarithmierten Likelihood
\end{eqnarray} \end{eqnarray}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Beispiel: Das arithmetische Mittel} \subsection{Example: the arithmetic mean}
Wenn die Me{\ss}daten $x_1, x_2, \ldots x_n$ der Normalverteilung Wenn die Me{\ss}daten $x_1, x_2, \ldots x_n$ der Normalverteilung
\eqnref{normpdfmean} entstammen, und wir den Mittelwert $\mu=\theta$ als \eqnref{normpdfmean} entstammen, und wir den Mittelwert $\mu=\theta$ als