|
|
|
|
@@ -80,12 +80,12 @@ used to illustrate the standard deviation of the data
|
|
|
|
|
\begin{figure}[t]
|
|
|
|
|
\includegraphics[width=1\textwidth]{median}
|
|
|
|
|
\titlecaption{\label{medianfig} Median, mean and mode of a
|
|
|
|
|
probability distribution.}{Left: Median, mean and mode are
|
|
|
|
|
identical for the symmetric and unimodal normal distribution.
|
|
|
|
|
Right: for asymmetric distributions these three measures differ. A
|
|
|
|
|
heavy tail of a distribution pulls out the mean most strongly. In
|
|
|
|
|
contrast, the median is more robust against heavy tails, but not
|
|
|
|
|
necessarily identical with the mode.}
|
|
|
|
|
probability distribution.}{Left: Median, mean and mode coincide
|
|
|
|
|
for the symmetric and unimodal normal distribution. Right: for
|
|
|
|
|
asymmetric distributions these three measures differ. A heavy tail
|
|
|
|
|
of a distribution pulls out the mean most strongly. In contrast,
|
|
|
|
|
the median is more robust against heavy tails, but not necessarily
|
|
|
|
|
identical with the mode.}
|
|
|
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
The \enterm{mode} is the most frequent value, i.e. the position of the maximum of the probability distribution.
|
|
|
|
|
@@ -113,7 +113,10 @@ not smaller than the median (\figref{medianfig}).
|
|
|
|
|
|
|
|
|
|
\begin{figure}[t]
|
|
|
|
|
\includegraphics[width=1\textwidth]{quartile}
|
|
|
|
|
\titlecaption{\label{quartilefig} Median and quartiles of a normal distribution.}{}
|
|
|
|
|
\titlecaption{\label{quartilefig} Median and quartiles of a normal
|
|
|
|
|
distribution.}{ The interquartile range between the first and the
|
|
|
|
|
third quartile contains 50\,\% of the data and contains the
|
|
|
|
|
median.}
|
|
|
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
The distribution of data can be further characterized by the position
|
|
|
|
|
@@ -164,7 +167,9 @@ The distribution of values in a data set is estimated by histograms
|
|
|
|
|
$N=\sum_{i=1}^M n_i$ measurements in each of $M$ bins $i$
|
|
|
|
|
(\figref{diehistogramsfig} left). The bins tile the data range
|
|
|
|
|
usually into intervals of the same size. The width of the bins is
|
|
|
|
|
called the bin width.
|
|
|
|
|
called the bin width. The frequencies $n_i$ plotted against the
|
|
|
|
|
categories $i$ is the \enterm{histogram}, or the \enterm{frequency
|
|
|
|
|
histogram}.
|
|
|
|
|
|
|
|
|
|
\begin{figure}[t]
|
|
|
|
|
\includegraphics[width=1\textwidth]{diehistograms}
|
|
|
|
|
@@ -219,7 +224,7 @@ category $i$, i.e. of getting a data value in the $i$-th bin.
|
|
|
|
|
\subsection{Probability densities functions}
|
|
|
|
|
|
|
|
|
|
In cases where we deal with data sets of measurements of a real
|
|
|
|
|
quantity (e.g. the length of snakes, the weight of elephants, the time
|
|
|
|
|
quantity (e.g. lengths of snakes, weights of elephants, times
|
|
|
|
|
between succeeding spikes) there is no natural bin width for computing
|
|
|
|
|
a histogram. In addition, the probability of measuring a data value that
|
|
|
|
|
equals exactly a specific real number like, e.g., 0.123456789 is zero, because
|
|
|
|
|
@@ -230,7 +235,7 @@ range. For example, we can ask for the probability $P(1.2<x<1.3)$ to
|
|
|
|
|
get a measurement between 0 and 1 (\figref{pdfprobabilitiesfig}). More
|
|
|
|
|
generally, we want to know the probability $P(x_0<x<x_1)$ to obtain a
|
|
|
|
|
measurement between $x_0$ and $x_1$. If we define the width of the
|
|
|
|
|
range defined by $x_0$ and $x_1$ is $\Delta x = x_1 - x_0$ then the
|
|
|
|
|
range between $x_0$ and $x_1$ as $\Delta x = x_1 - x_0$ then the
|
|
|
|
|
probability can also be expressed as $P(x_0<x<x_0 + \Delta x)$.
|
|
|
|
|
|
|
|
|
|
In the limit to very small ranges $\Delta x$ the probability of
|
|
|
|
|
@@ -238,44 +243,45 @@ getting a measurement between $x_0$ and $x_0+\Delta x$ scales down to
|
|
|
|
|
zero with $\Delta x$:
|
|
|
|
|
\[ P(x_0<x<x_0+\Delta x) \approx p(x_0) \cdot \Delta x \; . \]
|
|
|
|
|
In here the quantity $p(x_00)$ is a so called \enterm{probability
|
|
|
|
|
density}. This is not a unitless probability with values between 0
|
|
|
|
|
and 1, but a number that takes on any positive real number and has as
|
|
|
|
|
a unit the inverse of the unit of the data values --- hence the name
|
|
|
|
|
``density''.
|
|
|
|
|
density} that is larger than zero and that described the
|
|
|
|
|
distribution of the data values. The probability density is not a
|
|
|
|
|
unitless probability with values between 0 and 1, but a number that
|
|
|
|
|
takes on any positive real number and has as a unit the inverse of the
|
|
|
|
|
unit of the data values --- hence the name ``density''.
|
|
|
|
|
|
|
|
|
|
\begin{figure}[t]
|
|
|
|
|
\includegraphics[width=1\textwidth]{pdfprobabilities}
|
|
|
|
|
\titlecaption{\label{pdfprobabilitiesfig} Probability of a
|
|
|
|
|
probability density.}{The probability of a data value $x$ between,
|
|
|
|
|
e.g., zero and one is the integral (red area) over the probability
|
|
|
|
|
e.g., zero and one is the integral (red area) of the probability
|
|
|
|
|
density (blue).}
|
|
|
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
The probability to get a value $x$ between $x_1$ and $x_2$ is
|
|
|
|
|
given by the integral over the probability density:
|
|
|
|
|
given by the integral of the probability density:
|
|
|
|
|
\[ P(x_1 < x < x2) = \int\limits_{x_1}^{x_2} p(x) \, dx \; . \]
|
|
|
|
|
Because the probability to get any value $x$ is one, the integral over
|
|
|
|
|
the probability density
|
|
|
|
|
|
|
|
|
|
Da die Wahrscheinlichkeit irgendeines Wertes $x$ Eins ergeben muss gilt die Normierung
|
|
|
|
|
Because the probability to get any value $x$ is one, the integral of
|
|
|
|
|
the probability density over the whole real axis must be one:
|
|
|
|
|
\begin{equation}
|
|
|
|
|
\label{pdfnorm}
|
|
|
|
|
P(-\infty < x < \infty) = \int\limits_{-\infty}^{+\infty} p(x) \, dx = 1 \; .
|
|
|
|
|
\end{equation}
|
|
|
|
|
|
|
|
|
|
\pagebreak[2]
|
|
|
|
|
Die gesamte Funktion $p(x)$, die jedem Wert $x$ einen
|
|
|
|
|
Wahrscheinlichkeitsdichte zuordnet wir auch
|
|
|
|
|
\determ{Wahrscheinlichkeitsdichtefunktion} (\enterm{probability
|
|
|
|
|
density function}, \enterm[pdf|see{probability density
|
|
|
|
|
function}]{pdf}, oder kurz \enterm[density|see{probability density
|
|
|
|
|
function}]{density}) genannt. Die bekannteste
|
|
|
|
|
Wahrscheinlichkeitsdichtefunktion ist die der \determ{Normalverteilung}
|
|
|
|
|
\[ p_g(x) =
|
|
|
|
|
\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} \]
|
|
|
|
|
--- die \determ{Gau{\ss}sche-Glockenkurve} mit Mittelwert $\mu$ und
|
|
|
|
|
Standardabweichung $\sigma$.
|
|
|
|
|
The function $p(x)$, that assigns to every $x$ a probability density,
|
|
|
|
|
is called \enterm{probability density function},
|
|
|
|
|
\enterm[pdf|see{probability density function}]{pdf}, or just
|
|
|
|
|
\enterm[density|see{probability density function}]{density}
|
|
|
|
|
(\determ{Wahrscheinlichkeitsdichtefunktion}). The well known
|
|
|
|
|
\enterm{normal distribution} (\determ{Normalverteilung}) is an example of a
|
|
|
|
|
probability density function
|
|
|
|
|
\[ p_g(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}} \]
|
|
|
|
|
--- the \enterm{Guassian distribution}
|
|
|
|
|
(\determ{Gau{\ss}sche-Glockenkurve}) with mean $\mu$ and standard
|
|
|
|
|
deviation $\sigma$.
|
|
|
|
|
The factor in front of the exponential function ensures the normalization to
|
|
|
|
|
$\int p_g(x) \, dx = 1$, \eqnref{pdfnorm}.
|
|
|
|
|
|
|
|
|
|
\newpage
|
|
|
|
|
\begin{exercise}{gaussianpdf.m}{gaussianpdf.out}
|
|
|
|
|
\begin{enumerate}
|
|
|
|
|
\item Plot the probability density of the normal distribution $p_g(x)$.
|
|
|
|
|
@@ -288,6 +294,38 @@ Standardabweichung $\sigma$.
|
|
|
|
|
\end{enumerate}
|
|
|
|
|
\end{exercise}
|
|
|
|
|
|
|
|
|
|
\newpage
|
|
|
|
|
Histograms of real valued data depend on both the number of data
|
|
|
|
|
values and the chosen bin width. As in the example with the die
|
|
|
|
|
(\figref{diehistogramsfig} left), the height of the histogram gets
|
|
|
|
|
larger the larger the size of the data set. Also, as the bin width is
|
|
|
|
|
increased the hight of the histogram increases, because more data
|
|
|
|
|
values fall within each bin (\figref{pdfhistogramfig} left).
|
|
|
|
|
|
|
|
|
|
\begin{exercise}{gaussianbins.m}{}
|
|
|
|
|
Draw 100 random data from a Gaussian distribution and plot
|
|
|
|
|
histograms with different bin sizes of the data. What do you
|
|
|
|
|
observe?
|
|
|
|
|
\end{exercise}
|
|
|
|
|
|
|
|
|
|
To turn such histograms to estimates of probability densities they
|
|
|
|
|
need to be normalized such that according to \eqnref{pdfnorm} their
|
|
|
|
|
integral equals one. While histograms of categorial data are
|
|
|
|
|
normalized such that their sum equals one, here we need to integrate
|
|
|
|
|
over the histogram. The integral is the area (not the height) of the
|
|
|
|
|
histogram bars. Each bar has the height $n_i$ and the width $\Delta
|
|
|
|
|
x$. The total area $A$ of the histogram is thus
|
|
|
|
|
\[ A = \sum_{i=1}^N ( n_i \cdot \Delta x ) = \Delta x \sum_{i=1}^N n_i = N \, \Delta x \]
|
|
|
|
|
and the normalized histogram has the heights
|
|
|
|
|
\[ p(x_i) = \frac{n_i}{\Delta x \sum_{i=1}^N n_i} = \frac{n_i}{N
|
|
|
|
|
\Delta x} \; .\]
|
|
|
|
|
A histogram needs to be divided by both the sum of the frequencies
|
|
|
|
|
$n_i$ and the bin width $\Delta x$ to results in an estimate of the
|
|
|
|
|
corresponding probability density. Only then can the distribution be
|
|
|
|
|
compared with other distributions and in particular with theoretical
|
|
|
|
|
probability density functions like the one of the normal distribution
|
|
|
|
|
(\figref{pdfhistogramfig} right).
|
|
|
|
|
|
|
|
|
|
\begin{figure}[t]
|
|
|
|
|
\includegraphics[width=1\textwidth]{pdfhistogram}
|
|
|
|
|
\titlecaption{\label{pdfhistogramfig} Histograms with different bin
|
|
|
|
|
@@ -300,36 +338,106 @@ Standardabweichung $\sigma$.
|
|
|
|
|
normal distributions (blue).}
|
|
|
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
\pagebreak[4]
|
|
|
|
|
\begin{exercise}{gaussianbins.m}{}
|
|
|
|
|
Draw 100 random data from a Gaussian distribution and plot
|
|
|
|
|
histograms with different bin sizes of the data. What do you
|
|
|
|
|
observe?
|
|
|
|
|
\end{exercise}
|
|
|
|
|
|
|
|
|
|
Damit Histogramme von reellen Messwerten trotz unterschiedlicher
|
|
|
|
|
Anzahl von Messungen und unterschiedlicher Klassenbreiten
|
|
|
|
|
untereinander vergleichbar werden und mit bekannten
|
|
|
|
|
Wahrscheinlichkeitsdichtefunktionen verglichen werden k\"onnen,
|
|
|
|
|
m\"ussen sie auf das Integral Eins normiert werden
|
|
|
|
|
\eqnref{pdfnorm}. Das Integral (nicht die Summe) \"uber das Histogramm
|
|
|
|
|
soll Eins ergeben --- denn die Wahrscheinlichkeit, dass irgendeiner
|
|
|
|
|
der Messwerte auftritt mu{\ss} Eins sein. Das Integral ist die
|
|
|
|
|
Fl\"ache des Histogramms, die sich aus der Fl\"ache der einzelnen
|
|
|
|
|
Histogrammbalken zusammen setzt. Die Balken des Histogramms haben die
|
|
|
|
|
H\"ohe $n_i$ und die Breite $\Delta x$. Die Gesamtfl\"ache $A$ des
|
|
|
|
|
Histogramms ist also
|
|
|
|
|
\[ A = \sum_{i=1}^N ( n_i \cdot \Delta x ) = \Delta x \sum_{i=1}^N n_i \]
|
|
|
|
|
und das normierte Histogramm hat die H\"ohe
|
|
|
|
|
\[ p(x_i) = \frac{n_i}{\Delta x \sum_{i=1}^N n_i} \]
|
|
|
|
|
Es muss also nicht nur durch die Summe, sondern auch durch die Breite
|
|
|
|
|
$\Delta x$ der Klassen geteilt werden (\figref{pdfhistogramfig}).
|
|
|
|
|
|
|
|
|
|
\newpage
|
|
|
|
|
\begin{exercise}{gaussianbinsnorm.m}{}
|
|
|
|
|
Normiere das Histogramm der vorherigen \"Ubung zu einer Wahrscheinlichkeitsdichte.
|
|
|
|
|
Normalize the histogram of the previous exercise to a probability density.
|
|
|
|
|
\end{exercise}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
\newpage
|
|
|
|
|
\subsection{Kernel densities}
|
|
|
|
|
|
|
|
|
|
A problem of using histograms for estimating probability densities is
|
|
|
|
|
that the have hard bin edges. Depending on where the bin edges are placed
|
|
|
|
|
a data value falls in one or the other bin.
|
|
|
|
|
|
|
|
|
|
\begin{figure}[t]
|
|
|
|
|
\includegraphics[width=1\textwidth]{kerneldensity}
|
|
|
|
|
\titlecaption{\label{kerneldensityfig} Kernel densities.}{Left: The
|
|
|
|
|
histogram-based estimation of the probability density is dependent
|
|
|
|
|
also on the position of the bins. In the bottom plot the bins have
|
|
|
|
|
bin shifted by half a bin width (here $\Delta x=0.4$) and as a
|
|
|
|
|
result details of the probability density look different. Look,
|
|
|
|
|
for example at the height of the largest bin. Right: In contrast,
|
|
|
|
|
a kernel density is uniquely defined for a given kernel width
|
|
|
|
|
(here Gaussian kernels with standard deviation of $\sigma=2$).}
|
|
|
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
To avoid this problem one can use so called \enterm {kernel densities}
|
|
|
|
|
for estimating probability densities from data. Here every data point
|
|
|
|
|
is replaced by a kernel (a function with integral one, like for
|
|
|
|
|
example the Gaussian function) that is moved exactly to the position
|
|
|
|
|
indicated by the data value. Then all the kernels of all the data
|
|
|
|
|
values are summed up, the sum is divided by the number of data values,
|
|
|
|
|
and we get an estimate of the probability density.
|
|
|
|
|
|
|
|
|
|
As for the histogram, where we need to choose a bin width, we need to
|
|
|
|
|
choose the width of the kernels appropriately.
|
|
|
|
|
|
|
|
|
|
\newpage
|
|
|
|
|
\begin{exercise}{gaussiankerneldensity.m}{}
|
|
|
|
|
Construct and plot a kernel density of the data from the previous
|
|
|
|
|
two exercises.
|
|
|
|
|
\end{exercise}
|
|
|
|
|
|
|
|
|
|
\subsection{Cumulative distributions}
|
|
|
|
|
The \enterm{cumulative distribution function},
|
|
|
|
|
\enterm[cdf|see{cumulative distribution function}]{cdf}, or
|
|
|
|
|
\enterm[cumulative density function|see{cumulative distribution
|
|
|
|
|
function}]{cumulative density function}
|
|
|
|
|
(\determ{kumulative Verteilung}) is the integral over the probability density
|
|
|
|
|
up to any value $x$:
|
|
|
|
|
\[ F(x) = \int_{-\infty}^x p(x') \, dx' \]
|
|
|
|
|
As such the cumulative distribution is a probability. It is the
|
|
|
|
|
probability of getting a value smaller than $x$.
|
|
|
|
|
|
|
|
|
|
For estimating the cumulative distribution from a set of data values
|
|
|
|
|
we do not need to rely on histograms or kernel densities. Instead, it
|
|
|
|
|
can be computed from the data directly without the need of a bin width
|
|
|
|
|
or width of a kernel. For a data set of $N$ data values $x_i$ the
|
|
|
|
|
probability of a data value smaller than $x$ is the number of data
|
|
|
|
|
points with values smaller than $x$ divided by $N$. If we sort the
|
|
|
|
|
data values than at each data value $x_i$ the number of data elements
|
|
|
|
|
smaller than $x_i$ is increased by one and the corresponding
|
|
|
|
|
probability of getting a value smaller than $x_i$ is increased by $1/N$.
|
|
|
|
|
That is, the cumulative distribution is
|
|
|
|
|
\[ F(x_i) = \frac{i}{N} \]
|
|
|
|
|
See \figref{cumulativefig} for an example.
|
|
|
|
|
|
|
|
|
|
The cumulative distribution tells you the fraction of data that are
|
|
|
|
|
below a certain value and can therefore be used to evaluate significance
|
|
|
|
|
from Null-hypothesis constructed from data, as it is done with bootstrap methods
|
|
|
|
|
(see chapter \ref{bootstrapchapter}). The other way around the values of quartiles
|
|
|
|
|
and percentiles can be determined from the inverse cumulative function.
|
|
|
|
|
|
|
|
|
|
\begin{figure}[t]
|
|
|
|
|
\includegraphics[width=1\textwidth]{cumulative}
|
|
|
|
|
\titlecaption{\label{cumulativefig} Estimation of the cumulative
|
|
|
|
|
distribution.}{The cumulative distribution $F(x)$ estimated from
|
|
|
|
|
100 data values drawn from a normal distribution (red) in
|
|
|
|
|
comparison to the true cumulative distribution function computed
|
|
|
|
|
by numerically integrating the normal distribution function
|
|
|
|
|
(blue). From the cumulative distribution function one can read of
|
|
|
|
|
the probabilities of getting values smaller than a given value
|
|
|
|
|
(here: $P(x \ge -1) \approx 0.15$). From the inverse cumulative
|
|
|
|
|
distribution the position of percentiles can be computed (here:
|
|
|
|
|
the median (50\,\% percentile) is as expected close to zero and
|
|
|
|
|
the third quartile (75\,\% percentile) at $x=0.68$.}
|
|
|
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
\begin{exercise}{cumulative.m}{cumulative.out}
|
|
|
|
|
Generate 200 normally distributed data values and construct an
|
|
|
|
|
estimate of the cumulative distribution function from this data.
|
|
|
|
|
|
|
|
|
|
Compare this estimate with an integral over the normal distribution.
|
|
|
|
|
|
|
|
|
|
Use the estimate to compute the probability of having data values
|
|
|
|
|
smaller than $-1$.
|
|
|
|
|
|
|
|
|
|
Use the estimate to compute the value of the 5\,\% percentile.
|
|
|
|
|
\end{exercise}
|
|
|
|
|
|
|
|
|
|
\newpage
|
|
|
|
|
\section{Correlations}
|
|
|
|
|
|
|
|
|
|
Until now we described properties of univariate data sets. In
|
|
|
|
|
@@ -353,7 +461,10 @@ data in a correlation coefficient close to zero
|
|
|
|
|
|
|
|
|
|
\begin{figure}[tp]
|
|
|
|
|
\includegraphics[width=1\textwidth]{correlation}
|
|
|
|
|
\titlecaption{\label{correlationfig} Korrelationen zwischen Datenpaaren.}{}
|
|
|
|
|
\titlecaption{\label{correlationfig} Correlations between pairs of
|
|
|
|
|
data.}{Shown are scatter plots of four data sets. Each point is a
|
|
|
|
|
single data pair. The correlation coefficient $r$ is given in the top
|
|
|
|
|
left of each plot.}
|
|
|
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
\begin{exercise}{correlations.m}{}
|
|
|
|
|
|
