diff --git a/regression/lecture/regression-chapter.tex b/regression/lecture/regression-chapter.tex index e24239d..b235d10 100644 --- a/regression/lecture/regression-chapter.tex +++ b/regression/lecture/regression-chapter.tex @@ -16,6 +16,11 @@ \include{regression} +\subsection{Notes} +\begin{itemize} +\item Fig 8.2 right: this should be a chi-squared distribution with one degree of freedom! +\end{itemize} + \subsection{Start with one-dimensional problem!} \begin{itemize} \item Just the root mean square as a function of the slope diff --git a/regression/lecture/regression.tex b/regression/lecture/regression.tex index 0b78a16..4f98aa5 100644 --- a/regression/lecture/regression.tex +++ b/regression/lecture/regression.tex @@ -283,10 +283,11 @@ the partial derivatives using the difference quotient (Box~\ref{differentialquotientbox}) for small steps $\Delta m$ and $\Delta b$. For example, the partial derivative with respect to $m$ can be computed as -\[\frac{\partial f_{cost}(m,b)}{\partial m} = \lim\limits_{\Delta m \to +\begin{equation} + \frac{\partial f_{cost}(m,b)}{\partial m} = \lim\limits_{\Delta m \to 0} \frac{f_{cost}(m + \Delta m, b) - f_{cost}(m,b)}{\Delta m} -\approx \frac{f_{cost}(m + \Delta m, b) - f_{cost}(m,b)}{\Delta m} \; -. \] +\approx \frac{f_{cost}(m + \Delta m, b) - f_{cost}(m,b)}{\Delta m} \; . +\end{equation} The length of the gradient indicates the steepness of the slope (\figref{gradientquiverfig}). Since want to go down the hill, we choose the opposite direction. @@ -341,7 +342,9 @@ descent works as follows: sufficiently close to zero (e.g. \varcode{norm(gradient) < 0.1}). \item \label{gradientstep} If the length of the gradient exceeds the threshold we take a small step into the opposite direction: - \[p_{i+1} = p_i - \epsilon \cdot \nabla f_{cost}(m_i, b_i)\] + \begin{equation} + p_{i+1} = p_i - \epsilon \cdot \nabla f_{cost}(m_i, b_i) + \end{equation} where $\epsilon = 0.01$ is a factor linking the gradient to appropriate steps in the parameter space. \item Repeat steps \ref{computegradient} -- \ref{gradientstep}.