From 78f6b352a2565450cab45264e2159464338fbd2c Mon Sep 17 00:00:00 2001 From: Jan Grewe Date: Fri, 3 Nov 2017 18:05:15 +0100 Subject: [PATCH] [debugging] add section about unit testing in matlab --- debugging/lecture/debugging.tex | 194 +++++++++++++++++++++++++++++--- 1 file changed, 181 insertions(+), 13 deletions(-) diff --git a/debugging/lecture/debugging.tex b/debugging/lecture/debugging.tex index e9e7f2b..d0af656 100644 --- a/debugging/lecture/debugging.tex +++ b/debugging/lecture/debugging.tex @@ -112,8 +112,8 @@ to a number and uses this number to address the element in thus the 65th element of \varcode{my\_array} is returned. \subsection{\codeterm{Assignment error}} -Related to the Indexing error this error occurs when we want to write -data into a variable, that does not fit into it. Listing +Related to the Indexing error, an assignment error occurs when we want +to write data into a variable, that does not fit into it. Listing \ref{assignmenterror} shows the simple case for 1-d data but, of course, it extents to n-dimensional data. The data that is to be filled into a matrix hat to fit in all dimensions. The command in line @@ -172,11 +172,12 @@ expected elementwise multiplication. \section{Logical error} Sometimes a program runs smoothly and terminates without any -complaint. This, however, does not necessarily mean that the program is -correct. We may have made a \codeterm{logical error}. Logical errors -are hard to find, \matlab{} has no chance to find such an error and can -not help us fixing bugs origination from these. We are on our own but -there are a few strategies that should help us. +complaint. This, however, does not necessarily mean that the program +is correct. We may have made a \codeterm{logical error}. Logical +errors are hard to find, \matlab{} has no chance to detect such errors +since they do not violate the syntax or cause the throwing of an +error. Thus, we are on our own to find and fix the bug. There are a +few strategies that should we can employ to solve the task. \begin{enumerate} \item Be sceptical: especially when a program executes without any @@ -185,21 +186,25 @@ there are a few strategies that should help us. it. Comment, but only where necessary. Correctly indent your code. Use descriptive variable and function names. \item Keep it simple. +\item Test your code by writing \codeterm{unit tests} that test every + aspect of your program (\ref{unittests}). \item Use scripts and functions and call them from the command line. \matlab{} can then provide you with more information. It will then point to the line where the error happens. \item If you still find yourself in trouble: Apply debugging - strategies to find and fix bugs (below). + strategies to find and fix bugs (\ref{debugging}). \end{enumerate} -\subsection{Avoiding errors --- Keep it small and simple} - +\section{Avoiding errors} It would be great if we could just sit down, write a program, run it, and be done with the task. Most likely this will not happen. Rather, we will make mistakes and have to bebug the code. There are a few guidelines that help to reduce the number of errors. + +\subsection{Keep it small and simple} + \shortquote{Debugging time increases as a square of the program's size.}{Chris Wenham} @@ -247,10 +252,173 @@ into the development of the most elegant solution relative to its importance in the project? The decision is yours. -\section{Debugging strategies} +\subsection{Unit tests}\label{unittests} + +The idea of unit tests to write small programs that test \emph{all} +functions of a program by testing the program's results against +expectations. The pure lore of test-driven development requires that +the tests are written \textbf{before} the actual program is +written. In parts the tests put the \codeterm{functional + specification}, the agreement between customer and programmer, into +code. This helps to guarantee that the delivered program works as +specified. In the scientific context, we tend to be a little bit more +relaxed and write unit tests, where we think them helpful and often +test only the obvious things. To write \emph{complete} test suits that +lead to full \codeterm{test coverage} is a lot of work and is often +considered a waste of time. The first claim is true, the second, +however, may be doubted. Consider that you change a tiny bit of a +standing program to adjust it to the current needs, how will you be +able to tell that it is still valid for the previous purpose? Of +course you could try it out and be satisfied, if it terminates without +an error, but, remember, there may be logical errors hiding behind the +facade of a working program. + +Writing unit tests costs time, but provides the means to guarantee +validity. + +\subsubsection{Unit testing in \matlab{}} + +Matlab offers a unit testing framework in which small scripts are +written that test the features of the program. We will follow the +example given in the \matlab{} help and assume that there is a +function \code{rightTriangle} (listing\,\ref{trianglelisting}). + + +\begin{lstlisting}[label=trianglelisting, caption={Slightly more readable version of the example given in the \matlab{} help system. Note: The variable name for the angles have been capitalized in order to not override the matlab defined functions \code{alpha, beta,} and \code{gamma}.}] +function angles = rightTriangle(length_a, length_b) + ALPHA = atand(length_a / length_b); + BETA = atand(length_a / length_b); + hypotenuse = length_a / sind(ALPHA); + GAMMA = asind(hypotenuse * sind(ALPHA) / length_a); + + angles = [ALPHA BETA GAMMA]; +end +\end{lstlisting} + +This function expects two input arguments that are the length of the +sides $a$ and $b$ and assumes a right angle between them. From this +information it calculates and returns the angles $\alpha, \beta,$ and +$\gamma$. + +Let's test this function: To do so, create a script in the current +folder that follows the following rules. +\begin{enumerate} +\item The name of the script file must start or end with the word + 'test', which is case-insensitive. +\item Each unit test should be placed in a separate section/cell of the script. +\item After the \code{\%\%} that defines the cell, a name for the + particular unit test may be given. +\end{enumerate} + +Further there are a few things that are different in tests compared to normal scripts. +\begin{enumerate} +\item The code that appears before the first section is the in the so + called \emph{shared variables section} and the variables are available to + all tests within this script. +\item In the \emph{shared variables section}, one can define + preconditions necessary for your tests. If these preconditions are + not met, the remaining tests will not be run and the test will be + considered failed and incomplete. +\item When a script is run as a test, all variables that need to be + accessible in all test have to be defined in the \emph{shared + variables section}. +\item Variables defined in other workspaces are not accessible to the + tests. +\end{enumerate} + +The test script for the \code{rightTrianlge} function +(listing\,\ref{trianglelisting}) may look like in +listing\,\ref{testscript}. + +\begin{lstlisting}[label=testscript, caption={Unit test for the \code{rightTriangle} function stored in an m-file testRightTriangle.m}] +tolerance = 1e-10; + +% preconditions +angles = rightTriangle(7, 9); +assert(angles(3) == 90, 'Fundamental problem: rightTriangle is not producing a right triangle') + +%% Test 1: sum of angles +angles = rightTriangle(7, 7); +assert((sum(angles) - 180) <= tolerance) + +angles = rightTriangle(7, 7); +assert((sum(angles) - 180) <= tolerance) + +angles = rightTriangle(2, 2 * sqrt(3)); +assert((sum(angles) - 180) <= tolerance) + +angles = rightTriangle(1, 150); +assert((sum(angles) - 180) <= tolerance) + +%% Test: isosceles triangles +angles = rightTriangle(4, 4); +assert(abs(angles(1) - 45) <= tolerance) +assert(angles(1) == angles(2)) + +%% Test: 30-60-90 triangle +angles = rightTriangle(2, 2 * sqrt(3)); +assert(abs(angles(1) - 30) <= tolerance) +assert(abs(angles(2) - 60) <= tolerance) +assert(abs(angles(3) - 90) <= tolerance) + +%% Test: Small angle approx +angles = rightTriangle(1, 1500); +smallAngle = (pi / 180) * angles(1); % radians +approx = sin(smallAngle); +assert(abs(approx - smallAngle) <= tolerance, 'Problem with small angle approximation') +\end{lstlisting} + +In a test script we can execute any code. The actual test whether or +not the results match our predictions is done using the +\code{assert()}{assert} function. This function basically expects a +boolean value and if this is not true, it raises an error that, in the +context of the test does not lead to a termination of the program. In +the tests above, the argument to assert is always a boolean expression +which evaluates to \code{true} or \code{false}. Before the first unit +test (``Test 1: sum of angles'', that starts in line 5, +listing\,\ref{testscript}) a precondition is defined. The test assumes +that the $\gamma$ angle must always be 90$^\circ$ since we aim for a +right triangle. If this is not true, the further tests, will not be +executed. We further define a \varcode{tolerance} variable that is +used when comparing double values (Why might the test on equality of +double values be tricky?). + +\begin{lstlisting}[label=runtestlisting, caption={Run the test!}] +result = runtests('testRightTriangle') +\end{lstlisting} + +During the run, \matlab{} will put out error messages onto the command +line and a summary of the test results is then stored within the +\varcode{result} variable. These can be displayed using the function +\code{table(result)} + +\begin{lstlisting}[label=testresults, caption={The test results.}, basicstyle=\ttfamily\scriptsize] +table(result) +ans = + 4x6 table + + Name Passed Failed Incomplete Duration Details +_________________________________ ______ ______ ___________ ________ ____________ + +'testR.../Test_SumOfAngles' true false false 0.011566 [1x1 struct] +'testR.../Test_IsoscelesTriangles' true false false 0.004893 [1x1 struct] +'testR.../Test_30_60_90Triangle' true false false 0.005057 [1x1 struct] +'testR.../Test_SmallAngleApprox' true false false 0.0049 [1x1 struct] +\end{lstlisting} + +So far so good, all tests pass and our function appears to do what it +is supposed to do. But tests are only as good as the programmer who +designed them. The attentive reader may have noticed that the tests +only check a few conditions. But what if we passed something else than +a numeric value as the length of the sides $a$ and $b$? Or a negative +number, or zero? + + + +\section{Debugging strategies}\label{debugging} -If you find yourself in trouble you can apply a few strategies to -solve the problem. +If you still find yourself in trouble you can apply a few strategies +that help to solve the problem. \begin{enumerate} \item Lean back and take a breath.