~mccane/practical-programming/trunk

\chapter{Modules and TDD}
\label{lec-tdd}

This chapter primarily deals with a development methodology called Test Driven Development (TDD). Before we get to TDD though, we need a couple of extra parts of python: modules and triple quoted strings.

\section{Modules, files and the \pythoninline{import} statement}

A \myidx{module} is simply a Python file that contains
definitions (usually function definitions). As programs get large, it
makes sense to separate the program into different modules. This
allows the programmer to minimize the complexity of the programming
task. Typically, similar functionality or related functions are
grouped into a single module and separate modules are as independent
of each other as possible. Creating your own modules is as easy as
creating a Python script with definitions in it; in fact you have already 
written lots of Python modules.

The Python standard library contains a very large number of modules
for many different tasks. It is worthwhile browsing the library just
to get a feel for all the useful things it can do. The current
documentation can be accessed at:
\myurl{http://www.python.org/doc/current/library/index.html}, or by
choosing \menu{Help \rarr Python Docs ...} in IDLE. We will
look at the \pythoninline{doctest} module later in this lecture, and
throughout the course several other modules will be introduced.

There are three ways of using \pythoninline{import} to import a module
and subsequently use what was imported. The \pythoninline{math} module contains many 
useful math functions, we will use it to illustrate:

\begin{enumerate}

\item

\begin{python}
import math
print math.cos(math.pi/2.0)
\end{python}
This statement imports every function from the \pythoninline{math} module which
 are then accessed using dot notation.

\item

\begin{python}
from math import cos, pi
print cos(pi/2.0)
\end{python}
This statement imports only the definitions of \pythoninline{cos} and
\pythoninline{pi} from the math library. Nothing else is imported.
\item

\begin{python}
from math import *
print cos(pi/2.0)
\end{python}
This statement also imports everything from the \pythoninline{math}
module, the difference being that dot notation is not needed to access
module members.

\end{enumerate}
%Also, do you need/want to include the Restart between each example, otherwise the previous import is still imported.
It is more common to use the first or second alternative than the last.
The first has the advantage of avoiding naming conflicts (two
different modules defining \pythoninline{cos} for example), at the
cost of lengthier function calls. The second has the
opposite advantages and disadvantages.

We can use the \pythoninline{help} function to see the functions and
data contained within modules. The keyword module
contains a single function, \pythoninline{iskeyword}, which as its
name suggests is a boolean function that returns \pythoninline{True}
if a string passed to it is a keyword:

\begin{pythonInteractive}
>>> from keyword import *
>>> iskeyword("for")
True
>>> iskeyword("all")
False
>>>
\end{pythonInteractive}
The data item, \pythoninline{kwlist} contains a list of all the
current keywords in Python:
\begin{pythonInteractive}
>>> from keyword import *
>>> print kwlist
['and', 'as', 'assert', 'break', 'class', 'continue', 'def', 'del', 'elif',
'else', 'except', 'exec', 'finally', 'for', 'from', 'global', 'if', 'import',
'in', 'is', 'lambda', 'not', 'or', 'pass', 'print', 'raise', 'return', 'try',
'while', 'with', 'yield']
>>>
\end{pythonInteractive}
We encourage you to check out \url{http://docs.python.org/library/}
to explore the extensive libraries that 
come with Python. There are so many treasures to discover!


\section{Triple quoted strings}
In addition to the single and double quoted strings we first saw in
Lecture \ref{lec-variables}, Python also has \emph{triple quoted strings}. We will need triple quoted strings for Unit Testing (next section). Here are some examples:

\begin{minipage}{\textwidth}
\begin{pythonInteractive}
>>> type("""This is a triple quoted string using 3 double quotes.""")
<type 'str'>
>>> type('''This triple quoted strings uses 3 single quotes.''')
<type 'str'>
>>>
\end{pythonInteractive}
\end{minipage}

Triple quoted strings can contain both single and double quotes inside
them:

\begin{minipage}{\textwidth}
\begin{pythonInteractive}
>>> print '''"Oh no", she exclaimed, "Ben's bike is broken!"'''
"Oh no", she exclaimed, "Ben's bike is broken!"
>>>
\end{pythonInteractive}
\end{minipage}

Finally, triple quoted strings can span multiple lines:

\noindent
\begin{minipage}{\textwidth}
\begin{pythonInteractive}
>>> message = """This message will
... span several
... lines."""
>>> print message
This message will
span several
lines.
>>>
\end{pythonInteractive}
\end{minipage}


\section{Unit testing with \pythoninline{doctest}}
\label{sec-doctest}
It is a common best practice in software development these days to
include automatic \myidx{unit testing} of source code. Unit testing
provides a way to automatically verify that individual pieces of code,
such as functions, are working correctly. This makes it possible to
change the implementation of a function at a later time and quickly
test that it still does what it was intended to do.
Python has a built-in doctest module for easy unit testing. Doctests
can be written within a triple quoted string on the \emph{first line}
of the body of a function or script. They consist of sample
interpreter sessions with a series of inputs to a Python prompt
followed by the expected output from the Python interpreter.
The doctest module automatically runs any statement beginning with
$>$$>$$>$ (followed by a space) and compares the following line with the output from the
interpreter.
To see how this works, put the following in a script named
\myurl{first\_doctest.py}\footnote{There are four underscores in
  \pythoninline{\_\_name\_\_}, two at the start and two at the end.}:

\begin{minipage}{\textwidth}
\begin{python}
def is_divisible_by_2_or_5(n):
    """
    >>> is_divisible_by_2_or_5(8)
    True
    """


if __name__ == "__main__":
    import doctest
    doctest.testmod()
\end{python}
\end{minipage}

The last three lines are what make doctest run. Put them in the main routine
of any file that includes doctests. 
Running the script will produce the following output:

\begin{minipage}{\textwidth}
\begin{pythonOutput}
**********************************************************************
File "myfunctions.py", line 3, in __main__.is_divisible_by_2_or_5
Failed example:
    is_divisible_by_2_or_5(8)
Expected:
    True
Got nothing
**********************************************************************
1 items had failures:
   1 of   1 in __main__.is_divisible_by_2_or_5
***Test Failed*** 1 failures.
\end{pythonOutput}
\end{minipage}

This is an example of a \emph{failing test}. The test says: if you
call \pythoninline{is\_divisible\_by\_2\_or\_5(8)} the result should
be \pythoninline{True}. Since 
\pythoninline{is\_divisible\_by\_2\_or\_5} as written doesn't return
anything at all, the test fails, and doctest tells us that it expected
\pythoninline{True} but got nothing.

We can make this test pass by returning True:

\begin{minipage}{\textwidth}
\begin{python}
def is_divisible_by_2_or_5(n):
    """
    >>> is_divisible_by_2_or_5(8)
    True
    """
    return True


if __name__ == "__main__":
    import doctest
    doctest.testmod()
\end{python}
\end{minipage}

If we run it now, there will be no output, which indicates that the
test passed. Note again that the doctest string must be placed
immediately after the function definition header in order to run. To
see more detailed output, add the keyword argument
\pythoninline{verbose=True} to the \pythoninline{testmod} method call

\begin{python}
doctest.testmod(verbose=True)
\end{python}

and run the module again.  This will produce output showing the result
of running the tests and whether they pass or not.

\begin{minipage}{\textwidth}
\begin{pythonOutput}
Trying:
    is_divisible_by_2_or_5(8)
Expecting:
    True
ok
1 items had no tests:
    __main__
1 items passed all tests:
   1 tests in __main__.is_divisible_by_2_or_5
1 tests in 2 items.
1 passed and 0 failed.
Test passed.
\end{pythonOutput}
\end{minipage}

While the test passed, our test suite is clearly inadequate, since
\pythoninline{is\_divisible\_by\_2\_or\_5} will now return
\pythoninline{True} no matter what argument is passed to it. Here is a
completed version with a more complete test suite and code that makes
the tests pass:

\begin{minipage}{\textwidth}
\begin{python}
def is_divisible_by_2_or_5(n):
    """
    >>> is_divisible_by_2_or_5(8)
    True
    >>> is_divisible_by_2_or_5(7)
    False
    >>> is_divisible_by_2_or_5(5)
    True
    >>> is_divisible_by_2_or_5(9)
    False
    """
    return n % 2 == 0 or n % 5 == 0 


if __name__ == "__main__":
    import doctest
    doctest.testmod()
\end{python}
\end{minipage}
Run the module again and see what you get.

\section{Good Practice Development}
\label{sec-good-practice}
The difficult part of programming, ironically, is not the programming part. The difficult part is the problem solving part. That is, deciding how to solve the problem at hand. Most programming problems can be solved by splitting the problem into smaller problems until you get to problems you can solve. Then you join up those smaller solutions to solve the larger problem. Of course, it is not always obvious to know how to split up a large problem and this is where experience and practice plays a key role. Nevertheless, here are some things you can try when presented with a problem you have no idea how to solve.

Try splitting the problem into two sub-problems, and progressively split each sub-problem until you know how to solve something. For example:
\begin{enumerate}
\item Count the number of ``the''s in a string.
\begin{enumerate}
\item split the string into separate words
\item count the number of words that are equal to ``the''
\begin{enumerate}
\item test if a single word equals ``the''
\item scan a list of words and keep count of those that equal ``the''
\end{enumerate}
\end{enumerate}
\end{enumerate}

Even if you don't know how to solve a given sub-problem, and you can't think of how to split it further, at least you have a better chance of discovering how to solve a simpler problem (either by asking someone, or asking Google for example).

There are also simple rules to follow when you are developing your own test cases. You should include:
\begin{itemize}
\item typical input
\item atypical input (e.g. input at the extreme edge of allowable input)
\item incorrect or invalid input
\item the simplest type of input (if that makes sense)
\item the most complicated type of input (if that makes sense)
\item enough tests so that each line of code is exercised at least once in your test set (this is called test coverage).
\end{itemize}

\section{Test-driven development demonstrated}

At this point in the lecture, the lecturer will demonstrate how
test-driven development should be used. We will use the extension
exercise from Lecture \ref{lecture-fruity} as our example.


\section{Programming with style}
Readability is very important to programmers, since in practice
programs are read and modified far more often then they are
written. All the code examples in this book will be consistent with
the \emph{Python Enhancement Proposal 8} (PEP 8), a style guide
developed by the Python community.
We'll have more to say about style as our programs become more
complex, but a few pointers will be helpful already:
\begin{itemize}
\item use 4 spaces for indentation
\item imports should go at the top of the file
\item separate function definitions with two blank lines
\item keep function definitions together
\item keep top level statements, including function calls, together in the main routine
 of the program
\item use Test-driven development to develop your programs (this is not part of PEP 8, but is more a philosophy of software development).
\end{itemize}

\section{Glossary}
\begin{description}
\item[unit testing:]An automatic procedure used to validate that
  individual units of code are working properly. Python has doctest
  built in for this purpose.
\item[module:]A file containing Python definitions and statements
   intended for use in other Python programs. The contents of a module
   are made available to the other program by using the import
   statement.
\item[standard library:]A library is a collection of software used as
   tools in the development of other software. The standard library of
   a programming language is the set of such tools that are
   distributed with the core programming language. Python comes with
   an extensive standard library.
\item[import statement:]A statement which makes the objects contained
   in a module available for use. There are three forms for the import
   statement. Using a hypothetical module named \pythoninline{mymod} containing
   functions \pythoninline{f1} and \pythoninline{f2}, and variables
\pythoninline{v1} and \pythoninline{v2}, examples of these three 
   forms include:
\begin{python}
import mymod
\end{python}
and:
\begin{python}
from mymod import f1, f2, v1, v2
\end{python}
and:
\begin{python}
from mymod import *
\end{python}
\item[namespace:]A syntactic container providing a context for names
   so that the same name can reside in different namespaces without
   ambiguity. In Python, modules, classes, functions and methods all
   form namespaces.
\item[naming collision:]A situation in which two or more names in a
   given namespace cannot be unambiguously resolved. Using
\begin{python}
import mymodule
\end{python}
instead of
\begin{python}
from mymodule import *
\end{python}
prevents naming collisions.

\item[attribute:] A variable defined inside a module.  Module
  attributes are accessed by using the \myidx{dot operator} (.).
\item[dot operator:] The dot operator (.) permits access to
   attributes and functions of a module.
\item[docstring]A string constant on the first line of a function or
   module definition (and as we will see later, in class and method
   definitions as well). Docstrings provide a convenient way to
   associate documentation with code. Docstrings are also used by the
   doctest module for automated testing.
\end{description}

\newpage
\section{Laboratory exercises}
All of the exercises below should be added to a file named
\myurl{doctest\_ex.py} that contains the following in the main routine:

\begin{python}
if __name__ == "__main__":
    import doctest
    doctest.testmod(verbose=True)
\end{python}

After completing each exercise in turn, run the program to confirm
that the doctests for your new function pass.
\begin{enumerate}
\item Write a compare function that returns \pythoninline{1} if
\pythoninline{a $>$ b}, \pythoninline{0} if \pythoninline{a == b},
  and \pythoninline{-1} if \pythoninline{a $<$ b}.

\begin{python}
def compare(a, b):
    """
    Returns 1 if a>b, 0 if a equals b, and -1 if a<b
    >>> compare(5, 4)
    1
    >>> compare(7, 7)
    0
    >>> compare(2, 3)
    -1
    >>> compare(42, 1)
    1
    """
    #  Your function body should begin here.
\end{python}

Fill in the body of the function so the doctests pass.
\item Use incremental development to write a function called
hypotenuse that returns the length of the hypotenuse of a right
triangle given the lengths of the other two sides as parameters. Record
each stage of the incremental development process as you go.

\begin{python}
def hypotenuse(a, b):
    """
    Compute the hypotenuse of a right triangle with sides of length a
    and b.
    >>> hypotenuse(3, 4)
    5.0
    >>> hypotenuse(12, 5)
    13.0
    >>> hypotenuse(7, 24)
    25.0
    >>> hypotenuse(9, 12)
    15.0
    """
\end{python}

When you are finished add your completed function with the doctests to
\myurl{doctest\_ex.py} and confirm that the doctests pass.

\checkpoint

\item Write a body for the function definition of
\pythoninline{fahrenheit\_to\_celsius}
designed to return 
the integer value of the nearest degree Celsius for a given temperature
in Fahrenheit. Use your favourite web search engine to find the
equation for doing the conversion if you don't already know it.
(\emph{Hint:} you may want to make use of the built-in
function, \pythoninline{round}. Try typing \pythoninline{help(round)}
in a Python 
shell and experimenting with round until you are comfortable with how
it works.)

\begin{python}
def fahrenheit_to_celsius(t):
    """
    >>> fahrenheit_to_celsius(212)
    100
    >>> fahrenheit_to_celsius(32)
    0
    >>> fahrenheit_to_celsius(-40)
    -40
    >>> fahrenheit_to_celsius(36)
    2
    >>> fahrenheit_to_celsius(37)
    3
    >>> fahrenheit_to_celsius(38)
    3
    >>> fahrenheit_to_celsius(39)
    4
    """
\end{python}

\item Add a function body for \pythoninline{celsius\_to\_fahrenheit}
to convert from Celsius to Fahrenheit. 

\begin{python}
def celsius_to_fahrenheit(t):
    """
    >>> celsius_to_fahrenheit(0)
    32
    >>> celsius_to_fahrenheit(100)
    212
    >>> celsius_to_fahrenheit(-40)
    -40
    >>> celsius_to_fahrenheit(12)
    54
    >>> celsius_to_fahrenheit(18)
    64
    >>> celsius_to_fahrenheit(-48)
    -54
    """
\end{python}

\item \emph{Extension Exercise:} Write a function to convert kilograms
  to pounds. Devise a sensible test suite for your function.

\item \emph{Extension Exercise:} Write a function that takes the
  width, height and depth of a cuboid and returns its surface area. It
  should return -1 if an argument is negative. Devise a sensible test
  suite for your function. 

\begin{minipage}{\textwidth}
\item \emph{Extension Exercise:} Write a function that returns the
  volume of a square based pyramid. It should take as arguments the
  length of a bottom edge and the height. It should return -1 if
  either parameter is less than 0. Your function should pass the
  doctests below. 

\begin{python}
def vol_of_a_pyramid(edge, height):
    """
    >>> vol_of_a_pyramid(0, 3)
    0.0
    >>> vol_of_a_pyramid(3, 0)
    0.0
    >>> vol_of_a_pyramid(2, 3)
    4.0
    >>> vol_of_a_pyramid(4, 3)
    16.0
    >>> vol_of_a_pyramid(6, 4)
    48.0
    >>> vol_of_a_pyramid(5, 4.5)
    37.5
    >>> vol_of_a_pyramid(-2, 4)
    -1
    >>> vol_of_a_pyramid(2, -4)
    -1
    """
\end{python}
\end{minipage}

\end{enumerate}

\submitreminder