~mccane/practical-programming/trunk

\chapter{Modules and TDD}

\label{lec-tdd}

This chapter primarily deals with a development methodology called Test Driven Development (TDD). Before we get to TDD though, we need a couple of extra parts of python: modules and triple quoted strings.

\section{Modules, files and the \pythoninline{import} statement}

A \myidx{module} is simply a Python file that contains

definitions (usually function definitions). As programs get large, it

makes sense to separate the program into different modules. This

allows the programmer to minimize the complexity of the programming

task. Typically, similar functionality or related functions are

grouped into a single module and separate modules are as independent

of each other as possible. Creating your own modules is as easy as

creating a Python script with definitions in it; in fact you have already 

written lots of Python modules.

The Python standard library contains a very large number of modules

for many different tasks. It is worthwhile browsing the library just

to get a feel for all the useful things it can do. The current

documentation can be accessed at:

\myurl{http://www.python.org/doc/current/library/index.html}, or by

choosing \menu{Help \rarr Python Docs ...} in IDLE. We will

look at the \pythoninline{doctest} module later in this lecture, and

throughout the course several other modules will be introduced.

There are three ways of using \pythoninline{import} to import a module

and subsequently use what was imported. The \pythoninline{math} module contains many 

useful math functions, we will use it to illustrate:

\begin{enumerate}

\item

\begin{python}

import math

print math.cos(math.pi/2.0)

\end{python}

This statement imports every function from the \pythoninline{math} module which

 are then accessed using dot notation.

\item

\begin{python}

from math import cos, pi

print cos(pi/2.0)

\end{python}

This statement imports only the definitions of \pythoninline{cos} and

\pythoninline{pi} from the math library. Nothing else is imported.

\item

\begin{python}

from math import *

print cos(pi/2.0)

\end{python}

This statement also imports everything from the \pythoninline{math}

module, the difference being that dot notation is not needed to access

module members.

\end{enumerate}

%Also, do you need/want to include the Restart between each example, otherwise the previous import is still imported.

It is more common to use the first or second alternative than the last.

The first has the advantage of avoiding naming conflicts (two

different modules defining \pythoninline{cos} for example), at the

cost of lengthier function calls. The second has the

opposite advantages and disadvantages.

We can use the \pythoninline{help} function to see the functions and

data contained within modules. The keyword module

contains a single function, \pythoninline{iskeyword}, which as its

name suggests is a boolean function that returns \pythoninline{True}

if a string passed to it is a keyword:

\begin{pythonInteractive}

>>> from keyword import *

>>> iskeyword("for")

True

>>> iskeyword("all")

False

>>>

\end{pythonInteractive}

The data item, \pythoninline{kwlist} contains a list of all the

current keywords in Python:

\begin{pythonInteractive}

>>> from keyword import *

>>> print kwlist

['and', 'as', 'assert', 'break', 'class', 'continue', 'def', 'del', 'elif',

'else', 'except', 'exec', 'finally', 'for', 'from', 'global', 'if', 'import',

'in', 'is', 'lambda', 'not', 'or', 'pass', 'print', 'raise', 'return', 'try',

'while', 'with', 'yield']

>>>

\end{pythonInteractive}

We encourage you to check out \url{http://docs.python.org/library/}

to explore the extensive libraries that 

come with Python. There are so many treasures to discover!

\section{Triple quoted strings}

In addition to the single and double quoted strings we first saw in

Lecture \ref{lec-variables}, Python also has \emph{triple quoted strings}. We will need triple quoted strings for Unit Testing (next section). Here are some examples:

\begin{minipage}{\textwidth}

\begin{pythonInteractive}

>>> type("""This is a triple quoted string using 3 double quotes.""")

<type 'str'>

>>> type('''This triple quoted strings uses 3 single quotes.''')

<type 'str'>

>>>

\end{pythonInteractive}

\end{minipage}

Triple quoted strings can contain both single and double quotes inside

them:

\begin{minipage}{\textwidth}

\begin{pythonInteractive}

>>> print '''"Oh no", she exclaimed, "Ben's bike is broken!"'''

"Oh no", she exclaimed, "Ben's bike is broken!"

>>>

\end{pythonInteractive}

\end{minipage}

Finally, triple quoted strings can span multiple lines:

\noindent

\begin{minipage}{\textwidth}

\begin{pythonInteractive}

>>> message = """This message will

... span several

... lines."""

>>> print message

This message will

span several

lines.

>>>

\end{pythonInteractive}

\end{minipage}

\section{Unit testing with \pythoninline{doctest}}

\label{sec-doctest}

It is a common best practice in software development these days to

include automatic \myidx{unit testing} of source code. Unit testing

provides a way to automatically verify that individual pieces of code,

such as functions, are working correctly. This makes it possible to

change the implementation of a function at a later time and quickly

test that it still does what it was intended to do.

Python has a built-in doctest module for easy unit testing. Doctests

can be written within a triple quoted string on the \emph{first line}

of the body of a function or script. They consist of sample

interpreter sessions with a series of inputs to a Python prompt

followed by the expected output from the Python interpreter.

The doctest module automatically runs any statement beginning with

$>$$>$$>$ (followed by a space) and compares the following line with the output from the

interpreter.

To see how this works, put the following in a script named

\myurl{first\_doctest.py}\footnote{There are four underscores in

  \pythoninline{\_\_name\_\_}, two at the start and two at the end.}:

\begin{minipage}{\textwidth}

\begin{python}

def is_divisible_by_2_or_5(n):

    """

    >>> is_divisible_by_2_or_5(8)

    True

    """

if __name__ == "__main__":

    import doctest

    doctest.testmod()

\end{python}

\end{minipage}

The last three lines are what make doctest run. Put them in the main routine

of any file that includes doctests. 

Running the script will produce the following output:

\begin{minipage}{\textwidth}

\begin{pythonOutput}

**********************************************************************

File "myfunctions.py", line 3, in __main__.is_divisible_by_2_or_5

Failed example:

    is_divisible_by_2_or_5(8)

Expected:

    True

Got nothing

**********************************************************************

1 items had failures:

   1 of   1 in __main__.is_divisible_by_2_or_5

***Test Failed*** 1 failures.

\end{pythonOutput}

\end{minipage}

This is an example of a \emph{failing test}. The test says: if you

call \pythoninline{is\_divisible\_by\_2\_or\_5(8)} the result should

be \pythoninline{True}. Since 

\pythoninline{is\_divisible\_by\_2\_or\_5} as written doesn't return

anything at all, the test fails, and doctest tells us that it expected

\pythoninline{True} but got nothing.

We can make this test pass by returning True:

\begin{minipage}{\textwidth}

\begin{python}

def is_divisible_by_2_or_5(n):

    """

    >>> is_divisible_by_2_or_5(8)

    True

    """

    return True

if __name__ == "__main__":

    import doctest

    doctest.testmod()

\end{python}

\end{minipage}

If we run it now, there will be no output, which indicates that the

test passed. Note again that the doctest string must be placed

immediately after the function definition header in order to run. To

see more detailed output, add the keyword argument

\pythoninline{verbose=True} to the \pythoninline{testmod} method call

\begin{python}

doctest.testmod(verbose=True)

\end{python}

and run the module again.  This will produce output showing the result

of running the tests and whether they pass or not.

\begin{minipage}{\textwidth}

\begin{pythonOutput}

Trying:

    is_divisible_by_2_or_5(8)

Expecting:

    True

ok

1 items had no tests:

    __main__

1 items passed all tests:

   1 tests in __main__.is_divisible_by_2_or_5

1 tests in 2 items.

1 passed and 0 failed.

Test passed.

\end{pythonOutput}

\end{minipage}

While the test passed, our test suite is clearly inadequate, since

\pythoninline{is\_divisible\_by\_2\_or\_5} will now return

\pythoninline{True} no matter what argument is passed to it. Here is a

completed version with a more complete test suite and code that makes

the tests pass:

\begin{minipage}{\textwidth}

\begin{python}

def is_divisible_by_2_or_5(n):

    """

    >>> is_divisible_by_2_or_5(8)

    True

    >>> is_divisible_by_2_or_5(7)

    False

    >>> is_divisible_by_2_or_5(5)

    True

    >>> is_divisible_by_2_or_5(9)

    False

    """

    return n % 2 == 0 or n % 5 == 0 

if __name__ == "__main__":

    import doctest

    doctest.testmod()

\end{python}

\end{minipage}

Run the module again and see what you get.

\section{Good Practice Development}

\label{sec-good-practice}

The difficult part of programming, ironically, is not the programming part. The difficult part is the problem solving part. That is, deciding how to solve the problem at hand. Most programming problems can be solved by splitting the problem into smaller problems until you get to problems you can solve. Then you join up those smaller solutions to solve the larger problem. Of course, it is not always obvious to know how to split up a large problem and this is where experience and practice plays a key role. Nevertheless, here are some things you can try when presented with a problem you have no idea how to solve.

Try splitting the problem into two sub-problems, and progressively split each sub-problem until you know how to solve something. For example:

\begin{enumerate}

\item Count the number of ``the''s in a string.

\begin{enumerate}

\item split the string into separate words

\item count the number of words that are equal to ``the''

\begin{enumerate}

\item test if a single word equals ``the''

\item scan a list of words and keep count of those that equal ``the''

\end{enumerate}

\end{enumerate}

\end{enumerate}

Even if you don't know how to solve a given sub-problem, and you can't think of how to split it further, at least you have a better chance of discovering how to solve a simpler problem (either by asking someone, or asking Google for example).

There are also simple rules to follow when you are developing your own test cases. You should include:

\begin{itemize}

\item typical input

\item atypical input (e.g. input at the extreme edge of allowable input)

\item incorrect or invalid input

\item the simplest type of input (if that makes sense)

\item the most complicated type of input (if that makes sense)

\item enough tests so that each line of code is exercised at least once in your test set (this is called test coverage).

\end{itemize}

\section{Test-driven development demonstrated}

At this point in the lecture, the lecturer will demonstrate how

test-driven development should be used. We will use the extension

exercise from Lecture \ref{lecture-fruity} as our example.

\section{Programming with style}

Readability is very important to programmers, since in practice

programs are read and modified far more often then they are

written. All the code examples in this book will be consistent with

the \emph{Python Enhancement Proposal 8} (PEP 8), a style guide

developed by the Python community.

We'll have more to say about style as our programs become more

complex, but a few pointers will be helpful already:

\begin{itemize}

\item use 4 spaces for indentation

\item imports should go at the top of the file

\item separate function definitions with two blank lines

\item keep function definitions together

\item keep top level statements, including function calls, together in the main routine

 of the program

\item use Test-driven development to develop your programs (this is not part of PEP 8, but is more a philosophy of software development).

\end{itemize}

\section{Glossary}

\begin{description}

\item[unit testing:]An automatic procedure used to validate that

  individual units of code are working properly. Python has doctest

  built in for this purpose.

\item[module:]A file containing Python definitions and statements

   intended for use in other Python programs. The contents of a module

   are made available to the other program by using the import

   statement.

\item[standard library:]A library is a collection of software used as

   tools in the development of other software. The standard library of

   a programming language is the set of such tools that are

   distributed with the core programming language. Python comes with

   an extensive standard library.

\item[import statement:]A statement which makes the objects contained

   in a module available for use. There are three forms for the import

   statement. Using a hypothetical module named \pythoninline{mymod} containing

   functions \pythoninline{f1} and \pythoninline{f2}, and variables

\pythoninline{v1} and \pythoninline{v2}, examples of these three 

   forms include:

\begin{python}

import mymod

\end{python}

and:

\begin{python}

from mymod import f1, f2, v1, v2

\end{python}

and:

\begin{python}

from mymod import *

\end{python}

\item[namespace:]A syntactic container providing a context for names

   so that the same name can reside in different namespaces without

   ambiguity. In Python, modules, classes, functions and methods all

   form namespaces.

\item[naming collision:]A situation in which two or more names in a

   given namespace cannot be unambiguously resolved. Using

\begin{python}

import mymodule

\end{python}

instead of

\begin{python}

from mymodule import *

\end{python}

prevents naming collisions.

\item[attribute:] A variable defined inside a module.  Module

  attributes are accessed by using the \myidx{dot operator} (.).

\item[dot operator:] The dot operator (.) permits access to

   attributes and functions of a module.

\item[docstring]A string constant on the first line of a function or

   module definition (and as we will see later, in class and method

   definitions as well). Docstrings provide a convenient way to

   associate documentation with code. Docstrings are also used by the

   doctest module for automated testing.

\end{description}

\newpage

\section{Laboratory exercises}

All of the exercises below should be added to a file named

\myurl{doctest\_ex.py} that contains the following in the main routine:

\begin{python}

if __name__ == "__main__":

    import doctest

    doctest.testmod(verbose=True)

\end{python}

After completing each exercise in turn, run the program to confirm

that the doctests for your new function pass.

\begin{enumerate}

\item Write a compare function that returns \pythoninline{1} if

\pythoninline{a $>$ b}, \pythoninline{0} if \pythoninline{a == b},

  and \pythoninline{-1} if \pythoninline{a $<$ b}.

\begin{python}

def compare(a, b):

    """

    Returns 1 if a>b, 0 if a equals b, and -1 if a<b

    >>> compare(5, 4)

    1

    >>> compare(7, 7)

    0

    >>> compare(2, 3)

    -1

    >>> compare(42, 1)

    1

    """

    #  Your function body should begin here.

\end{python}

Fill in the body of the function so the doctests pass.

\item Use incremental development to write a function called

hypotenuse that returns the length of the hypotenuse of a right

triangle given the lengths of the other two sides as parameters. Record

each stage of the incremental development process as you go.

\begin{python}

def hypotenuse(a, b):

    """

    Compute the hypotenuse of a right triangle with sides of length a

    and b.

    >>> hypotenuse(3, 4)

    5.0

    >>> hypotenuse(12, 5)

    13.0

    >>> hypotenuse(7, 24)

    25.0

    >>> hypotenuse(9, 12)

    15.0

    """

\end{python}

When you are finished add your completed function with the doctests to

\myurl{doctest\_ex.py} and confirm that the doctests pass.

\checkpoint

\item Write a body for the function definition of

\pythoninline{fahrenheit\_to\_celsius}

designed to return 

the integer value of the nearest degree Celsius for a given temperature

in Fahrenheit. Use your favourite web search engine to find the

equation for doing the conversion if you don't already know it.

(\emph{Hint:} you may want to make use of the built-in

function, \pythoninline{round}. Try typing \pythoninline{help(round)}

in a Python 

shell and experimenting with round until you are comfortable with how

it works.)

\begin{python}

def fahrenheit_to_celsius(t):

    """

    >>> fahrenheit_to_celsius(212)

    100

    >>> fahrenheit_to_celsius(32)

    0

    >>> fahrenheit_to_celsius(-40)

    -40

    >>> fahrenheit_to_celsius(36)

    2

    >>> fahrenheit_to_celsius(37)

    3

    >>> fahrenheit_to_celsius(38)

    3

    >>> fahrenheit_to_celsius(39)

    4

    """

\end{python}

\item Add a function body for \pythoninline{celsius\_to\_fahrenheit}

to convert from Celsius to Fahrenheit. 

\begin{python}

def celsius_to_fahrenheit(t):

    """

    >>> celsius_to_fahrenheit(0)

    32

    >>> celsius_to_fahrenheit(100)

    212

    >>> celsius_to_fahrenheit(-40)

    -40

    >>> celsius_to_fahrenheit(12)

    54

    >>> celsius_to_fahrenheit(18)

    64

    >>> celsius_to_fahrenheit(-48)

    -54

    """

\end{python}

\item \emph{Extension Exercise:} Write a function to convert kilograms

  to pounds. Devise a sensible test suite for your function.

\item \emph{Extension Exercise:} Write a function that takes the

  width, height and depth of a cuboid and returns its surface area. It

  should return -1 if an argument is negative. Devise a sensible test

  suite for your function. 

\begin{minipage}{\textwidth}

\item \emph{Extension Exercise:} Write a function that returns the

  volume of a square based pyramid. It should take as arguments the

  length of a bottom edge and the height. It should return -1 if

  either parameter is less than 0. Your function should pass the

  doctests below. 

\begin{python}

def vol_of_a_pyramid(edge, height):

    """

    >>> vol_of_a_pyramid(0, 3)

    0.0

    >>> vol_of_a_pyramid(3, 0)

    0.0

    >>> vol_of_a_pyramid(2, 3)

    4.0

    >>> vol_of_a_pyramid(4, 3)

    16.0

    >>> vol_of_a_pyramid(6, 4)

    48.0

    >>> vol_of_a_pyramid(5, 4.5)

    37.5

    >>> vol_of_a_pyramid(-2, 4)

    -1

    >>> vol_of_a_pyramid(2, -4)

    -1

    """

\end{python}

\end{minipage}

\end{enumerate}

\submitreminder