1
<?xml version="1.0"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
2
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml" lang="en"><head><title>Twisted Documentation: Unit Tests in Twisted</title><link href="../../howto/stylesheet.css" type="text/css" rel="stylesheet" /></head><body bgcolor="white"><h1 class="title">Unit Tests in Twisted</h1><div class="toc"><ol><li><a href="#auto0">Unit Tests in the Twisted Philosophy</a></li><li><a href="#auto1">What to Test, What Not to Test</a></li><li><a href="#auto2">Running the Tests</a></li><ul><li><a href="#auto3">How</a></li><li><a href="#auto4">When</a></li></ul><li><a href="#auto5">Adding a Test</a></li><li><a href="#auto6">Skipping tests, TODO items</a></li><ul><li><a href="#auto7">.todo and Testing New Functionality </a></li></ul><li><a href="#auto8">Associating Test Cases With Source Files</a></li><li><a href="#auto9">Links</a></li></ol></div><div class="content"><span></span><p>Each <em>unit test</em> tests one bit of functionality in the
3
software. Unit tests are entirely automated and complete quickly.
4
Unit tests for the entire system are gathered into one test suite,
5
and may all be run in a single batch. The result of a unit test
6
is simple: either it passes, or it doesn't. All this means you
7
can test the entire system at any time without inconvenience, and
8
quickly see what passes and what fails.</p><h2>Unit Tests in the Twisted Philosophy<a name="auto0"></a></h2><p>The Twisted development team
9
adheres to the practice of <a href="http://c2.com/cgi/wiki?ExtremeProgramming">Extreme
10
Programming</a> (XP), and the usage of unit tests is a cornerstone
11
XP practice. Unit tests are a tool to give you increased
12
confidence. You changed an algorithm -- did you break something?
13
Run the unit tests. If a test fails, you know where to look,
14
because each test covers only a small amount of code, and you know
15
it has something to do with the changes you just made. If all the
16
tests pass, you're good to go, and you don't need to second-guess
17
yourself or worry that you just accidently broke someone else's
18
program.</p><h2>What to Test, What Not to Test<a name="auto1"></a></h2><blockquote><p>You don't have to write a test for every single
19
method you write, only production methods that could possibly break.</p></blockquote><p>-- Kent Beck, <cite>Extreme Programming Explained</cite>, p. 58.</p><h2>Running the Tests<a name="auto2"></a></h2><h3>How<a name="auto3"></a></h3><p>From the root of the Twisted source tree, run:</p><pre class="shell">
20
$ bin/trial -R twisted
21
</pre><p>You'll find that having something like this in your emacs init
22
files is quite handy:</p><pre class="elisp">
23
(defun runtests () (interactive)
24
(compile "python /somepath/Twisted/bin/trial -R /somepath/Twisted"))
26
(global-set-key [(alt t)] 'runtests)
27
</pre><h3>When<a name="auto4"></a></h3><p>Always always <em>always</em> be sure <a href="http://www.xprogramming.com/xpmag/expUnitTestsAt100.htm">all the
28
tests pass</a> before committing any code. If someone else
29
checks out code at the start of a development session and finds
30
failing tests, they will not be happy and may decide to <em>hunt
31
you down</em>.</p><p>Since this is a geographically dispersed team, the person who
32
can help you get your code working probably isn't in the room with
33
you. You may want to share your work in progress over the
34
network, but you want to leave the main Subversion tree in good working
35
order. So <a href="http://svnbook.red-bean.com/svnbook/ch04.html">use a branch</a>,
36
and merge your changes back in only after your problem is solved
37
and all the unit tests pass again.</p><h2>Adding a Test<a name="auto5"></a></h2><p>Please don't add new modules to Twisted without adding tests
38
for them too. Otherwise we could change something which breaks
39
your module and not find out until later, making it hard to know
40
exactly what the change that broke it was, or until after a
41
release, and nobody wants broken code in a release.</p><p>Tests go in Twisted/twisted/test/, and are named <code>test_foo.py</code>,
42
where <code>foo</code> is the name of the module or package being tested.
43
Extensive documentation on using the PyUnit framework for writing
44
unit tests can be found in the <a href="#links">links
45
section</a> below.</p><p>One deviation from the standard PyUnit documentation: To ensure
46
that any variations in test results are due to variations in the
47
code or environment and not the test process itself, Twisted ships
48
with its own, compatible, testing framework. That just
49
means that when you import the unittest module, you will <code class="python">from twisted.trial import unittest</code> instead of the
50
standard <code class="python">import unittest</code>.</p><p>As long as you have followed the module naming and placement
51
conventions, <code class="shell">trial</code> will be smart
52
enough to pick up any new tests you write.</p><h2>Skipping tests, TODO items<a name="auto6"></a></h2><p>Trial, the Twisted unit test framework, has some extensions which are
53
designed to encourage developers to add new tests. One common situation is
54
that a test exercises some optional functionality: maybe it depends upon
55
certain external libraries being available, maybe it only works on certain
56
operating systems. The important common factor is that nobody considers
57
these limitations to be a bug.</p><p>To make it easy to test as much as possible, some tests may be skipped in
58
certain situations. Individual test cases can raise the
59
<code>SkipTest</code> exception to indicate that they should be skipped, and
60
the remainder of the test is not run. In the summary (the very last thing
61
printed, at the bottom of the test output) the test is counted as a
62
<q>skip</q> instead of a <q>success</q> or <q>fail</q>. This should be used
63
inside a conditional which looks for the necessary prerequisites:</p><pre class="python">
64
<span class="py-src-keyword">def</span> <span class="py-src-identifier">testSSHClient</span>(<span class="py-src-parameter">self</span>):
65
<span class="py-src-keyword">if</span> <span class="py-src-keyword">not</span> <span class="py-src-variable">ssh_path</span>:
66
<span class="py-src-keyword">raise</span> <span class="py-src-variable">unittest</span>.<span class="py-src-variable">SkipTest</span>, <span class="py-src-string">"cannot find ssh, nothing to test"</span>
67
<span class="py-src-variable">foo</span>() <span class="py-src-comment"># do actual test after the SkipTest</span>
68
</pre><p>You can also set the <code>.skip</code> attribute on the method, with a string to
69
indicate why the test is being skipped. This is convenient for temporarily
70
turning off a test case, but it can also be set conditionally (by
71
manipulating the class attributes after they've been defined):</p><pre class="python">
72
<span class="py-src-keyword">def</span> <span class="py-src-identifier">testThing</span>(<span class="py-src-parameter">self</span>):
73
<span class="py-src-variable">dotest</span>()
74
<span class="py-src-variable">testThing</span>.<span class="py-src-variable">skip</span> = <span class="py-src-string">"disabled locally"</span>
75
</pre><pre class="python">
76
<span class="py-src-keyword">class</span> <span class="py-src-identifier">MyTestCase</span>(<span class="py-src-parameter">unittest</span>.<span class="py-src-parameter">TestCase</span>):
77
<span class="py-src-keyword">def</span> <span class="py-src-identifier">testOne</span>(<span class="py-src-parameter">self</span>):
79
<span class="py-src-keyword">def</span> <span class="py-src-identifier">testThing</span>(<span class="py-src-parameter">self</span>):
80
<span class="py-src-variable">dotest</span>()
82
<span class="py-src-keyword">if</span> <span class="py-src-keyword">not</span> <span class="py-src-variable">haveThing</span>:
83
<span class="py-src-variable">MyTestCase</span>.<span class="py-src-variable">testThing</span>.<span class="py-src-variable">im_func</span>.<span class="py-src-variable">skip</span> = <span class="py-src-string">"cannot test without Thing"</span>
84
<span class="py-src-comment"># but testOne() will still run
85
</span></pre><p>Finally, you can turn off an entire TestCase at once by setting the .skip
86
attribute on the class. If you organize your tests by the functionality they
87
depend upon, this is a convenient way to disable just the tests which cannot
88
be run.</p><pre class="python">
89
<span class="py-src-keyword">class</span> <span class="py-src-identifier">SSLTestCase</span>(<span class="py-src-parameter">unittest</span>.<span class="py-src-parameter">TestCase</span>):
91
<span class="py-src-keyword">class</span> <span class="py-src-identifier">TCPTestCase</span>(<span class="py-src-parameter">unittest</span>.<span class="py-src-parameter">TestCase</span>):
94
<span class="py-src-keyword">if</span> <span class="py-src-keyword">not</span> <span class="py-src-variable">haveSSL</span>:
95
<span class="py-src-variable">SSLTestCase</span>.<span class="py-src-variable">skip</span> = <span class="py-src-string">"cannot test without SSL support"</span>
96
<span class="py-src-comment"># but TCPTestCase will still run
97
</span></pre><h3>.todo and Testing New Functionality <a name="auto7"></a></h3><p>Two good practices which arise from the <q>XP</q> development process are
98
sometimes at odds with each other:</p><ul><li>Unit tests are a good thing. Good developers recoil in horror when
99
they see a failing unit test. They should drop everything until the test
100
has been fixed.</li><li>Good developers write the unit tests first. Once tests are done, they
101
write implementation code until the unit tests pass. Then they stop.</li></ul><p>These two goals will sometimes conflict. The unit tests that are written
102
first, before any implementation has been done, are certain to fail. We want
103
developers to commit their code frequently, for reliability and to improve
104
coordination between multiple people working on the same problem together.
105
While the code is being written, other developers (those not involved in the
106
new feature) should not have to pay attention to failures in the new code.
107
We should not dilute our well-indoctrinated Failing Test Horror Syndrome by
108
crying wolf when an incomplete module has not yet started passing its unit
109
tests. To do so would either teach the module author to put off writing or
110
committing their unit tests until <em>after</em> all the functionality is
111
working, or it would teach the other developers to ignore failing test
112
cases. Both are bad things.</p><p><q>.todo</q> is intended to solve this problem. When a developer first
113
starts writing the unit tests for functionality that has not yet been
114
implemented, they can set the <code>.todo</code> attribute on the test
115
methods that are expected to fail. These methods will still be run, but
116
their failure will not be counted the same as normal failures: they will go
117
into an <q>expected failures</q> category. Developers should learn to treat
118
this category as a second-priority queue, behind actual test failures.</p><p>As the developer implements the feature, the tests will eventually start
119
passing. This is surprising: after all those tests are marked as being
120
expected to fail. The .todo tests which nevertheless pass are put into a
121
<q>unexpected success</q> category. The developer should remove the .todo
122
tag from these tests. At that point, they become normal tests, and their
123
failure is once again cause for immediate action by the entire development
124
team.</p><p>The life cycle of a test is thus:</p><ol><li>Test is created, marked <code>.todo</code>. Test fails: <q>expected
125
failure</q>.</li><li>Code is written, test starts to pass. <q>unexpected success</q>.</li><li><code>.todo</code> tag is removed. Test passes. <q>success</q>.</li><li>Code is broken, test starts to fail. <q>failure</q>. Developers spring
126
into action.</li><li>Code is fixed, test passes once more. <q>success</q>.</li></ol><p>Any test which remains marked with <code>.todo</code> for too long should
127
be examined. Either it represents functionality which nobody is working on,
128
or the test is broken in some fashion and needs to be fixed.</p><h2>Associating Test Cases With Source Files<a name="auto8"></a></h2><p>Please add a <code>test-case-name</code> tag to the source file that is
129
covered by your new test. This is a comment at the beginning of the file
130
which looks like one of the following:</p><pre class="python">
131
<span class="py-src-comment"># -*- test-case-name: twisted.test.test_defer -*-
132
</span></pre><p>or</p><pre class="python">
133
<span class="py-src-comment">#!/usr/bin/python
134
</span><span class="py-src-comment"># -*- test-case-name: twisted.test.test_defer -*-
135
</span></pre><p>This format is understood by emacs to mark <q>File Variables</q>. The
136
intention is to accept <code>test-case-name</code> anywhere emacs would on
137
the first or second line of the file (but not in the <code>File
138
Variables:</code> block that emacs accepts at the end of the file). If you
139
need to define other emacs file variables, you can either put them in the
140
<code>File Variables:</code> block or use a semicolon-separated list of
141
variable definitions:</p><pre class="python">
142
<span class="py-src-comment"># -*- test-case-name: twisted.test.test_defer; fill-column: 75; -*-
143
</span></pre><p>If the code is exercised by multiple test cases, those may be marked by
144
using a comma-separated list of tests, as follows: (NOTE: not all tools can
145
handle this yet.. <code>trial --testmodule</code> does, though)</p><pre class="python">
146
<span class="py-src-comment"># -*- test-case-name: twisted.test.test_defer,twisted.test.test_tcp -*-
147
</span></pre><p>The <code>test-case-name</code> tag will allow <code class="shell">trial
148
--testmodule twisted/dir/myfile.py</code> to determine which test cases need
149
to be run to exercise the code in <code>myfile.py</code>. Several tools (as
150
well as <code>twisted-dev.el</code>'s F9 command) use this to automatically
151
run the right tests.</p><h2 id="links">Links<a name="auto9"></a></h2><a name="links"></a><ul><li>A chapter on <a href="http://diveintopython.org/roman_divein.html">Unit Testing</a>
152
in Mark Pilgrim's <a href="http://diveintopython.org">Dive Into
153
Python</a>.</li><li><a href="http://www.python.org/doc/current/lib/module-unittest.html"><code>unittest</code></a> module documentation, in the <a href="http://www.python.org/doc/current/lib/">Python Library
154
Reference</a>.</li><li><a href="http://c2.com/cgi/wiki?UnitTests">UnitTests</a> on
155
the <a href="http://c2.com/cgi/wiki">PortlandPatternRepository
156
Wiki</a>, where all the cool <a href="http://c2.com/cgi/wiki?ExtremeProgramming">ExtremeProgramming</a> kids hang out.</li><li><a href="http://www.extremeprogramming.org/rules/unittests.html">Unit
157
Tests</a> in <a href="http://www.extremeprogramming.org">Extreme Programming: A Gentle Introduction</a>.</li><li>Ron Jeffries espouses on the importance of <a href="http://www.xprogramming.com/xpmag/expUnitTestsAt100.htm">Unit
158
Tests at 100%</a>.</li><li>Ron Jeffries writes about the <a href="http://www.xprogramming.com/Practices/PracUnitTest.html">Unit
159
Test</a> in the <a href="http://www.xprogramming.com/Practices/xpractices.htm">Extreme
160
Programming practices of C3</a>.</li><li><a href="http://pyunit.sourceforge.net">PyUnit's homepage</a>.</li><li>The <a href="http://svn.twistedmatrix.com/cvs/trunk/twisted/?root=Twisted">twisted/test directory</a> in Subversion.</li></ul><p>See also <a href="../testing.html">Tips for writing tests for Twisted
161
code</a>.</p></div><p><a href="../../howto/index.html">Index</a></p><span class="version">Version: 2.4.0</span></body></html>
b'\\ No newline at end of file'