7
:Revision: $Revision: 4163 $
8
:Date: $Date: 2005-12-09 05:21:34 +0100 (Fri, 09 Dec 2005) $
9
:Copyright: This document has been placed in the public domain.
11
.. _Docutils: http://docutils.sourceforge.net/
15
When adding new functionality (or fixing bugs), be sure to add test
16
cases to the test suite. Practise test-first programming; it's fun,
17
it's addictive, and it works!
19
This document describes how to run the Docutils test suite, how the
20
tests are organized and how to add new tests or modify existing tests.
23
Running the Test Suite
24
======================
26
Before checking in any changes, run the entire Docutils test suite to
27
be sure that you haven't broken anything. From a shell::
36
The Docutils 0.4 release supports Python 2.1 [#py21]_ or later, with
37
some features only working (and being tested) with Python 2.3+.
38
Therefore, you should actually have Pythons 2.1 [#py21]_, 2.2, 2.3, as
39
well as the latest Python installed and always run the tests on all of
40
them. (A good way to do that is to always run the test suite through
41
a short script that runs ``alltests.py`` under each version of
42
Python.) If you can't afford intalling 3 or more Python versions, the
43
edge cases (2.1 and 2.3) should cover most of it.
45
.. [#py21] Python 2.1 may be used providing the compiler package is
46
installed. The compiler package can be found in the Tools/
47
directory of Python 2.1's source distribution.
49
Good resources covering the differences between Python versions:
51
* `What's New in Python 2.2`__
52
* `What's New in Python 2.3`__
53
* `What's New in Python 2.4`__
54
* `PEP 290 - Code Migration and Modernization`__
56
__ http://www.python.org/doc/2.2.3/whatsnew/whatsnew22.html
57
__ http://www.python.org/doc/2.3.5/whatsnew/whatsnew23.html
58
__ http://www.python.org/doc/2.4.1/whatsnew/whatsnew24.html
59
__ http://www.python.org/peps/pep-0290.html
61
.. _Python Check-in Policies: http://www.python.org/dev/tools.html
62
.. _sandbox directory:
63
http://svn.berlios.de/viewcvs/docutils/trunk/sandbox/
64
.. _nightly repository tarball:
65
http://svn.berlios.de/svndumps/docutils-repos.gz
71
Unit tests test single functions or modules (i.e. whitebox testing).
73
If you are implementing a new feature, be sure to write a test case
74
covering its functionality. It happens very frequently that your
75
implementation (or even only a part of it) doesn't work with an older
76
(or even newer) Python version, and the only reliable way to detect
77
those cases is using tests.
79
Often, it's easier to write the test first and then implement the
80
functionality required to make the test pass.
86
When writing new tests, it very often helps to see how a similar test
87
is implemented. For example, the files in the
88
``test_parsers/test_rst/`` directory all look very similar. So when
89
adding a test, you don't have to reinvent the wheel.
91
If there is no similar test, you can write a new test from scratch
92
using Python's ``unittest`` module. For an example, please have a
93
look at the following imaginary ``test_square.py``::
95
#! /usr/bin/env python
98
# Contact: your email address
99
# Revision: $Revision: 4163 $
100
# Date: $Date: 2005-12-09 05:21:34 +0100 (Fri, 09 Dec 2005) $
101
# Copyright: This module has been placed in the public domain.
104
Test module for docutils.square.
108
import docutils.square
111
class SquareTest(unittest.TestCase):
113
def test_square(self):
114
self.assertEqual(docutils.square.square(0), 0)
115
self.assertEqual(docutils.square.square(5), 25)
116
self.assertEqual(docutils.square.square(7), 49)
118
def test_square_root(self):
119
self.assertEqual(docutils.square.sqrt(49), 7)
120
self.assertEqual(docutils.square.sqrt(0), 0)
121
self.assertRaises(docutils.square.SquareRootError,
122
docutils.square.sqrt, 20)
125
if __name__ == '__main__':
128
For more details on how to write tests, please refer to the
129
documentation of the ``unittest`` module.
137
The directory ``test/functional/`` contains data for functional tests.
139
Performing functional testing means testing the Docutils system as a
140
whole (i.e. blackbox testing).
146
+ ``functional/`` The main data directory.
148
+ ``input/`` The input files.
150
- ``some_test.txt``, for example.
152
+ ``output/`` The actual output.
154
- ``some_test.html``, for example.
156
+ ``expected/`` The expected output.
158
- ``some_test.html``, for example.
160
+ ``tests/`` The config files for processing the input files.
162
- ``some_test.py``, for example.
164
- ``_default.py``, the `default configuration file`_.
170
When running ``test_functional.py``, all config files in
171
``functional/tests/`` are processed. (Config files whose names begin
172
with an underscore are ignored.) The current working directory is
173
always Docutils' main test directory (``test/``).
175
For example, ``functional/tests/some_test.py`` could read like this::
177
# Source and destination file names.
178
test_source = "some_test.txt"
179
test_destination = "some_test.html"
181
# Keyword parameters passed to publish_file.
182
reader_name = "standalone"
185
settings_overrides['output-encoding'] = 'utf-8'
186
# Relative to main ``test/`` directory.
187
settings_overrides['stylesheet_path'] = '../docutils/writers/html4css1/html4css1.css'
189
The two variables ``test_source`` and ``test_destination`` contain the
190
input file name (relative to ``functional/input/``) and the output
191
file name (relative to ``functional/output/`` and
192
``functional/expected/``). Note that the file names can be chosen
193
arbitrarily. However, the file names in ``functional/output/`` *must*
194
match the file names in ``functional/expected/``.
196
If defined, ``_test_more`` must be a function with the following
199
def _test_more(expected_dir, output_dir, test_case, parameters):
201
This function is called from the test case to perform tests beyond the
202
simple comparison of expected and actual output files.
204
``test_source`` and ``test_destination`` are removed from the
205
namespace, as are all variables whose names begin with an underscore
206
("_"). The remaining names are passed as keyword arguments to
207
``docutils.core.publish_file``, so you can set reader, parser, writer
208
and anything else you want to configure. Note that
209
``settings_overrides`` is already initialized as a dictionary *before*
210
the execution of the config file.
216
In order to create a new test, put the input test file into
217
``functional/input/``. Then create a config file in
218
``functional/tests/`` which sets at least input and output file names,
219
reader, parser and writer.
221
Now run ``test_functional.py``. The test will fail, of course,
222
because you do not have an expected output yet. However, an output
223
file will have been generated in ``functional/output/``. Check this
224
output file for validity and correctness. Then copy the file to
225
``functional/expected/``.
227
If you rerun ``test_functional.py`` now, it should pass.
229
If you run ``test_functional.py`` later and the actual output doesn't
230
match the expected output anymore, the test will fail.
232
If this is the case and you made an intentional change, check the
233
actual output for validity and correctness, copy it to
234
``functional/expected/`` (overwriting the old expected output), and
238
.. _default configuration file:
240
The Default Configuration File
241
------------------------------
243
The file ``functional/tests/_default.py`` contains default settings.
244
It is executed just before the actual configuration files, which has
245
the same effect as if the contents of ``_default.py`` were prepended
246
to every configuration file.
added
removed
docs/dev/testing.txt
