2
prev_title: Extension API
3
prev_url: extensions/api.html
5
next_url: change_log.html
9
Python-Markdown comes with a test suite which uses the [Nose] testing
10
framework and [YAML]. The test suite primarily serves to ensure that new bugs
11
are not introduced as existing bugs are patched or new features are added. It
12
also allows Python-Markdown to be tested with the tests from other
13
implementations such as John Gruber's [Perl] implementation or Michel
14
Fortin's [PHP] implementation.
16
The test suite can be run by calling the `run_tests.py` command at the root of
17
the distribution tarball or by calling the `nosetests` command directly. Either
18
way, Nose will need to be installed on your system first (run `easy_install
19
nose`). Any standard nosetests configuration options can be passed in on the command
20
line (i.e.: verbosity level or use of a plugin like coverage).
22
Additionally, a nicely formatted HTML report of all output is written to a
23
temporary file in `test-output.html`. Open the file in a browser to view
26
A `tox.ini` file is also provided, so [tox] can be used to automatically create
27
virtual environments, install all testing dependencies and run the tests on
28
each supported Python version. See the wiki for instructions on
29
[setting up a testing environment] to use tox.
31
The test suite contains two kinds of tests: Markdown Syntax Tests and Unit
34
## Markdown Syntax Tests
36
The Syntax Tests are in the various directories contained within the 'tests'
37
directory of the packaged tarball. Each test consists of a matching pair of text
38
and HTML files. The text file contains a snippet of Markdown source text
39
formatted for a specific syntax feature and the HTML file contains the expected
40
HTML output of that snippet. When the test suite is run, each text file is run
41
through Markdown and the output is compared with the HTML file as a separate
44
In fact, this is the primary reason for using Nose, it gives us an easy way to
45
treat each of these tests as a separate unit test which is reported on
46
separately. Additionally, with the help of a couple custom Nose plugins which
47
are included with the Markdown Test Suite, we are able to get back an easy to
48
read diff of the actual output compared to expected output when a test fails.
50
Here is some sample output with a test that is failing because of some
51
insignificant white space differences:
54
..........................................................M...........
55
............................SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
56
SSSSSSSSSS.................S..........................................
58
======================================================================
59
MarkdownSyntaxError: TestSyntax: "misc/lists3"
60
----------------------------------------------------------------------
61
MarkdownSyntaxError: Output from "/home/waylan/code/python-markdown/te
62
sts/misc/lists3.txt" failed to match expected output.
64
--- /home/waylan/code/python-markdown/tests/misc/lists3.html
65
+++ actual_output.html
69
-sdf asdf asdf asdf asdf
70
-asda asdf asdfasd</li>
71
+ sdf asdf asdf asdf asdf
72
+ asda asdf asdfasd</li>
75
----------------------------------------------------------------------
76
Ran 219 tests in 7.698s
78
FAILED (MarkdownSyntaxError=1, SKIP=53)
80
Note that 219 tests were run, one of which failed with a `MarkdownSyntaxError`.
81
Only Markdown Syntax Tests should fail with a `MarkdownSyntaxError`. Nose then
82
formats the error reports for `MarkdownSyntaxError`s so that they only include
83
useful information. Namely the text file which failed and a unified diff showing
84
the failure. Without the plugin, you would also get a useless traceback showing
85
how the code stepped through the test framework, but nothing about how Markdown
88
If, on the other hand, a Syntax Test failed because some other exception gets
89
raised by either Markdown or the test suite, then that would be reported as per
90
a normal unit test failure with the appropriate traceback for debugging
93
### Syntax Test Configuration Settings
95
The other thing to note about the above example is that 53 tests were skipped.
96
Those tests have been explicitly configured to be skipped as they are primarily
97
tests from either PHP or Perl which are known to fail for various reasons. In
98
fact, a number of different configuration settings can be set for any specific
101
Each Syntax Test directory contains a `test.cfg` file in the [YAML] format. The
102
file may contain a separate section for each text file named exactly as the file
103
is named minus the file extension (i.e.; the section for a test in `foo.txt`
104
would be `foo`). All settings are optional. Default settings for the entire
105
directory can be set under the `DEFAULT` section (must be all caps). Any
106
settings under a specific file section will override anything in the
107
`DEFAULT` section for that specific test only.
109
Below are the configuration options available and the defaults used when they
110
are not explicitly set.
112
* `normalize`: Switches white space normalization of the test output on or off.
113
Defaults to `False` (off). Note: This requires that [PyTidyLib] be installed on
114
the system. Otherwise the test will be skipped, regardless of any other
116
* `skip`: Switches skipping of the test on and off. Defaults to `False` (off).
117
* `input_ext`: Extension of input file. Defaults to `.txt`. Useful for tests
118
from other implementations.
119
* `output_ext`: Extension of output file. Defaults to `.html`. Useful for tests
120
from other implementations.
121
* Any keyword argument accepted by the Markdown class. If not set, Markdown's
126
Unit Tests are used as regression tests for Python-Markdown's API.
127
All Unit Tests shipped with Python-Markdown are standard Python Unit Tests and
128
are all contained in `tests/test_apis.py` and `tests/test_extensions.py`.
129
Standard discovery methods are used to find and run the tests. Therefore, when
130
writing new tests, those standards and naming conventions should be followed.
132
[Nose]: http://somethingaboutorange.com/mrl/projects/nose/
133
[Perl]: http://daringfireball.net/projects/markdown/
134
[PHP]: http://michelf.com/projects/php-markdown/
135
[PyTidyLib]: http://countergram.com/open-source/pytidylib/
136
[tox]: http://testrun.org/tox/latest/
137
[setting up a testing environment]: https://github.com/waylan/Python-Markdown/wiki/Test-Environment-Setup
138
[YAML]: http://yaml.org/
added
removed
zinc/python/zinc/util/Markdown-2.6.2/docs/test_suite.txt
