1
:mod:`unittest` --- Unit testing framework
2
==========================================
5
:synopsis: Unit testing framework for Python.
6
.. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com>
7
.. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com>
8
.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
9
.. sectionauthor:: Raymond Hettinger <python@rcn.com>
13
(If you are already familiar with the basic concepts of testing, you might want
14
to skip to :ref:`the list of assert methods <assert-methods>`.)
16
The Python unit testing framework, sometimes referred to as "PyUnit," is a
17
Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in
18
turn, a Java version of Kent's Smalltalk testing framework. Each is the de
19
facto standard unit testing framework for its respective language.
21
:mod:`unittest` supports test automation, sharing of setup and shutdown code for
22
tests, aggregation of tests into collections, and independence of the tests from
23
the reporting framework. The :mod:`unittest` module provides classes that make
24
it easy to support these qualities for a set of tests.
26
To achieve this, :mod:`unittest` supports some important concepts:
29
A :dfn:`test fixture` represents the preparation needed to perform one or more
30
tests, and any associate cleanup actions. This may involve, for example,
31
creating temporary or proxy databases, directories, or starting a server
35
A :dfn:`test case` is the smallest unit of testing. It checks for a specific
36
response to a particular set of inputs. :mod:`unittest` provides a base class,
37
:class:`TestCase`, which may be used to create new test cases.
40
A :dfn:`test suite` is a collection of test cases, test suites, or both. It is
41
used to aggregate tests that should be executed together.
44
A :dfn:`test runner` is a component which orchestrates the execution of tests
45
and provides the outcome to the user. The runner may use a graphical interface,
46
a textual interface, or return a special value to indicate the results of
49
The test case and test fixture concepts are supported through the
50
:class:`TestCase` and :class:`FunctionTestCase` classes; the former should be
51
used when creating new tests, and the latter can be used when integrating
52
existing test code with a :mod:`unittest`\ -driven framework. When building test
53
fixtures using :class:`TestCase`, the :meth:`~TestCase.setUp` and
54
:meth:`~TestCase.tearDown` methods can be overridden to provide initialization
55
and cleanup for the fixture. With :class:`FunctionTestCase`, existing functions
56
can be passed to the constructor for these purposes. When the test is run, the
57
fixture initialization is run first; if it succeeds, the cleanup method is run
58
after the test has been executed, regardless of the outcome of the test. Each
59
instance of the :class:`TestCase` will only be used to run a single test method,
60
so a new fixture is created for each test.
62
Test suites are implemented by the :class:`TestSuite` class. This class allows
63
individual tests and test suites to be aggregated; when the suite is executed,
64
all tests added directly to the suite and in "child" test suites are run.
66
A test runner is an object that provides a single method,
67
:meth:`~TestRunner.run`, which accepts a :class:`TestCase` or :class:`TestSuite`
68
object as a parameter, and returns a result object. The class
69
:class:`TestResult` is provided for use as the result object. :mod:`unittest`
70
provides the :class:`TextTestRunner` as an example test runner which reports
71
test results on the standard error stream by default. Alternate runners can be
72
implemented for other environments (such as graphical environments) without any
73
need to derive from a specific class.
79
Another test-support module with a very different flavor.
81
`unittest2: A backport of new unittest features for Python 2.4-2.6 <https://pypi.python.org/pypi/unittest2>`_
82
Many new features were added to unittest in Python 2.7, including test
83
discovery. unittest2 allows you to use these features with earlier
86
`Simple Smalltalk Testing: With Patterns <https://web.archive.org/web/20150315073817/http://www.xprogramming.com/testfram.htm>`_
87
Kent Beck's original paper on testing frameworks using the pattern shared
90
`Nose <https://nose.readthedocs.org/en/latest/>`_ and `py.test <http://pytest.org>`_
91
Third-party unittest frameworks with a lighter-weight syntax for writing
92
tests. For example, ``assert func(10) == 42``.
94
`The Python Testing Tools Taxonomy <https://wiki.python.org/moin/PythonTestingToolsTaxonomy>`_
95
An extensive list of Python testing tools including functional testing
96
frameworks and mock object libraries.
98
`Testing in Python Mailing List <http://lists.idyll.org/listinfo/testing-in-python>`_
99
A special-interest-group for discussion of testing, and testing tools,
103
.. _unittest-minimal-example:
108
The :mod:`unittest` module provides a rich set of tools for constructing and
109
running tests. This section demonstrates that a small subset of the tools
110
suffice to meet the needs of most users.
112
Here is a short script to test three string methods::
116
class TestStringMethods(unittest.TestCase):
118
def test_upper(self):
119
self.assertEqual('foo'.upper(), 'FOO')
121
def test_isupper(self):
122
self.assertTrue('FOO'.isupper())
123
self.assertFalse('Foo'.isupper())
125
def test_split(self):
127
self.assertEqual(s.split(), ['hello', 'world'])
128
# check that s.split fails when the separator is not a string
129
with self.assertRaises(TypeError):
132
if __name__ == '__main__':
136
A testcase is created by subclassing :class:`unittest.TestCase`. The three
137
individual tests are defined with methods whose names start with the letters
138
``test``. This naming convention informs the test runner about which methods
141
The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an
142
expected result; :meth:`~TestCase.assertTrue` or :meth:`~TestCase.assertFalse`
143
to verify a condition; or :meth:`~TestCase.assertRaises` to verify that a
144
specific exception gets raised. These methods are used instead of the
145
:keyword:`assert` statement so the test runner can accumulate all test results
146
and produce a report.
148
The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you
149
to define instructions that will be executed before and after each test method.
150
They are covered in more detail in the section :ref:`organizing-tests`.
152
The final block shows a simple way to run the tests. :func:`unittest.main`
153
provides a command-line interface to the test script. When run from the command
154
line, the above script produces an output that looks like this::
157
----------------------------------------------------------------------
158
Ran 3 tests in 0.000s
162
Instead of :func:`unittest.main`, there are other ways to run the tests with a
163
finer level of control, less terse output, and no requirement to be run from the
164
command line. For example, the last two lines may be replaced with::
166
suite = unittest.TestLoader().loadTestsFromTestCase(TestStringMethods)
167
unittest.TextTestRunner(verbosity=2).run(suite)
169
Running the revised script from the interpreter or another script produces the
172
test_isupper (__main__.TestStringMethods) ... ok
173
test_split (__main__.TestStringMethods) ... ok
174
test_upper (__main__.TestStringMethods) ... ok
176
----------------------------------------------------------------------
177
Ran 3 tests in 0.001s
181
The above examples show the most commonly used :mod:`unittest` features which
182
are sufficient to meet many everyday testing needs. The remainder of the
183
documentation explores the full feature set from first principles.
186
.. _unittest-command-line-interface:
188
Command-Line Interface
189
----------------------
191
The unittest module can be used from the command line to run tests from
192
modules, classes or even individual test methods::
194
python -m unittest test_module1 test_module2
195
python -m unittest test_module.TestClass
196
python -m unittest test_module.TestClass.test_method
198
You can pass in a list with any combination of module names, and fully
199
qualified class or method names.
201
You can run tests with more detail (higher verbosity) by passing in the -v flag::
203
python -m unittest -v test_module
205
For a list of all the command-line options::
207
python -m unittest -h
209
.. versionchanged:: 2.7
210
In earlier versions it was only possible to run individual test methods and
211
not modules or classes.
217
:program:`unittest` supports these command-line options:
219
.. program:: unittest
221
.. cmdoption:: -b, --buffer
223
The standard output and standard error streams are buffered during the test
224
run. Output during a passing test is discarded. Output is echoed normally
225
on test fail or error and is added to the failure messages.
227
.. cmdoption:: -c, --catch
229
:kbd:`Control-C` during the test run waits for the current test to end and then
230
reports all the results so far. A second :kbd:`Control-C` raises the normal
231
:exc:`KeyboardInterrupt` exception.
233
See `Signal Handling`_ for the functions that provide this functionality.
235
.. cmdoption:: -f, --failfast
237
Stop the test run on the first error or failure.
239
.. versionadded:: 2.7
240
The command-line options ``-b``, ``-c`` and ``-f`` were added.
242
The command line can also be used for test discovery, for running all of the
243
tests in a project or just a subset.
246
.. _unittest-test-discovery:
251
.. versionadded:: 2.7
253
Unittest supports simple test discovery. In order to be compatible with test
254
discovery, all of the test files must be :ref:`modules <tut-modules>` or
255
:ref:`packages <tut-packages>` importable from the top-level directory of
256
the project (this means that their filenames must be valid
257
:ref:`identifiers <identifiers>`).
259
Test discovery is implemented in :meth:`TestLoader.discover`, but can also be
260
used from the command line. The basic command-line usage is::
263
python -m unittest discover
265
The ``discover`` sub-command has the following options:
267
.. program:: unittest discover
269
.. cmdoption:: -v, --verbose
273
.. cmdoption:: -s, --start-directory directory
275
Directory to start discovery (``.`` default)
277
.. cmdoption:: -p, --pattern pattern
279
Pattern to match test files (``test*.py`` default)
281
.. cmdoption:: -t, --top-level-directory directory
283
Top level directory of project (defaults to start directory)
285
The :option:`-s`, :option:`-p`, and :option:`-t` options can be passed in
286
as positional arguments in that order. The following two command lines
289
python -m unittest discover -s project_directory -p "*_test.py"
290
python -m unittest discover project_directory "*_test.py"
292
As well as being a path it is possible to pass a package name, for example
293
``myproject.subpackage.test``, as the start directory. The package name you
294
supply will then be imported and its location on the filesystem will be used
295
as the start directory.
299
Test discovery loads tests by importing them. Once test discovery has
300
found all the test files from the start directory you specify it turns the
301
paths into package names to import. For example :file:`foo/bar/baz.py` will be
302
imported as ``foo.bar.baz``.
304
If you have a package installed globally and attempt test discovery on
305
a different copy of the package then the import *could* happen from the
306
wrong place. If this happens test discovery will warn you and exit.
308
If you supply the start directory as a package name rather than a
309
path to a directory then discover assumes that whichever location it
310
imports from is the location you intended, so you will not get the
313
Test modules and packages can customize test loading and discovery by through
314
the `load_tests protocol`_.
317
.. _organizing-tests:
322
The basic building blocks of unit testing are :dfn:`test cases` --- single
323
scenarios that must be set up and checked for correctness. In :mod:`unittest`,
324
test cases are represented by instances of :mod:`unittest`'s :class:`TestCase`
325
class. To make your own test cases you must write subclasses of
326
:class:`TestCase`, or use :class:`FunctionTestCase`.
328
An instance of a :class:`TestCase`\ -derived class is an object that can
329
completely run a single test method, together with optional set-up and tidy-up
332
The testing code of a :class:`TestCase` instance should be entirely self
333
contained, such that it can be run either in isolation or in arbitrary
334
combination with any number of other test cases.
336
The simplest :class:`TestCase` subclass will simply override the
337
:meth:`~TestCase.runTest` method in order to perform specific testing code::
341
class DefaultWidgetSizeTestCase(unittest.TestCase):
343
widget = Widget('The widget')
344
self.assertEqual(widget.size(), (50, 50), 'incorrect default size')
346
Note that in order to test something, we use one of the :meth:`assert\*`
347
methods provided by the :class:`TestCase` base class. If the test fails, an
348
exception will be raised, and :mod:`unittest` will identify the test case as a
349
:dfn:`failure`. Any other exceptions will be treated as :dfn:`errors`. This
350
helps you identify where the problem is: :dfn:`failures` are caused by incorrect
351
results - a 5 where you expected a 6. :dfn:`Errors` are caused by incorrect
352
code - e.g., a :exc:`TypeError` caused by an incorrect function call.
354
The way to run a test case will be described later. For now, note that to
355
construct an instance of such a test case, we call its constructor without
358
testCase = DefaultWidgetSizeTestCase()
360
Now, such test cases can be numerous, and their set-up can be repetitive. In
361
the above case, constructing a :class:`Widget` in each of 100 Widget test case
362
subclasses would mean unsightly duplication.
364
Luckily, we can factor out such set-up code by implementing a method called
365
:meth:`~TestCase.setUp`, which the testing framework will automatically call for
366
us when we run the test::
370
class SimpleWidgetTestCase(unittest.TestCase):
372
self.widget = Widget('The widget')
374
class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
376
self.assertEqual(self.widget.size(), (50,50),
377
'incorrect default size')
379
class WidgetResizeTestCase(SimpleWidgetTestCase):
381
self.widget.resize(100,150)
382
self.assertEqual(self.widget.size(), (100,150),
383
'wrong size after resize')
385
If the :meth:`~TestCase.setUp` method raises an exception while the test is
386
running, the framework will consider the test to have suffered an error, and the
387
:meth:`~TestCase.runTest` method will not be executed.
389
Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up
390
after the :meth:`~TestCase.runTest` method has been run::
394
class SimpleWidgetTestCase(unittest.TestCase):
396
self.widget = Widget('The widget')
399
self.widget.dispose()
402
If :meth:`~TestCase.setUp` succeeded, the :meth:`~TestCase.tearDown` method will
403
be run whether :meth:`~TestCase.runTest` succeeded or not.
405
Such a working environment for the testing code is called a :dfn:`fixture`.
407
Often, many small test cases will use the same fixture. In this case, we would
408
end up subclassing :class:`SimpleWidgetTestCase` into many small one-method
409
classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and
410
discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler
415
class WidgetTestCase(unittest.TestCase):
417
self.widget = Widget('The widget')
420
self.widget.dispose()
423
def test_default_size(self):
424
self.assertEqual(self.widget.size(), (50,50),
425
'incorrect default size')
427
def test_resize(self):
428
self.widget.resize(100,150)
429
self.assertEqual(self.widget.size(), (100,150),
430
'wrong size after resize')
432
Here we have not provided a :meth:`~TestCase.runTest` method, but have instead
433
provided two different test methods. Class instances will now each run one of
434
the :meth:`test_\*` methods, with ``self.widget`` created and destroyed
435
separately for each instance. When creating an instance we must specify the
436
test method it is to run. We do this by passing the method name in the
439
defaultSizeTestCase = WidgetTestCase('test_default_size')
440
resizeTestCase = WidgetTestCase('test_resize')
442
Test case instances are grouped together according to the features they test.
443
:mod:`unittest` provides a mechanism for this: the :dfn:`test suite`,
444
represented by :mod:`unittest`'s :class:`TestSuite` class::
446
widgetTestSuite = unittest.TestSuite()
447
widgetTestSuite.addTest(WidgetTestCase('test_default_size'))
448
widgetTestSuite.addTest(WidgetTestCase('test_resize'))
450
For the ease of running tests, as we will see later, it is a good idea to
451
provide in each test module a callable object that returns a pre-built test
455
suite = unittest.TestSuite()
456
suite.addTest(WidgetTestCase('test_default_size'))
457
suite.addTest(WidgetTestCase('test_resize'))
463
tests = ['test_default_size', 'test_resize']
465
return unittest.TestSuite(map(WidgetTestCase, tests))
467
Since it is a common pattern to create a :class:`TestCase` subclass with many
468
similarly named test functions, :mod:`unittest` provides a :class:`TestLoader`
469
class that can be used to automate the process of creating a test suite and
470
populating it with individual tests. For example, ::
472
suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
474
will create a test suite that will run ``WidgetTestCase.test_default_size()`` and
475
``WidgetTestCase.test_resize``. :class:`TestLoader` uses the ``'test'`` method
476
name prefix to identify test methods automatically.
478
Note that the order in which the various test cases will be run is
479
determined by sorting the test function names with respect to the
480
built-in ordering for strings.
482
Often it is desirable to group suites of test cases together, so as to run tests
483
for the whole system at once. This is easy, since :class:`TestSuite` instances
484
can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be
485
added to a :class:`TestSuite`::
487
suite1 = module1.TheTestSuite()
488
suite2 = module2.TheTestSuite()
489
alltests = unittest.TestSuite([suite1, suite2])
491
You can place the definitions of test cases and test suites in the same modules
492
as the code they are to test (such as :file:`widget.py`), but there are several
493
advantages to placing the test code in a separate module, such as
494
:file:`test_widget.py`:
496
* The test module can be run standalone from the command line.
498
* The test code can more easily be separated from shipped code.
500
* There is less temptation to change test code to fit the code it tests without
503
* Test code should be modified much less frequently than the code it tests.
505
* Tested code can be refactored more easily.
507
* Tests for modules written in C must be in separate modules anyway, so why not
510
* If the testing strategy changes, there is no need to change the source code.
513
.. _legacy-unit-tests:
515
Re-using old test code
516
----------------------
518
Some users will find that they have existing test code that they would like to
519
run from :mod:`unittest`, without converting every old test function to a
520
:class:`TestCase` subclass.
522
For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.
523
This subclass of :class:`TestCase` can be used to wrap an existing test
524
function. Set-up and tear-down functions can also be provided.
526
Given the following test function::
529
something = makeSomething()
530
assert something.name is not None
533
one can create an equivalent test case instance as follows::
535
testcase = unittest.FunctionTestCase(testSomething)
537
If there are additional set-up and tear-down methods that should be called as
538
part of the test case's operation, they can also be provided like so::
540
testcase = unittest.FunctionTestCase(testSomething,
541
setUp=makeSomethingDB,
542
tearDown=deleteSomethingDB)
544
To make migrating existing test suites easier, :mod:`unittest` supports tests
545
raising :exc:`AssertionError` to indicate test failure. However, it is
546
recommended that you use the explicit :meth:`TestCase.fail\*` and
547
:meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest`
548
may treat :exc:`AssertionError` differently.
552
Even though :class:`FunctionTestCase` can be used to quickly convert an
553
existing test base over to a :mod:`unittest`\ -based system, this approach is
554
not recommended. Taking the time to set up proper :class:`TestCase`
555
subclasses will make future test refactorings infinitely easier.
557
In some cases, the existing tests may have been written using the :mod:`doctest`
558
module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can
559
automatically build :class:`unittest.TestSuite` instances from the existing
560
:mod:`doctest`\ -based tests.
563
.. _unittest-skipping:
565
Skipping tests and expected failures
566
------------------------------------
568
.. versionadded:: 2.7
570
Unittest supports skipping individual test methods and even whole classes of
571
tests. In addition, it supports marking a test as an "expected failure," a test
572
that is broken and will fail, but shouldn't be counted as a failure on a
575
Skipping a test is simply a matter of using the :func:`skip` :term:`decorator`
576
or one of its conditional variants.
578
Basic skipping looks like this::
580
class MyTestCase(unittest.TestCase):
582
@unittest.skip("demonstrating skipping")
583
def test_nothing(self):
584
self.fail("shouldn't happen")
586
@unittest.skipIf(mylib.__version__ < (1, 3),
587
"not supported in this library version")
588
def test_format(self):
589
# Tests that work for only a certain version of the library.
592
@unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
593
def test_windows_support(self):
594
# windows specific testing code
597
This is the output of running the example above in verbose mode::
599
test_format (__main__.MyTestCase) ... skipped 'not supported in this library version'
600
test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping'
601
test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows'
603
----------------------------------------------------------------------
604
Ran 3 tests in 0.005s
608
Classes can be skipped just like methods::
610
@unittest.skip("showing class skipping")
611
class MySkippedTestCase(unittest.TestCase):
612
def test_not_run(self):
615
:meth:`TestCase.setUp` can also skip the test. This is useful when a resource
616
that needs to be set up is not available.
618
Expected failures use the :func:`expectedFailure` decorator. ::
620
class ExpectedFailureTestCase(unittest.TestCase):
621
@unittest.expectedFailure
623
self.assertEqual(1, 0, "broken")
625
It's easy to roll your own skipping decorators by making a decorator that calls
626
:func:`skip` on the test when it wants it to be skipped. This decorator skips
627
the test unless the passed object has a certain attribute::
629
def skipUnlessHasattr(obj, attr):
630
if hasattr(obj, attr):
631
return lambda func: func
632
return unittest.skip("{!r} doesn't have {!r}".format(obj, attr))
634
The following decorators implement test skipping and expected failures:
636
.. function:: skip(reason)
638
Unconditionally skip the decorated test. *reason* should describe why the
639
test is being skipped.
641
.. function:: skipIf(condition, reason)
643
Skip the decorated test if *condition* is true.
645
.. function:: skipUnless(condition, reason)
647
Skip the decorated test unless *condition* is true.
649
.. function:: expectedFailure
651
Mark the test as an expected failure. If the test fails when run, the test
652
is not counted as a failure.
654
.. exception:: SkipTest(reason)
656
This exception is raised to skip a test.
658
Usually you can use :meth:`TestCase.skipTest` or one of the skipping
659
decorators instead of raising this directly.
661
Skipped tests will not have :meth:`setUp` or :meth:`tearDown` run around them.
662
Skipped classes will not have :meth:`setUpClass` or :meth:`tearDownClass` run.
665
.. _unittest-contents:
667
Classes and functions
668
---------------------
670
This section describes in depth the API of :mod:`unittest`.
673
.. _testcase-objects:
678
.. class:: TestCase(methodName='runTest')
680
Instances of the :class:`TestCase` class represent the smallest testable units
681
in the :mod:`unittest` universe. This class is intended to be used as a base
682
class, with specific tests being implemented by concrete subclasses. This class
683
implements the interface needed by the test runner to allow it to drive the
684
test, and methods that the test code can use to check for and report various
687
Each instance of :class:`TestCase` will run a single test method: the method
688
named *methodName*. If you remember, we had an earlier example that went
689
something like this::
692
suite = unittest.TestSuite()
693
suite.addTest(WidgetTestCase('test_default_size'))
694
suite.addTest(WidgetTestCase('test_resize'))
697
Here, we create two instances of :class:`WidgetTestCase`, each of which runs a
700
*methodName* defaults to :meth:`runTest`.
702
:class:`TestCase` instances provide three groups of methods: one group used
703
to run the test, another used by the test implementation to check conditions
704
and report failures, and some inquiry methods allowing information about the
705
test itself to be gathered.
707
Methods in the first group (running the test) are:
712
Method called to prepare the test fixture. This is called immediately
713
before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`,
714
any exception raised by this method will be considered an error rather than
715
a test failure. The default implementation does nothing.
718
.. method:: tearDown()
720
Method called immediately after the test method has been called and the
721
result recorded. This is called even if the test method raised an
722
exception, so the implementation in subclasses may need to be particularly
723
careful about checking internal state. Any exception, other than :exc:`AssertionError`
724
or :exc:`SkipTest`, raised by this method will be considered an error rather than a
725
test failure. This method will only be called if the :meth:`setUp` succeeds,
726
regardless of the outcome of the test method. The default implementation does nothing.
729
.. method:: setUpClass()
731
A class method called before tests in an individual class run.
732
``setUpClass`` is called with the class as the only argument
733
and must be decorated as a :func:`classmethod`::
739
See `Class and Module Fixtures`_ for more details.
741
.. versionadded:: 2.7
744
.. method:: tearDownClass()
746
A class method called after tests in an individual class have run.
747
``tearDownClass`` is called with the class as the only argument
748
and must be decorated as a :meth:`classmethod`::
751
def tearDownClass(cls):
754
See `Class and Module Fixtures`_ for more details.
756
.. versionadded:: 2.7
759
.. method:: run(result=None)
761
Run the test, collecting the result into the test result object passed as
762
*result*. If *result* is omitted or ``None``, a temporary result
763
object is created (by calling the :meth:`defaultTestResult` method) and
764
used. The result object is not returned to :meth:`run`'s caller.
766
The same effect may be had by simply calling the :class:`TestCase`
770
.. method:: skipTest(reason)
772
Calling this during a test method or :meth:`setUp` skips the current
773
test. See :ref:`unittest-skipping` for more information.
775
.. versionadded:: 2.7
780
Run the test without collecting the result. This allows exceptions raised
781
by the test to be propagated to the caller, and can be used to support
782
running tests under a debugger.
786
The :class:`TestCase` class provides several assert methods to check for and
787
report failures. The following table lists the most commonly used methods
788
(see the tables below for more assert methods):
790
+-----------------------------------------+-----------------------------+---------------+
791
| Method | Checks that | New in |
792
+=========================================+=============================+===============+
793
| :meth:`assertEqual(a, b) | ``a == b`` | |
794
| <TestCase.assertEqual>` | | |
795
+-----------------------------------------+-----------------------------+---------------+
796
| :meth:`assertNotEqual(a, b) | ``a != b`` | |
797
| <TestCase.assertNotEqual>` | | |
798
+-----------------------------------------+-----------------------------+---------------+
799
| :meth:`assertTrue(x) | ``bool(x) is True`` | |
800
| <TestCase.assertTrue>` | | |
801
+-----------------------------------------+-----------------------------+---------------+
802
| :meth:`assertFalse(x) | ``bool(x) is False`` | |
803
| <TestCase.assertFalse>` | | |
804
+-----------------------------------------+-----------------------------+---------------+
805
| :meth:`assertIs(a, b) | ``a is b`` | 2.7 |
806
| <TestCase.assertIs>` | | |
807
+-----------------------------------------+-----------------------------+---------------+
808
| :meth:`assertIsNot(a, b) | ``a is not b`` | 2.7 |
809
| <TestCase.assertIsNot>` | | |
810
+-----------------------------------------+-----------------------------+---------------+
811
| :meth:`assertIsNone(x) | ``x is None`` | 2.7 |
812
| <TestCase.assertIsNone>` | | |
813
+-----------------------------------------+-----------------------------+---------------+
814
| :meth:`assertIsNotNone(x) | ``x is not None`` | 2.7 |
815
| <TestCase.assertIsNotNone>` | | |
816
+-----------------------------------------+-----------------------------+---------------+
817
| :meth:`assertIn(a, b) | ``a in b`` | 2.7 |
818
| <TestCase.assertIn>` | | |
819
+-----------------------------------------+-----------------------------+---------------+
820
| :meth:`assertNotIn(a, b) | ``a not in b`` | 2.7 |
821
| <TestCase.assertNotIn>` | | |
822
+-----------------------------------------+-----------------------------+---------------+
823
| :meth:`assertIsInstance(a, b) | ``isinstance(a, b)`` | 2.7 |
824
| <TestCase.assertIsInstance>` | | |
825
+-----------------------------------------+-----------------------------+---------------+
826
| :meth:`assertNotIsInstance(a, b) | ``not isinstance(a, b)`` | 2.7 |
827
| <TestCase.assertNotIsInstance>` | | |
828
+-----------------------------------------+-----------------------------+---------------+
830
All the assert methods (except :meth:`assertRaises`,
831
:meth:`assertRaisesRegexp`)
832
accept a *msg* argument that, if specified, is used as the error message on
833
failure (see also :data:`longMessage`).
835
.. method:: assertEqual(first, second, msg=None)
837
Test that *first* and *second* are equal. If the values do not compare
838
equal, the test will fail.
840
In addition, if *first* and *second* are the exact same type and one of
841
list, tuple, dict, set, frozenset or unicode or any type that a subclass
842
registers with :meth:`addTypeEqualityFunc` the type-specific equality
843
function will be called in order to generate a more useful default
844
error message (see also the :ref:`list of type-specific methods
845
<type-specific-methods>`).
847
.. versionchanged:: 2.7
848
Added the automatic calling of type-specific equality function.
851
.. method:: assertNotEqual(first, second, msg=None)
853
Test that *first* and *second* are not equal. If the values do compare
854
equal, the test will fail.
856
.. method:: assertTrue(expr, msg=None)
857
assertFalse(expr, msg=None)
859
Test that *expr* is true (or false).
861
Note that this is equivalent to ``bool(expr) is True`` and not to ``expr
862
is True`` (use ``assertIs(expr, True)`` for the latter). This method
863
should also be avoided when more specific methods are available (e.g.
864
``assertEqual(a, b)`` instead of ``assertTrue(a == b)``), because they
865
provide a better error message in case of failure.
868
.. method:: assertIs(first, second, msg=None)
869
assertIsNot(first, second, msg=None)
871
Test that *first* and *second* evaluate (or don't evaluate) to the same object.
873
.. versionadded:: 2.7
876
.. method:: assertIsNone(expr, msg=None)
877
assertIsNotNone(expr, msg=None)
879
Test that *expr* is (or is not) None.
881
.. versionadded:: 2.7
884
.. method:: assertIn(first, second, msg=None)
885
assertNotIn(first, second, msg=None)
887
Test that *first* is (or is not) in *second*.
889
.. versionadded:: 2.7
892
.. method:: assertIsInstance(obj, cls, msg=None)
893
assertNotIsInstance(obj, cls, msg=None)
895
Test that *obj* is (or is not) an instance of *cls* (which can be a
896
class or a tuple of classes, as supported by :func:`isinstance`).
897
To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`.
899
.. versionadded:: 2.7
902
It is also possible to check that exceptions and warnings are raised using
903
the following methods:
905
+---------------------------------------------------------+--------------------------------------+------------+
906
| Method | Checks that | New in |
907
+=========================================================+======================================+============+
908
| :meth:`assertRaises(exc, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | |
909
| <TestCase.assertRaises>` | | |
910
+---------------------------------------------------------+--------------------------------------+------------+
911
| :meth:`assertRaisesRegexp(exc, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | 2.7 |
912
| <TestCase.assertRaisesRegexp>` | and the message matches regex *r* | |
913
+---------------------------------------------------------+--------------------------------------+------------+
915
.. method:: assertRaises(exception, callable, *args, **kwds)
916
assertRaises(exception)
918
Test that an exception is raised when *callable* is called with any
919
positional or keyword arguments that are also passed to
920
:meth:`assertRaises`. The test passes if *exception* is raised, is an
921
error if another exception is raised, or fails if no exception is raised.
922
To catch any of a group of exceptions, a tuple containing the exception
923
classes may be passed as *exception*.
925
If only the *exception* argument is given, returns a context manager so
926
that the code under test can be written inline rather than as a function::
928
with self.assertRaises(SomeException):
931
The context manager will store the caught exception object in its
932
:attr:`exception` attribute. This can be useful if the intention
933
is to perform additional checks on the exception raised::
935
with self.assertRaises(SomeException) as cm:
938
the_exception = cm.exception
939
self.assertEqual(the_exception.error_code, 3)
941
.. versionchanged:: 2.7
942
Added the ability to use :meth:`assertRaises` as a context manager.
945
.. method:: assertRaisesRegexp(exception, regexp, callable, *args, **kwds)
946
assertRaisesRegexp(exception, regexp)
948
Like :meth:`assertRaises` but also tests that *regexp* matches
949
on the string representation of the raised exception. *regexp* may be
950
a regular expression object or a string containing a regular expression
951
suitable for use by :func:`re.search`. Examples::
953
self.assertRaisesRegexp(ValueError, "invalid literal for.*XYZ'$",
958
with self.assertRaisesRegexp(ValueError, 'literal'):
961
.. versionadded:: 2.7
965
There are also other methods used to perform more specific checks, such as:
967
+---------------------------------------+--------------------------------+--------------+
968
| Method | Checks that | New in |
969
+=======================================+================================+==============+
970
| :meth:`assertAlmostEqual(a, b) | ``round(a-b, 7) == 0`` | |
971
| <TestCase.assertAlmostEqual>` | | |
972
+---------------------------------------+--------------------------------+--------------+
973
| :meth:`assertNotAlmostEqual(a, b) | ``round(a-b, 7) != 0`` | |
974
| <TestCase.assertNotAlmostEqual>` | | |
975
+---------------------------------------+--------------------------------+--------------+
976
| :meth:`assertGreater(a, b) | ``a > b`` | 2.7 |
977
| <TestCase.assertGreater>` | | |
978
+---------------------------------------+--------------------------------+--------------+
979
| :meth:`assertGreaterEqual(a, b) | ``a >= b`` | 2.7 |
980
| <TestCase.assertGreaterEqual>` | | |
981
+---------------------------------------+--------------------------------+--------------+
982
| :meth:`assertLess(a, b) | ``a < b`` | 2.7 |
983
| <TestCase.assertLess>` | | |
984
+---------------------------------------+--------------------------------+--------------+
985
| :meth:`assertLessEqual(a, b) | ``a <= b`` | 2.7 |
986
| <TestCase.assertLessEqual>` | | |
987
+---------------------------------------+--------------------------------+--------------+
988
| :meth:`assertRegexpMatches(s, r) | ``r.search(s)`` | 2.7 |
989
| <TestCase.assertRegexpMatches>` | | |
990
+---------------------------------------+--------------------------------+--------------+
991
| :meth:`assertNotRegexpMatches(s, r) | ``not r.search(s)`` | 2.7 |
992
| <TestCase.assertNotRegexpMatches>` | | |
993
+---------------------------------------+--------------------------------+--------------+
994
| :meth:`assertItemsEqual(a, b) | sorted(a) == sorted(b) and | 2.7 |
995
| <TestCase.assertItemsEqual>` | works with unhashable objs | |
996
+---------------------------------------+--------------------------------+--------------+
997
| :meth:`assertDictContainsSubset(a, b) | all the key/value pairs | 2.7 |
998
| <TestCase.assertDictContainsSubset>` | in *a* exist in *b* | |
999
+---------------------------------------+--------------------------------+--------------+
1002
.. method:: assertAlmostEqual(first, second, places=7, msg=None, delta=None)
1003
assertNotAlmostEqual(first, second, places=7, msg=None, delta=None)
1005
Test that *first* and *second* are approximately (or not approximately)
1006
equal by computing the difference, rounding to the given number of
1007
decimal *places* (default 7), and comparing to zero. Note that these
1008
methods round the values to the given number of *decimal places* (i.e.
1009
like the :func:`round` function) and not *significant digits*.
1011
If *delta* is supplied instead of *places* then the difference
1012
between *first* and *second* must be less or equal to (or greater than) *delta*.
1014
Supplying both *delta* and *places* raises a ``TypeError``.
1016
.. versionchanged:: 2.7
1017
:meth:`assertAlmostEqual` automatically considers almost equal objects
1018
that compare equal. :meth:`assertNotAlmostEqual` automatically fails
1019
if the objects compare equal. Added the *delta* keyword argument.
1023
.. method:: assertGreater(first, second, msg=None)
1024
assertGreaterEqual(first, second, msg=None)
1025
assertLess(first, second, msg=None)
1026
assertLessEqual(first, second, msg=None)
1028
Test that *first* is respectively >, >=, < or <= than *second* depending
1029
on the method name. If not, the test will fail::
1031
>>> self.assertGreaterEqual(3, 4)
1032
AssertionError: "3" unexpectedly not greater than or equal to "4"
1034
.. versionadded:: 2.7
1037
.. method:: assertRegexpMatches(text, regexp, msg=None)
1039
Test that a *regexp* search matches *text*. In case
1040
of failure, the error message will include the pattern and the *text* (or
1041
the pattern and the part of *text* that unexpectedly matched). *regexp*
1042
may be a regular expression object or a string containing a regular
1043
expression suitable for use by :func:`re.search`.
1045
.. versionadded:: 2.7
1048
.. method:: assertNotRegexpMatches(text, regexp, msg=None)
1050
Verifies that a *regexp* search does not match *text*. Fails with an error
1051
message including the pattern and the part of *text* that matches. *regexp*
1052
may be a regular expression object or a string containing a regular
1053
expression suitable for use by :func:`re.search`.
1055
.. versionadded:: 2.7
1058
.. method:: assertItemsEqual(actual, expected, msg=None)
1060
Test that sequence *expected* contains the same elements as *actual*,
1061
regardless of their order. When they don't, an error message listing the
1062
differences between the sequences will be generated.
1064
Duplicate elements are *not* ignored when comparing *actual* and
1065
*expected*. It verifies if each element has the same count in both
1066
sequences. It is the equivalent of ``assertEqual(sorted(expected),
1067
sorted(actual))`` but it works with sequences of unhashable objects as
1070
In Python 3, this method is named ``assertCountEqual``.
1072
.. versionadded:: 2.7
1075
.. method:: assertDictContainsSubset(expected, actual, msg=None)
1077
Tests whether the key/value pairs in dictionary *actual* are a
1078
superset of those in *expected*. If not, an error message listing
1079
the missing keys and mismatched values is generated.
1081
.. versionadded:: 2.7
1086
.. _type-specific-methods:
1088
The :meth:`assertEqual` method dispatches the equality check for objects of
1089
the same type to different type-specific methods. These methods are already
1090
implemented for most of the built-in types, but it's also possible to
1091
register new methods using :meth:`addTypeEqualityFunc`:
1093
.. method:: addTypeEqualityFunc(typeobj, function)
1095
Registers a type-specific method called by :meth:`assertEqual` to check
1096
if two objects of exactly the same *typeobj* (not subclasses) compare
1097
equal. *function* must take two positional arguments and a third msg=None
1098
keyword argument just as :meth:`assertEqual` does. It must raise
1099
:data:`self.failureException(msg) <failureException>` when inequality
1100
between the first two parameters is detected -- possibly providing useful
1101
information and explaining the inequalities in details in the error
1104
.. versionadded:: 2.7
1106
The list of type-specific methods automatically used by
1107
:meth:`~TestCase.assertEqual` are summarized in the following table. Note
1108
that it's usually not necessary to invoke these methods directly.
1110
+-----------------------------------------+-----------------------------+--------------+
1111
| Method | Used to compare | New in |
1112
+=========================================+=============================+==============+
1113
| :meth:`assertMultiLineEqual(a, b) | strings | 2.7 |
1114
| <TestCase.assertMultiLineEqual>` | | |
1115
+-----------------------------------------+-----------------------------+--------------+
1116
| :meth:`assertSequenceEqual(a, b) | sequences | 2.7 |
1117
| <TestCase.assertSequenceEqual>` | | |
1118
+-----------------------------------------+-----------------------------+--------------+
1119
| :meth:`assertListEqual(a, b) | lists | 2.7 |
1120
| <TestCase.assertListEqual>` | | |
1121
+-----------------------------------------+-----------------------------+--------------+
1122
| :meth:`assertTupleEqual(a, b) | tuples | 2.7 |
1123
| <TestCase.assertTupleEqual>` | | |
1124
+-----------------------------------------+-----------------------------+--------------+
1125
| :meth:`assertSetEqual(a, b) | sets or frozensets | 2.7 |
1126
| <TestCase.assertSetEqual>` | | |
1127
+-----------------------------------------+-----------------------------+--------------+
1128
| :meth:`assertDictEqual(a, b) | dicts | 2.7 |
1129
| <TestCase.assertDictEqual>` | | |
1130
+-----------------------------------------+-----------------------------+--------------+
1134
.. method:: assertMultiLineEqual(first, second, msg=None)
1136
Test that the multiline string *first* is equal to the string *second*.
1137
When not equal a diff of the two strings highlighting the differences
1138
will be included in the error message. This method is used by default
1139
when comparing strings with :meth:`assertEqual`.
1141
.. versionadded:: 2.7
1144
.. method:: assertSequenceEqual(seq1, seq2, msg=None, seq_type=None)
1146
Tests that two sequences are equal. If a *seq_type* is supplied, both
1147
*seq1* and *seq2* must be instances of *seq_type* or a failure will
1148
be raised. If the sequences are different an error message is
1149
constructed that shows the difference between the two.
1151
This method is not called directly by :meth:`assertEqual`, but
1152
it's used to implement :meth:`assertListEqual` and
1153
:meth:`assertTupleEqual`.
1155
.. versionadded:: 2.7
1158
.. method:: assertListEqual(list1, list2, msg=None)
1159
assertTupleEqual(tuple1, tuple2, msg=None)
1161
Tests that two lists or tuples are equal. If not, an error message is
1162
constructed that shows only the differences between the two. An error
1163
is also raised if either of the parameters are of the wrong type.
1164
These methods are used by default when comparing lists or tuples with
1165
:meth:`assertEqual`.
1167
.. versionadded:: 2.7
1170
.. method:: assertSetEqual(set1, set2, msg=None)
1172
Tests that two sets are equal. If not, an error message is constructed
1173
that lists the differences between the sets. This method is used by
1174
default when comparing sets or frozensets with :meth:`assertEqual`.
1176
Fails if either of *set1* or *set2* does not have a :meth:`set.difference`
1179
.. versionadded:: 2.7
1182
.. method:: assertDictEqual(expected, actual, msg=None)
1184
Test that two dictionaries are equal. If not, an error message is
1185
constructed that shows the differences in the dictionaries. This
1186
method will be used by default to compare dictionaries in
1187
calls to :meth:`assertEqual`.
1189
.. versionadded:: 2.7
1193
.. _other-methods-and-attrs:
1195
Finally the :class:`TestCase` provides the following methods and attributes:
1198
.. method:: fail(msg=None)
1200
Signals a test failure unconditionally, with *msg* or ``None`` for
1204
.. attribute:: failureException
1206
This class attribute gives the exception raised by the test method. If a
1207
test framework needs to use a specialized exception, possibly to carry
1208
additional information, it must subclass this exception in order to "play
1209
fair" with the framework. The initial value of this attribute is
1210
:exc:`AssertionError`.
1213
.. attribute:: longMessage
1215
If set to ``True`` then any explicit failure message you pass in to the
1216
:ref:`assert methods <assert-methods>` will be appended to the end of the
1217
normal failure message. The normal messages contain useful information
1218
about the objects involved, for example the message from assertEqual
1219
shows you the repr of the two unequal objects. Setting this attribute
1220
to ``True`` allows you to have a custom error message in addition to the
1223
This attribute defaults to ``False``, meaning that a custom message passed
1224
to an assert method will silence the normal message.
1226
The class setting can be overridden in individual tests by assigning an
1227
instance attribute to ``True`` or ``False`` before calling the assert methods.
1229
.. versionadded:: 2.7
1232
.. attribute:: maxDiff
1234
This attribute controls the maximum length of diffs output by assert
1235
methods that report diffs on failure. It defaults to 80*8 characters.
1236
Assert methods affected by this attribute are
1237
:meth:`assertSequenceEqual` (including all the sequence comparison
1238
methods that delegate to it), :meth:`assertDictEqual` and
1239
:meth:`assertMultiLineEqual`.
1241
Setting ``maxDiff`` to None means that there is no maximum length of
1244
.. versionadded:: 2.7
1247
Testing frameworks can use the following methods to collect information on
1251
.. method:: countTestCases()
1253
Return the number of tests represented by this test object. For
1254
:class:`TestCase` instances, this will always be ``1``.
1257
.. method:: defaultTestResult()
1259
Return an instance of the test result class that should be used for this
1260
test case class (if no other result instance is provided to the
1261
:meth:`run` method).
1263
For :class:`TestCase` instances, this will always be an instance of
1264
:class:`TestResult`; subclasses of :class:`TestCase` should override this
1270
Return a string identifying the specific test case. This is usually the
1271
full name of the test method, including the module and class name.
1274
.. method:: shortDescription()
1276
Returns a description of the test, or ``None`` if no description
1277
has been provided. The default implementation of this method
1278
returns the first line of the test method's docstring, if available,
1283
.. method:: addCleanup(function, *args, **kwargs)
1285
Add a function to be called after :meth:`tearDown` to cleanup resources
1286
used during the test. Functions will be called in reverse order to the
1287
order they are added (LIFO). They are called with any arguments and
1288
keyword arguments passed into :meth:`addCleanup` when they are
1291
If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called,
1292
then any cleanup functions added will still be called.
1294
.. versionadded:: 2.7
1297
.. method:: doCleanups()
1299
This method is called unconditionally after :meth:`tearDown`, or
1300
after :meth:`setUp` if :meth:`setUp` raises an exception.
1302
It is responsible for calling all the cleanup functions added by
1303
:meth:`addCleanup`. If you need cleanup functions to be called
1304
*prior* to :meth:`tearDown` then you can call :meth:`doCleanups`
1307
:meth:`doCleanups` pops methods off the stack of cleanup
1308
functions one at a time, so it can be called at any time.
1310
.. versionadded:: 2.7
1313
.. class:: FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None)
1315
This class implements the portion of the :class:`TestCase` interface which
1316
allows the test runner to drive the test, but does not provide the methods
1317
which test code can use to check and report errors. This is used to create
1318
test cases using legacy test code, allowing it to be integrated into a
1319
:mod:`unittest`-based test framework.
1325
For historical reasons, some of the :class:`TestCase` methods had one or more
1326
aliases that are now deprecated. The following table lists the correct names
1327
along with their deprecated aliases:
1329
============================== ===============================
1330
Method Name Deprecated alias(es)
1331
============================== ===============================
1332
:meth:`.assertEqual` failUnlessEqual, assertEquals
1333
:meth:`.assertNotEqual` failIfEqual
1334
:meth:`.assertTrue` failUnless, assert\_
1335
:meth:`.assertFalse` failIf
1336
:meth:`.assertRaises` failUnlessRaises
1337
:meth:`.assertAlmostEqual` failUnlessAlmostEqual
1338
:meth:`.assertNotAlmostEqual` failIfAlmostEqual
1339
============================== ===============================
1342
the aliases listed in the second column
1346
.. _testsuite-objects:
1351
.. class:: TestSuite(tests=())
1353
This class represents an aggregation of individual tests cases and test suites.
1354
The class presents the interface needed by the test runner to allow it to be run
1355
as any other test case. Running a :class:`TestSuite` instance is the same as
1356
iterating over the suite, running each test individually.
1358
If *tests* is given, it must be an iterable of individual test cases or other
1359
test suites that will be used to build the suite initially. Additional methods
1360
are provided to add test cases and suites to the collection later on.
1362
:class:`TestSuite` objects behave much like :class:`TestCase` objects, except
1363
they do not actually implement a test. Instead, they are used to aggregate
1364
tests into groups of tests that should be run together. Some additional
1365
methods are available to add tests to :class:`TestSuite` instances:
1368
.. method:: TestSuite.addTest(test)
1370
Add a :class:`TestCase` or :class:`TestSuite` to the suite.
1373
.. method:: TestSuite.addTests(tests)
1375
Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
1376
instances to this test suite.
1378
This is equivalent to iterating over *tests*, calling :meth:`addTest` for
1381
:class:`TestSuite` shares the following methods with :class:`TestCase`:
1384
.. method:: run(result)
1386
Run the tests associated with this suite, collecting the result into the
1387
test result object passed as *result*. Note that unlike
1388
:meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to
1394
Run the tests associated with this suite without collecting the
1395
result. This allows exceptions raised by the test to be propagated to the
1396
caller and can be used to support running tests under a debugger.
1399
.. method:: countTestCases()
1401
Return the number of tests represented by this test object, including all
1402
individual tests and sub-suites.
1405
.. method:: __iter__()
1407
Tests grouped by a :class:`TestSuite` are always accessed by iteration.
1408
Subclasses can lazily provide tests by overriding :meth:`__iter__`. Note
1409
that this method maybe called several times on a single suite
1410
(for example when counting tests or comparing for equality)
1411
so the tests returned must be the same for repeated iterations.
1413
.. versionchanged:: 2.7
1414
In earlier versions the :class:`TestSuite` accessed tests directly rather
1415
than through iteration, so overriding :meth:`__iter__` wasn't sufficient
1416
for providing tests.
1418
In the typical usage of a :class:`TestSuite` object, the :meth:`run` method
1419
is invoked by a :class:`TestRunner` rather than by the end-user test harness.
1422
Loading and running tests
1423
~~~~~~~~~~~~~~~~~~~~~~~~~
1425
.. class:: TestLoader()
1427
The :class:`TestLoader` class is used to create test suites from classes and
1428
modules. Normally, there is no need to create an instance of this class; the
1429
:mod:`unittest` module provides an instance that can be shared as
1430
:data:`unittest.defaultTestLoader`. Using a subclass or instance, however,
1431
allows customization of some configurable properties.
1433
:class:`TestLoader` objects have the following methods:
1436
.. method:: loadTestsFromTestCase(testCaseClass)
1438
Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
1439
:class:`testCaseClass`.
1442
.. method:: loadTestsFromModule(module)
1444
Return a suite of all tests cases contained in the given module. This
1445
method searches *module* for classes derived from :class:`TestCase` and
1446
creates an instance of the class for each test method defined for the
1451
While using a hierarchy of :class:`TestCase`\ -derived classes can be
1452
convenient in sharing fixtures and helper functions, defining test
1453
methods on base classes that are not intended to be instantiated
1454
directly does not play well with this method. Doing so, however, can
1455
be useful when the fixtures are different and defined in subclasses.
1457
If a module provides a ``load_tests`` function it will be called to
1458
load the tests. This allows modules to customize test loading.
1459
This is the `load_tests protocol`_.
1461
.. versionchanged:: 2.7
1462
Support for ``load_tests`` added.
1465
.. method:: loadTestsFromName(name, module=None)
1467
Return a suite of all tests cases given a string specifier.
1469
The specifier *name* is a "dotted name" that may resolve either to a
1470
module, a test case class, a test method within a test case class, a
1471
:class:`TestSuite` instance, or a callable object which returns a
1472
:class:`TestCase` or :class:`TestSuite` instance. These checks are
1473
applied in the order listed here; that is, a method on a possible test
1474
case class will be picked up as "a test method within a test case class",
1475
rather than "a callable object".
1477
For example, if you have a module :mod:`SampleTests` containing a
1478
:class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
1479
methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
1480
specifier ``'SampleTests.SampleTestCase'`` would cause this method to
1481
return a suite which will run all three test methods. Using the specifier
1482
``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test
1483
suite which will run only the :meth:`test_two` test method. The specifier
1484
can refer to modules and packages which have not been imported; they will
1485
be imported as a side-effect.
1487
The method optionally resolves *name* relative to the given *module*.
1490
.. method:: loadTestsFromNames(names, module=None)
1492
Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather
1493
than a single name. The return value is a test suite which supports all
1494
the tests defined for each name.
1497
.. method:: getTestCaseNames(testCaseClass)
1499
Return a sorted sequence of method names found within *testCaseClass*;
1500
this should be a subclass of :class:`TestCase`.
1503
.. method:: discover(start_dir, pattern='test*.py', top_level_dir=None)
1505
Find all the test modules by recursing into subdirectories from the
1506
specified start directory, and return a TestSuite object containing them.
1507
Only test files that match *pattern* will be loaded. (Using shell style
1508
pattern matching.) Only module names that are importable (i.e. are valid
1509
Python identifiers) will be loaded.
1511
All test modules must be importable from the top level of the project. If
1512
the start directory is not the top level directory then the top level
1513
directory must be specified separately.
1515
If importing a module fails, for example due to a syntax error, then this
1516
will be recorded as a single error and discovery will continue.
1518
If a test package name (directory with :file:`__init__.py`) matches the
1519
pattern then the package will be checked for a ``load_tests``
1520
function. If this exists then it will be called with *loader*, *tests*,
1523
If load_tests exists then discovery does *not* recurse into the package,
1524
``load_tests`` is responsible for loading all tests in the package.
1526
The pattern is deliberately not stored as a loader attribute so that
1527
packages can continue discovery themselves. *top_level_dir* is stored so
1528
``load_tests`` does not need to pass this argument in to
1529
``loader.discover()``.
1531
*start_dir* can be a dotted module name as well as a directory.
1533
.. versionadded:: 2.7
1535
The following attributes of a :class:`TestLoader` can be configured either by
1536
subclassing or assignment on an instance:
1539
.. attribute:: testMethodPrefix
1541
String giving the prefix of method names which will be interpreted as test
1542
methods. The default value is ``'test'``.
1544
This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
1548
.. attribute:: sortTestMethodsUsing
1550
Function to be used to compare method names when sorting them in
1551
:meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
1552
default value is the built-in :func:`cmp` function; the attribute can also
1553
be set to :const:`None` to disable the sort.
1556
.. attribute:: suiteClass
1558
Callable object that constructs a test suite from a list of tests. No
1559
methods on the resulting object are needed. The default value is the
1560
:class:`TestSuite` class.
1562
This affects all the :meth:`loadTestsFrom\*` methods.
1565
.. class:: TestResult
1567
This class is used to compile information about which tests have succeeded
1568
and which have failed.
1570
A :class:`TestResult` object stores the results of a set of tests. The
1571
:class:`TestCase` and :class:`TestSuite` classes ensure that results are
1572
properly recorded; test authors do not need to worry about recording the
1575
Testing frameworks built on top of :mod:`unittest` may want access to the
1576
:class:`TestResult` object generated by running a set of tests for reporting
1577
purposes; a :class:`TestResult` instance is returned by the
1578
:meth:`TestRunner.run` method for this purpose.
1580
:class:`TestResult` instances have the following attributes that will be of
1581
interest when inspecting the results of running a set of tests:
1584
.. attribute:: errors
1586
A list containing 2-tuples of :class:`TestCase` instances and strings
1587
holding formatted tracebacks. Each tuple represents a test which raised an
1588
unexpected exception.
1590
.. versionchanged:: 2.2
1591
Contains formatted tracebacks instead of :func:`sys.exc_info` results.
1594
.. attribute:: failures
1596
A list containing 2-tuples of :class:`TestCase` instances and strings
1597
holding formatted tracebacks. Each tuple represents a test where a failure
1598
was explicitly signalled using the :meth:`TestCase.assert\*` methods.
1600
.. versionchanged:: 2.2
1601
Contains formatted tracebacks instead of :func:`sys.exc_info` results.
1603
.. attribute:: skipped
1605
A list containing 2-tuples of :class:`TestCase` instances and strings
1606
holding the reason for skipping the test.
1608
.. versionadded:: 2.7
1610
.. attribute:: expectedFailures
1612
A list containing 2-tuples of :class:`TestCase` instances and strings
1613
holding formatted tracebacks. Each tuple represents an expected failure
1616
.. attribute:: unexpectedSuccesses
1618
A list containing :class:`TestCase` instances that were marked as expected
1619
failures, but succeeded.
1621
.. attribute:: shouldStop
1623
Set to ``True`` when the execution of tests should stop by :meth:`stop`.
1626
.. attribute:: testsRun
1628
The total number of tests run so far.
1631
.. attribute:: buffer
1633
If set to true, ``sys.stdout`` and ``sys.stderr`` will be buffered in between
1634
:meth:`startTest` and :meth:`stopTest` being called. Collected output will
1635
only be echoed onto the real ``sys.stdout`` and ``sys.stderr`` if the test
1636
fails or errors. Any output is also attached to the failure / error message.
1638
.. versionadded:: 2.7
1641
.. attribute:: failfast
1643
If set to true :meth:`stop` will be called on the first failure or error,
1644
halting the test run.
1646
.. versionadded:: 2.7
1649
.. method:: wasSuccessful()
1651
Return ``True`` if all tests run so far have passed, otherwise returns
1657
This method can be called to signal that the set of tests being run should
1658
be aborted by setting the :attr:`shouldStop` attribute to ``True``.
1659
:class:`TestRunner` objects should respect this flag and return without
1660
running any additional tests.
1662
For example, this feature is used by the :class:`TextTestRunner` class to
1663
stop the test framework when the user signals an interrupt from the
1664
keyboard. Interactive tools which provide :class:`TestRunner`
1665
implementations can use this in a similar manner.
1667
The following methods of the :class:`TestResult` class are used to maintain
1668
the internal data structures, and may be extended in subclasses to support
1669
additional reporting requirements. This is particularly useful in building
1670
tools which support interactive reporting while tests are being run.
1673
.. method:: startTest(test)
1675
Called when the test case *test* is about to be run.
1677
.. method:: stopTest(test)
1679
Called after the test case *test* has been executed, regardless of the
1682
.. method:: startTestRun()
1684
Called once before any tests are executed.
1686
.. versionadded:: 2.7
1689
.. method:: stopTestRun()
1691
Called once after all tests are executed.
1693
.. versionadded:: 2.7
1696
.. method:: addError(test, err)
1698
Called when the test case *test* raises an unexpected exception. *err* is a
1699
tuple of the form returned by :func:`sys.exc_info`: ``(type, value,
1702
The default implementation appends a tuple ``(test, formatted_err)`` to
1703
the instance's :attr:`errors` attribute, where *formatted_err* is a
1704
formatted traceback derived from *err*.
1707
.. method:: addFailure(test, err)
1709
Called when the test case *test* signals a failure. *err* is a tuple of
1710
the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
1712
The default implementation appends a tuple ``(test, formatted_err)`` to
1713
the instance's :attr:`failures` attribute, where *formatted_err* is a
1714
formatted traceback derived from *err*.
1717
.. method:: addSuccess(test)
1719
Called when the test case *test* succeeds.
1721
The default implementation does nothing.
1724
.. method:: addSkip(test, reason)
1726
Called when the test case *test* is skipped. *reason* is the reason the
1727
test gave for skipping.
1729
The default implementation appends a tuple ``(test, reason)`` to the
1730
instance's :attr:`skipped` attribute.
1733
.. method:: addExpectedFailure(test, err)
1735
Called when the test case *test* fails, but was marked with the
1736
:func:`expectedFailure` decorator.
1738
The default implementation appends a tuple ``(test, formatted_err)`` to
1739
the instance's :attr:`expectedFailures` attribute, where *formatted_err*
1740
is a formatted traceback derived from *err*.
1743
.. method:: addUnexpectedSuccess(test)
1745
Called when the test case *test* was marked with the
1746
:func:`expectedFailure` decorator, but succeeded.
1748
The default implementation appends the test to the instance's
1749
:attr:`unexpectedSuccesses` attribute.
1751
.. class:: TextTestResult(stream, descriptions, verbosity)
1753
A concrete implementation of :class:`TestResult` used by the
1754
:class:`TextTestRunner`.
1756
.. versionadded:: 2.7
1757
This class was previously named ``_TextTestResult``. The old name still
1758
exists as an alias but is deprecated.
1760
.. data:: defaultTestLoader
1762
Instance of the :class:`TestLoader` class intended to be shared. If no
1763
customization of the :class:`TestLoader` is needed, this instance can be used
1764
instead of repeatedly creating new instances.
1767
.. class:: TextTestRunner(stream=sys.stderr, descriptions=True, verbosity=1, \
1768
failfast=False, buffer=False, resultclass=None)
1770
A basic test runner implementation which prints results on standard error. It
1771
has a few configurable parameters, but is essentially very simple. Graphical
1772
applications which run test suites should provide alternate implementations.
1774
.. method:: _makeResult()
1776
This method returns the instance of ``TestResult`` used by :meth:`run`.
1777
It is not intended to be called directly, but can be overridden in
1778
subclasses to provide a custom ``TestResult``.
1780
``_makeResult()`` instantiates the class or callable passed in the
1781
``TextTestRunner`` constructor as the ``resultclass`` argument. It
1782
defaults to :class:`TextTestResult` if no ``resultclass`` is provided.
1783
The result class is instantiated with the following arguments::
1785
stream, descriptions, verbosity
1788
.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader[, exit[, verbosity[, failfast[, catchbreak[, buffer]]]]]]]]]])
1790
A command-line program that loads a set of tests from *module* and runs them;
1791
this is primarily for making test modules conveniently executable.
1792
The simplest use for this function is to include the following line at the
1793
end of a test script::
1795
if __name__ == '__main__':
1798
You can run tests with more detailed information by passing in the verbosity
1801
if __name__ == '__main__':
1802
unittest.main(verbosity=2)
1804
The *defaultTest* argument is the name of the test to run if no test names
1805
are specified via *argv*. If not specified or ``None`` and no test names are
1806
provided via *argv*, all tests found in *module* are run.
1808
The *argv* argument can be a list of options passed to the program, with the
1809
first element being the program name. If not specified or ``None``,
1810
the values of :data:`sys.argv` are used.
1812
The *testRunner* argument can either be a test runner class or an already
1813
created instance of it. By default ``main`` calls :func:`sys.exit` with
1814
an exit code indicating success or failure of the tests run.
1816
The *testLoader* argument has to be a :class:`TestLoader` instance,
1817
and defaults to :data:`defaultTestLoader`.
1819
``main`` supports being used from the interactive interpreter by passing in the
1820
argument ``exit=False``. This displays the result on standard output without
1821
calling :func:`sys.exit`::
1823
>>> from unittest import main
1824
>>> main(module='test_module', exit=False)
1826
The *failfast*, *catchbreak* and *buffer* parameters have the same
1827
effect as the same-name `command-line options`_.
1829
Calling ``main`` actually returns an instance of the ``TestProgram`` class.
1830
This stores the result of the tests run as the ``result`` attribute.
1832
.. versionchanged:: 2.7
1833
The *exit*, *verbosity*, *failfast*, *catchbreak* and *buffer*
1834
parameters were added.
1840
.. versionadded:: 2.7
1842
Modules or packages can customize how tests are loaded from them during normal
1843
test runs or test discovery by implementing a function called ``load_tests``.
1845
If a test module defines ``load_tests`` it will be called by
1846
:meth:`TestLoader.loadTestsFromModule` with the following arguments::
1848
load_tests(loader, standard_tests, None)
1850
It should return a :class:`TestSuite`.
1852
*loader* is the instance of :class:`TestLoader` doing the loading.
1853
*standard_tests* are the tests that would be loaded by default from the
1854
module. It is common for test modules to only want to add or remove tests
1855
from the standard set of tests.
1856
The third argument is used when loading packages as part of test discovery.
1858
A typical ``load_tests`` function that loads tests from a specific set of
1859
:class:`TestCase` classes may look like::
1861
test_cases = (TestCase1, TestCase2, TestCase3)
1863
def load_tests(loader, tests, pattern):
1865
for test_class in test_cases:
1866
tests = loader.loadTestsFromTestCase(test_class)
1867
suite.addTests(tests)
1870
If discovery is started, either from the command line or by calling
1871
:meth:`TestLoader.discover`, with a pattern that matches a package
1872
name then the package :file:`__init__.py` will be checked for ``load_tests``.
1876
The default pattern is ``'test*.py'``. This matches all Python files
1877
that start with ``'test'`` but *won't* match any test directories.
1879
A pattern like ``'test*'`` will match test packages as well as
1882
If the package :file:`__init__.py` defines ``load_tests`` then it will be
1883
called and discovery not continued into the package. ``load_tests``
1884
is called with the following arguments::
1886
load_tests(loader, standard_tests, pattern)
1888
This should return a :class:`TestSuite` representing all the tests
1889
from the package. (``standard_tests`` will only contain tests
1890
collected from :file:`__init__.py`.)
1892
Because the pattern is passed into ``load_tests`` the package is free to
1893
continue (and potentially modify) test discovery. A 'do nothing'
1894
``load_tests`` function for a test package would look like::
1896
def load_tests(loader, standard_tests, pattern):
1897
# top level directory cached on loader instance
1898
this_dir = os.path.dirname(__file__)
1899
package_tests = loader.discover(start_dir=this_dir, pattern=pattern)
1900
standard_tests.addTests(package_tests)
1901
return standard_tests
1904
Class and Module Fixtures
1905
-------------------------
1907
Class and module level fixtures are implemented in :class:`TestSuite`. When
1908
the test suite encounters a test from a new class then :meth:`tearDownClass`
1909
from the previous class (if there is one) is called, followed by
1910
:meth:`setUpClass` from the new class.
1912
Similarly if a test is from a different module from the previous test then
1913
``tearDownModule`` from the previous module is run, followed by
1914
``setUpModule`` from the new module.
1916
After all the tests have run the final ``tearDownClass`` and
1917
``tearDownModule`` are run.
1919
Note that shared fixtures do not play well with [potential] features like test
1920
parallelization and they break test isolation. They should be used with care.
1922
The default ordering of tests created by the unittest test loaders is to group
1923
all tests from the same modules and classes together. This will lead to
1924
``setUpClass`` / ``setUpModule`` (etc) being called exactly once per class and
1925
module. If you randomize the order, so that tests from different modules and
1926
classes are adjacent to each other, then these shared fixture functions may be
1927
called multiple times in a single test run.
1929
Shared fixtures are not intended to work with suites with non-standard
1930
ordering. A ``BaseTestSuite`` still exists for frameworks that don't want to
1931
support shared fixtures.
1933
If there are any exceptions raised during one of the shared fixture functions
1934
the test is reported as an error. Because there is no corresponding test
1935
instance an ``_ErrorHolder`` object (that has the same interface as a
1936
:class:`TestCase`) is created to represent the error. If you are just using
1937
the standard unittest test runner then this detail doesn't matter, but if you
1938
are a framework author it may be relevant.
1941
setUpClass and tearDownClass
1942
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1944
These must be implemented as class methods::
1948
class Test(unittest.TestCase):
1950
def setUpClass(cls):
1951
cls._connection = createExpensiveConnectionObject()
1954
def tearDownClass(cls):
1955
cls._connection.destroy()
1957
If you want the ``setUpClass`` and ``tearDownClass`` on base classes called
1958
then you must call up to them yourself. The implementations in
1959
:class:`TestCase` are empty.
1961
If an exception is raised during a ``setUpClass`` then the tests in the class
1962
are not run and the ``tearDownClass`` is not run. Skipped classes will not
1963
have ``setUpClass`` or ``tearDownClass`` run. If the exception is a
1964
:exc:`SkipTest` exception then the class will be reported as having been skipped
1965
instead of as an error.
1968
setUpModule and tearDownModule
1969
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1971
These should be implemented as functions::
1976
def tearDownModule():
1979
If an exception is raised in a ``setUpModule`` then none of the tests in the
1980
module will be run and the ``tearDownModule`` will not be run. If the exception is a
1981
:exc:`SkipTest` exception then the module will be reported as having been skipped
1982
instead of as an error.
1988
The :option:`-c/--catch <unittest -c>` command-line option to unittest,
1989
along with the ``catchbreak`` parameter to :func:`unittest.main()`, provide
1990
more friendly handling of control-C during a test run. With catch break
1991
behavior enabled control-C will allow the currently running test to complete,
1992
and the test run will then end and report all the results so far. A second
1993
control-c will raise a :exc:`KeyboardInterrupt` in the usual way.
1995
The control-c handling signal handler attempts to remain compatible with code or
1996
tests that install their own :const:`signal.SIGINT` handler. If the ``unittest``
1997
handler is called but *isn't* the installed :const:`signal.SIGINT` handler,
1998
i.e. it has been replaced by the system under test and delegated to, then it
1999
calls the default handler. This will normally be the expected behavior by code
2000
that replaces an installed handler and delegates to it. For individual tests
2001
that need ``unittest`` control-c handling disabled the :func:`removeHandler`
2002
decorator can be used.
2004
There are a few utility functions for framework authors to enable control-c
2005
handling functionality within test frameworks.
2007
.. function:: installHandler()
2009
Install the control-c handler. When a :const:`signal.SIGINT` is received
2010
(usually in response to the user pressing control-c) all registered results
2011
have :meth:`~TestResult.stop` called.
2013
.. versionadded:: 2.7
2015
.. function:: registerResult(result)
2017
Register a :class:`TestResult` object for control-c handling. Registering a
2018
result stores a weak reference to it, so it doesn't prevent the result from
2019
being garbage collected.
2021
Registering a :class:`TestResult` object has no side-effects if control-c
2022
handling is not enabled, so test frameworks can unconditionally register
2023
all results they create independently of whether or not handling is enabled.
2025
.. versionadded:: 2.7
2027
.. function:: removeResult(result)
2029
Remove a registered result. Once a result has been removed then
2030
:meth:`~TestResult.stop` will no longer be called on that result object in
2031
response to a control-c.
2033
.. versionadded:: 2.7
2035
.. function:: removeHandler(function=None)
2037
When called without arguments this function removes the control-c handler
2038
if it has been installed. This function can also be used as a test decorator
2039
to temporarily remove the handler whilst the test is being executed::
2041
@unittest.removeHandler
2042
def test_signal_handling(self):
2045
.. versionadded:: 2.7
added
removed
python-2.7.11-docs-html/_sources/library/unittest.txt
