~dkuhlman/python-training-materials/Materials

<h1>25.2. <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> — Test interactive Python examples<a class="headerlink" href="#module-doctest" title="Permalink to this headline">¶</a></h1>

The <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> module searches for pieces of text that look like interactive

Python sessions, and then executes those sessions to verify that they work

exactly as shown. There are several common ways to use doctest:

<li>To check that a module’s docstrings are up-to-date by verifying that all

interactive examples still work as documented.</li>

<li>To perform regression testing by verifying that interactive examples from a

test file or a test object work as expected.</li>

<li>To write tutorial documentation for a package, liberally illustrated with

input-output examples. Depending on whether the examples or the expository text

are emphasized, this has the flavor of “literate testing” or “executable

documentation”.</li>

</ul>

Here’s a complete but small example module:

<div class="highlight-python"><div class="highlight"><pre>"""

This is the "example" module.

The example module supplies one function, factorial(). For example,

>>> factorial(5)

120

100

"""

101

102

def factorial(n):

103

"""Return the factorial of n, an exact integer >= 0.

104

105

If the result is small enough to fit in an int, return an int.

106

Else return a long.

107

108

>>> [factorial(n) for n in range(6)]

109

[1, 1, 2, 6, 24, 120]

110

>>> [factorial(long(n)) for n in range(6)]

111

[1, 1, 2, 6, 24, 120]

112

>>> factorial(30)

113

265252859812191058636308480000000L

114

>>> factorial(30L)

115

265252859812191058636308480000000L

116

>>> factorial(-1)

117

Traceback (most recent call last):

118

...

119

ValueError: n must be >= 0

120

121

Factorials of floats are OK, but the float must be an exact integer:

122

>>> factorial(30.1)

123

Traceback (most recent call last):

124

...

125

ValueError: n must be exact integer

126

>>> factorial(30.0)

127

265252859812191058636308480000000L

128

129

It must also not be ridiculously large:

130

>>> factorial(1e100)

131

Traceback (most recent call last):

132

...

133

OverflowError: n too large

134

"""

135

136

import math

137

if not n >= 0:

138

raise ValueError("n must be >= 0")

139

if math.floor(n) != n:

140

raise ValueError("n must be exact integer")

141

if n+1 == n: # catch a value like 1e300

142

raise OverflowError("n too large")

143

result = 1

144

factor = 2

145

while factor <= n:

146

result *= factor

147

factor += 1

148

return result

149

150

151

if __name__ == "__main__":

152

import doctest

153

doctest.testmod()

154

</pre></div>

155

</div>

156

If you run <code class="file docutils literal">example.py</code> directly from the command line, <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a>

157

works its magic:

158

<div class="highlight-python"><div class="highlight"><pre>$ python example.py

159

160

</pre></div>

161

</div>

162

There’s no output! That’s normal, and it means all the examples worked. Pass

163

<code class="docutils literal">-v</code> to the script, and <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> prints a detailed log of what

164

it’s trying, and prints a summary at the end:

165

<div class="highlight-python"><div class="highlight"><pre>$ python example.py -v

166

Trying:

167

factorial(5)

168

Expecting:

169

120

170

171

Trying:

172

[factorial(n) for n in range(6)]

173

Expecting:

174

[1, 1, 2, 6, 24, 120]

175

176

Trying:

177

[factorial(long(n)) for n in range(6)]

178

Expecting:

179

[1, 1, 2, 6, 24, 120]

180

181

</pre></div>

182

</div>

183

And so on, eventually ending with:

184

<div class="highlight-python"><div class="highlight"><pre>Trying:

185

factorial(1e100)

186

Expecting:

187

Traceback (most recent call last):

188

...

189

OverflowError: n too large

190

191

2 items passed all tests:

192

1 tests in __main__

193

8 tests in __main__.factorial

194

9 tests in 2 items.

195

9 passed and 0 failed.

196

Test passed.

197

198

</pre></div>

199

</div>

200

That’s all you need to know to start making productive use of <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a>!

201

Jump in. The following sections provide full details. Note that there are many

202

examples of doctests in the standard Python test suite and libraries.

203

Especially useful examples can be found in the standard test file

204

<code class="file docutils literal">Lib/test/test_doctest.py</code>.

205

206

<h2>25.2.1. Simple Usage: Checking Examples in Docstrings<a class="headerlink" href="#simple-usage-checking-examples-in-docstrings" title="Permalink to this headline">¶</a></h2>

207

The simplest way to start using doctest (but not necessarily the way you’ll

208

continue to do it) is to end each module <code class="xref py py-mod docutils literal">M</code> with:

209

<div class="highlight-python"><div class="highlight"><pre>if __name__ == "__main__":

210

import doctest

211

doctest.testmod()

212

</pre></div>

213

</div>

214

<a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> then examines docstrings in module <code class="xref py py-mod docutils literal">M</code>.

215

Running the module as a script causes the examples in the docstrings to get

216

executed and verified:

217

<div class="highlight-python"><div class="highlight"><pre>python M.py

218

</pre></div>

219

</div>

220

This won’t display anything unless an example fails, in which case the failing

221

example(s) and the cause(s) of the failure(s) are printed to stdout, and the

222

final line of output is <code class="docutils literal">***Test Failed*** N failures.</code>, where N is the

223

number of examples that failed.

224

Run it with the <code class="docutils literal">-v</code> switch instead:

225

<div class="highlight-python"><div class="highlight"><pre>python M.py -v

226

</pre></div>

227

</div>

228

and a detailed report of all examples tried is printed to standard output, along

229

with assorted summaries at the end.

230

You can force verbose mode by passing <code class="docutils literal">verbose=True</code> to <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a>, or

231

prohibit it by passing <code class="docutils literal">verbose=False</code>. In either of those cases,

232

<code class="docutils literal">sys.argv</code> is not examined by <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a> (so passing <code class="docutils literal">-v</code> or not

233

has no effect).

234

Since Python 2.6, there is also a command line shortcut for running

235

<a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a>. You can instruct the Python interpreter to run the doctest

236

module directly from the standard library and pass the module name(s) on the

237

command line:

238

<div class="highlight-python"><div class="highlight"><pre>python -m doctest -v example.py

239

</pre></div>

240

</div>

241

This will import <code class="file docutils literal">example.py</code> as a standalone module and run

242

<a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a> on it. Note that this may not work correctly if the file is

243

part of a package and imports other submodules from that package.

244

For more information on <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a>, see section <a class="reference internal" href="#doctest-basic-api">Basic API</a>.

245

</div>

246

247

<h2>25.2.2. Simple Usage: Checking Examples in a Text File<a class="headerlink" href="#simple-usage-checking-examples-in-a-text-file" title="Permalink to this headline">¶</a></h2>

248

Another simple application of doctest is testing interactive examples in a text

249

file. This can be done with the <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a> function:

250

<div class="highlight-python"><div class="highlight"><pre>import doctest

251

doctest.testfile("example.txt")

252

</pre></div>

253

</div>

254

That short script executes and verifies any interactive Python examples

255

contained in the file <code class="file docutils literal">example.txt</code>. The file content is treated as if it

256

were a single giant docstring; the file doesn’t need to contain a Python

257

program! For example, perhaps <code class="file docutils literal">example.txt</code> contains this:

258

<div class="highlight-python"><div class="highlight"><pre>The ``example`` module

259

======================

260

261

Using ``factorial``

262

-------------------

263

264

This is an example text file in reStructuredText format. First import

265

``factorial`` from the ``example`` module:

266

267

>>> from example import factorial

268

269

Now use it:

270

271

>>> factorial(6)

272

120

273

</pre></div>

274

</div>

275

Running <code class="docutils literal">doctest.testfile("example.txt")</code> then finds the error in this

276

documentation:

277

<div class="highlight-python"><div class="highlight"><pre>File "./example.txt", line 14, in example.txt

278

Failed example:

279

factorial(6)

280

Expected:

281

120

282

Got:

283

720

284

</pre></div>

285

</div>

286

As with <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a>, <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a> won’t display anything unless an

287

example fails. If an example does fail, then the failing example(s) and the

288

cause(s) of the failure(s) are printed to stdout, using the same format as

289

<a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a>.

290

By default, <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a> looks for files in the calling module’s directory.

291

See section <a class="reference internal" href="#doctest-basic-api">Basic API</a> for a description of the optional arguments

292

that can be used to tell it to look for files in other locations.

293

Like <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a>, <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a>‘s verbosity can be set with the

294

<code class="docutils literal">-v</code> command-line switch or with the optional keyword argument

295

verbose.

296

Since Python 2.6, there is also a command line shortcut for running

297

<a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a>. You can instruct the Python interpreter to run the doctest

298

module directly from the standard library and pass the file name(s) on the

299

command line:

300

<div class="highlight-python"><div class="highlight"><pre>python -m doctest -v example.txt

301

</pre></div>

302

</div>

303

Because the file name does not end with <code class="file docutils literal">.py</code>, <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> infers that

304

it must be run with <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a>, not <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a>.

305

For more information on <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a>, see section <a class="reference internal" href="#doctest-basic-api">Basic API</a>.

306

</div>

307

308

<h2>25.2.3. How It Works<a class="headerlink" href="#how-it-works" title="Permalink to this headline">¶</a></h2>

309

This section examines in detail how doctest works: which docstrings it looks at,

310

how it finds interactive examples, what execution context it uses, how it

311

handles exceptions, and how option flags can be used to control its behavior.

312

This is the information that you need to know to write doctest examples; for

313

information about actually running doctest on these examples, see the following

314

sections.

315

316

<h3>25.2.3.1. Which Docstrings Are Examined?<a class="headerlink" href="#which-docstrings-are-examined" title="Permalink to this headline">¶</a></h3>

317

The module docstring, and all function, class and method docstrings are

318

searched. Objects imported into the module are not searched.

319

In addition, if <code class="docutils literal">M.__test__</code> exists and “is true”, it must be a dict, and each

320

entry maps a (string) name to a function object, class object, or string.

321

Function and class object docstrings found from <code class="docutils literal">M.__test__</code> are searched, and

322

strings are treated as if they were docstrings. In output, a key <code class="docutils literal">K</code> in

323

<code class="docutils literal">M.__test__</code> appears with name

324

<div class="highlight-python"><div class="highlight"><pre><name of M>.__test__.K

325

</pre></div>

326

</div>

327

Any classes found are recursively searched similarly, to test docstrings in

328

their contained methods and nested classes.

329

330

Changed in version 2.4: A “private name” concept is deprecated and no longer documented.

331

</div>

332

</div>

333

334

<h3>25.2.3.2. How are Docstring Examples Recognized?<a class="headerlink" href="#how-are-docstring-examples-recognized" title="Permalink to this headline">¶</a></h3>

335

In most cases a copy-and-paste of an interactive console session works fine,

336

but doctest isn’t trying to do an exact emulation of any specific Python shell.

337

<div class="highlight-python"><div class="highlight"><pre>>>> # comments are ignored

338

>>> x = 12

339

>>> x

340

12

341

>>> if x == 13:

342

... print "yes"

343

... else:

344

... print "no"

345

... print "NO"

346

... print "NO!!!"

347

...

348

no

349

NO

350

NO!!!

351

>>>

352

</pre></div>

353

</div>

354

Any expected output must immediately follow the final <code class="docutils literal">'>>> '</code> or <code class="docutils literal">'... '</code>

355

line containing the code, and the expected output (if any) extends to the next

356

<code class="docutils literal">'>>> '</code> or all-whitespace line.

357

The fine print:

358

<ul>

359

<li>Expected output cannot contain an all-whitespace line, since such a line is

360

taken to signal the end of expected output. If expected output does contain a

361

blank line, put <code class="docutils literal"><BLANKLINE></code> in your doctest example each place a blank line

362

is expected.

363

364

New in version 2.4: <code class="docutils literal"><BLANKLINE></code> was added; there was no way to use expected output containing

365

empty lines in previous versions.

366

</div>

367

</li>

368

<li>All hard tab characters are expanded to spaces, using 8-column tab stops.

369

Tabs in output generated by the tested code are not modified. Because any

370

hard tabs in the sample output are expanded, this means that if the code

371

output includes hard tabs, the only way the doctest can pass is if the

372

<a class="reference internal" href="#doctest.NORMALIZE_WHITESPACE" title="doctest.NORMALIZE_WHITESPACE"><code class="xref py py-const docutils literal">NORMALIZE_WHITESPACE</code></a> option or <a class="reference internal" href="#doctest-directives">directive</a>

373

is in effect.

374

Alternatively, the test can be rewritten to capture the output and compare it

375

to an expected value as part of the test. This handling of tabs in the

376

source was arrived at through trial and error, and has proven to be the least

377

error prone way of handling them. It is possible to use a different

378

algorithm for handling tabs by writing a custom <a class="reference internal" href="#doctest.DocTestParser" title="doctest.DocTestParser"><code class="xref py py-class docutils literal">DocTestParser</code></a> class.

379

380

Changed in version 2.4: Expanding tabs to spaces is new; previous versions tried to preserve hard tabs,

381

with confusing results.

382

</div>

383

</li>

384

<li>Output to stdout is captured, but not output to stderr (exception tracebacks

385

are captured via a different means).

386

</li>

387

<li>If you continue a line via backslashing in an interactive session, or for any

388

other reason use a backslash, you should use a raw docstring, which will

389

preserve your backslashes exactly as you type them:

390

<div class="highlight-python"><div class="highlight"><pre>>>> def f(x):

391

... r'''Backslashes in a raw docstring: m\n'''

392

>>> print f.__doc__

393

Backslashes in a raw docstring: m\n

394

</pre></div>

395

</div>

396

Otherwise, the backslash will be interpreted as part of the string. For example,

397

the <code class="docutils literal">\n</code> above would be interpreted as a newline character. Alternatively, you

398

can double each backslash in the doctest version (and not use a raw string):

399

400

... '''Backslashes in a raw docstring: m\\n'''

401

>>> print f.__doc__

402

Backslashes in a raw docstring: m\n

403

</pre></div>

404

</div>

405

</li>

406

<li>The starting column doesn’t matter:

407

<div class="highlight-python"><div class="highlight"><pre>>>> assert "Easy!"

408

>>> import math

409

>>> math.floor(1.9)

410

1.0

411

</pre></div>

412

</div>

413

and as many leading whitespace characters are stripped from the expected output

414

as appeared in the initial <code class="docutils literal">'>>> '</code> line that started the example.

415

</li>

416

</ul>

417

</div>

418

419

<h3>25.2.3.3. What’s the Execution Context?<a class="headerlink" href="#what-s-the-execution-context" title="Permalink to this headline">¶</a></h3>

420

By default, each time <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> finds a docstring to test, it uses a

421

shallow copy of <code class="xref py py-mod docutils literal">M</code>‘s globals, so that running tests doesn’t change the

422

module’s real globals, and so that one test in <code class="xref py py-mod docutils literal">M</code> can’t leave behind

423

crumbs that accidentally allow another test to work. This means examples can

424

freely use any names defined at top-level in <code class="xref py py-mod docutils literal">M</code>, and names defined earlier

425

in the docstring being run. Examples cannot see names defined in other

426

docstrings.

427

You can force use of your own dict as the execution context by passing

428

<code class="docutils literal">globs=your_dict</code> to <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a> or <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a> instead.

429

</div>

430

431

<h3>25.2.3.4. What About Exceptions?<a class="headerlink" href="#what-about-exceptions" title="Permalink to this headline">¶</a></h3>

432

No problem, provided that the traceback is the only output produced by the

433

example: just paste in the traceback. <a class="footnote-reference" href="#id2" id="id1">[1]</a> Since tracebacks contain details

434

that are likely to change rapidly (for example, exact file paths and line

435

numbers), this is one case where doctest works hard to be flexible in what it

436

accepts.

437

Simple example:

438

<div class="highlight-python"><div class="highlight"><pre>>>> [1, 2, 3].remove(42)

439

Traceback (most recent call last):

440

File "<stdin>", line 1, in ?

441

ValueError: list.remove(x): x not in list

442

</pre></div>

443

</div>

444

That doctest succeeds if <a class="reference internal" href="exceptions.html#exceptions.ValueError" title="exceptions.ValueError"><code class="xref py py-exc docutils literal">ValueError</code></a> is raised, with the <code class="docutils literal">list.remove(x):

445

x not in list</code> detail as shown.

446

The expected output for an exception must start with a traceback header, which

447

may be either of the following two lines, indented the same as the first line of

448

the example:

449

<div class="highlight-python"><div class="highlight"><pre>Traceback (most recent call last):

450

Traceback (innermost last):

451

</pre></div>

452

</div>

453

The traceback header is followed by an optional traceback stack, whose contents

454

are ignored by doctest. The traceback stack is typically omitted, or copied

455

verbatim from an interactive session.

456

The traceback stack is followed by the most interesting part: the line(s)

457

containing the exception type and detail. This is usually the last line of a

458

traceback, but can extend across multiple lines if the exception has a

459

multi-line detail:

460

<div class="highlight-python"><div class="highlight"><pre>>>> raise ValueError('multi\n line\ndetail')

461

Traceback (most recent call last):

462

File "<stdin>", line 1, in ?

463

ValueError: multi

464

line

465

detail

466

</pre></div>

467

</div>

468

The last three lines (starting with <a class="reference internal" href="exceptions.html#exceptions.ValueError" title="exceptions.ValueError"><code class="xref py py-exc docutils literal">ValueError</code></a>) are compared against the

469

exception’s type and detail, and the rest are ignored.

470

471

Changed in version 2.4: Previous versions were unable to handle multi-line exception details.

472

</div>

473

Best practice is to omit the traceback stack, unless it adds significant

474

documentation value to the example. So the last example is probably better as:

475

476

Traceback (most recent call last):

477

...

478

ValueError: multi

479

line

480

detail

481

</pre></div>

482

</div>

483

Note that tracebacks are treated very specially. In particular, in the

484

rewritten example, the use of <code class="docutils literal">...</code> is independent of doctest’s

485

486

could just as well be three (or three hundred) commas or digits, or an indented

487

transcript of a Monty Python skit.

488

Some details you should read once, but won’t need to remember:

489

<ul>

490

<li>Doctest can’t guess whether your expected output came from an exception

491

traceback or from ordinary printing. So, e.g., an example that expects

492

<code class="docutils literal">ValueError: 42 is prime</code> will pass whether <a class="reference internal" href="exceptions.html#exceptions.ValueError" title="exceptions.ValueError"><code class="xref py py-exc docutils literal">ValueError</code></a> is actually

493

raised or if the example merely prints that traceback text. In practice,

494

ordinary output rarely begins with a traceback header line, so this doesn’t

495

create real problems.

496

</li>

497

<li>Each line of the traceback stack (if present) must be indented further than

498

the first line of the example, or start with a non-alphanumeric character.

499

The first line following the traceback header indented the same and starting

500

with an alphanumeric is taken to be the start of the exception detail. Of

501

course this does the right thing for genuine tracebacks.

502

</li>

503

<li>When the <a class="reference internal" href="#doctest.IGNORE_EXCEPTION_DETAIL" title="doctest.IGNORE_EXCEPTION_DETAIL"><code class="xref py py-const docutils literal">IGNORE_EXCEPTION_DETAIL</code></a> doctest option is specified,

504

everything following the leftmost colon and any module information in the

505

exception name is ignored.

506

</li>

507

<li>The interactive shell omits the traceback header line for some

508

<a class="reference internal" href="exceptions.html#exceptions.SyntaxError" title="exceptions.SyntaxError"><code class="xref py py-exc docutils literal">SyntaxError</code></a>s. But doctest uses the traceback header line to

509

distinguish exceptions from non-exceptions. So in the rare case where you need

510

to test a <a class="reference internal" href="exceptions.html#exceptions.SyntaxError" title="exceptions.SyntaxError"><code class="xref py py-exc docutils literal">SyntaxError</code></a> that omits the traceback header, you will need to

511

manually add the traceback header line to your test example.

512

</li>

513

<li>For some <a class="reference internal" href="exceptions.html#exceptions.SyntaxError" title="exceptions.SyntaxError"><code class="xref py py-exc docutils literal">SyntaxError</code></a>s, Python displays the character position of the

514

syntax error, using a <code class="docutils literal">^</code> marker:

515

<div class="highlight-python"><div class="highlight"><pre>>>> 1 1

516

File "<stdin>", line 1

517

1 1

518

^

519

SyntaxError: invalid syntax

520

</pre></div>

521

</div>

522

Since the lines showing the position of the error come before the exception type

523

and detail, they are not checked by doctest. For example, the following test

524

would pass, even though it puts the <code class="docutils literal">^</code> marker in the wrong location:

525

<div class="highlight-python"><div class="highlight"><pre>>>> 1 1

526

File "<stdin>", line 1

527

1 1

528

^

529

SyntaxError: invalid syntax

530

</pre></div>

531

</div>

532

</li>

533

</ul>

534

</div>

535

536

<h3>25.2.3.5. Option Flags<a class="headerlink" href="#option-flags" title="Permalink to this headline">¶</a></h3>

537

A number of option flags control various aspects of doctest’s behavior.

538

Symbolic names for the flags are supplied as module constants, which can be

539

or’ed together and passed to various functions. The names can also be used in

540

<a class="reference internal" href="#doctest-directives">doctest directives</a>.

541

The first group of options define test semantics, controlling aspects of how

542

doctest decides whether actual output matches an example’s expected output:

543

544

545

<code class="descclassname">doctest.</code><code class="descname">DONT_ACCEPT_TRUE_FOR_1</code><a class="headerlink" href="#doctest.DONT_ACCEPT_TRUE_FOR_1" title="Permalink to this definition">¶</a></dt>

546

<dd>By default, if an expected output block contains just <code class="docutils literal">1</code>, an actual output

547

block containing just <code class="docutils literal">1</code> or just <code class="docutils literal">True</code> is considered to be a match, and

548

similarly for <code class="docutils literal">0</code> versus <code class="docutils literal">False</code>. When <a class="reference internal" href="#doctest.DONT_ACCEPT_TRUE_FOR_1" title="doctest.DONT_ACCEPT_TRUE_FOR_1"><code class="xref py py-const docutils literal">DONT_ACCEPT_TRUE_FOR_1</code></a> is

549

specified, neither substitution is allowed. The default behavior caters to that

550

Python changed the return type of many functions from integer to boolean;

551

doctests expecting “little integer” output still work in these cases. This

552

option will probably go away, but not for several years.

553

</dd></dl>

554

555

556

557

<code class="descclassname">doctest.</code><code class="descname">DONT_ACCEPT_BLANKLINE</code><a class="headerlink" href="#doctest.DONT_ACCEPT_BLANKLINE" title="Permalink to this definition">¶</a></dt>

558

<dd>By default, if an expected output block contains a line containing only the

559

string <code class="docutils literal"><BLANKLINE></code>, then that line will match a blank line in the actual

560

output. Because a genuinely blank line delimits the expected output, this is

561

the only way to communicate that a blank line is expected. When

562

563

</dd></dl>

564

565

566

567

<code class="descclassname">doctest.</code><code class="descname">NORMALIZE_WHITESPACE</code><a class="headerlink" href="#doctest.NORMALIZE_WHITESPACE" title="Permalink to this definition">¶</a></dt>

568

<dd>When specified, all sequences of whitespace (blanks and newlines) are treated as

569

equal. Any sequence of whitespace within the expected output will match any

570

sequence of whitespace within the actual output. By default, whitespace must

571

match exactly. <a class="reference internal" href="#doctest.NORMALIZE_WHITESPACE" title="doctest.NORMALIZE_WHITESPACE"><code class="xref py py-const docutils literal">NORMALIZE_WHITESPACE</code></a> is especially useful when a line of

572

expected output is very long, and you want to wrap it across multiple lines in

573

your source.

574

</dd></dl>

575

576

577

578

<code class="descclassname">doctest.</code><code class="descname">ELLIPSIS</code><a class="headerlink" href="#doctest.ELLIPSIS" title="Permalink to this definition">¶</a></dt>

579

<dd>When specified, an ellipsis marker (<code class="docutils literal">...</code>) in the expected output can match

580

any substring in the actual output. This includes substrings that span line

581

boundaries, and empty substrings, so it’s best to keep usage of this simple.

582

Complicated uses can lead to the same kinds of “oops, it matched too much!”

583

surprises that <code class="docutils literal">.*</code> is prone to in regular expressions.

584

</dd></dl>

585

586

587

588

<code class="descclassname">doctest.</code><code class="descname">IGNORE_EXCEPTION_DETAIL</code><a class="headerlink" href="#doctest.IGNORE_EXCEPTION_DETAIL" title="Permalink to this definition">¶</a></dt>

589

<dd>When specified, an example that expects an exception passes if an exception of

590

the expected type is raised, even if the exception detail does not match. For

591

example, an example expecting <code class="docutils literal">ValueError: 42</code> will pass if the actual

592

exception raised is <code class="docutils literal">ValueError: 3*14</code>, but will fail, e.g., if

593

<a class="reference internal" href="exceptions.html#exceptions.TypeError" title="exceptions.TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a> is raised.

594

It will also ignore the module name used in Python 3 doctest reports. Hence

595

both of these variations will work with the flag specified, regardless of

596

whether the test is run under Python 2.7 or Python 3.2 (or later versions):

597

<div class="highlight-python"><div class="highlight"><pre>>>> raise CustomError('message')

598

Traceback (most recent call last):

599

CustomError: message

600

601

>>> raise CustomError('message')

602

Traceback (most recent call last):

603

my_module.CustomError: message

604

</pre></div>

605

</div>

606

Note that <a class="reference internal" href="#doctest.ELLIPSIS" title="doctest.ELLIPSIS"><code class="xref py py-const docutils literal">ELLIPSIS</code></a> can also be used to ignore the

607

details of the exception message, but such a test may still fail based

608

on whether or not the module details are printed as part of the

609

exception name. Using <a class="reference internal" href="#doctest.IGNORE_EXCEPTION_DETAIL" title="doctest.IGNORE_EXCEPTION_DETAIL"><code class="xref py py-const docutils literal">IGNORE_EXCEPTION_DETAIL</code></a> and the details

610

from Python 2.3 is also the only clear way to write a doctest that doesn’t

611

care about the exception detail yet continues to pass under Python 2.3 or

612

earlier (those releases do not support <a class="reference internal" href="#doctest-directives">doctest directives</a> and ignore them as irrelevant comments). For example:

613

<div class="highlight-python"><div class="highlight"><pre>>>> (1, 2)[3] = 'moo'

614

Traceback (most recent call last):

615

File "<stdin>", line 1, in ?

616

TypeError: object doesn't support item assignment

617

</pre></div>

618

</div>

619

passes under Python 2.3 and later Python versions with the flag specified,

620

even though the detail

621

changed in Python 2.4 to say “does not” instead of “doesn’t”.

622

623

Changed in version 2.7: <a class="reference internal" href="#doctest.IGNORE_EXCEPTION_DETAIL" title="doctest.IGNORE_EXCEPTION_DETAIL"><code class="xref py py-const docutils literal">IGNORE_EXCEPTION_DETAIL</code></a> now also ignores any information

624

relating to the module containing the exception under test

625

</div>

626

</dd></dl>

627

628

629

630

<code class="descclassname">doctest.</code><code class="descname">SKIP</code><a class="headerlink" href="#doctest.SKIP" title="Permalink to this definition">¶</a></dt>

631

<dd>When specified, do not run the example at all. This can be useful in contexts

632

where doctest examples serve as both documentation and test cases, and an

633

example should be included for documentation purposes, but should not be

634

checked. E.g., the example’s output might be random; or the example might

635

depend on resources which would be unavailable to the test driver.

636

The SKIP flag can also be used for temporarily “commenting out” examples.

637

</dd></dl>

638

639

640

New in version 2.5.

641

</div>

642

643

644

<code class="descclassname">doctest.</code><code class="descname">COMPARISON_FLAGS</code><a class="headerlink" href="#doctest.COMPARISON_FLAGS" title="Permalink to this definition">¶</a></dt>

645

<dd>A bitmask or’ing together all the comparison flags above.

646

</dd></dl>

647

648

The second group of options controls how test failures are reported:

649

650

651

<code class="descclassname">doctest.</code><code class="descname">REPORT_UDIFF</code><a class="headerlink" href="#doctest.REPORT_UDIFF" title="Permalink to this definition">¶</a></dt>

652

<dd>When specified, failures that involve multi-line expected and actual outputs are

653

displayed using a unified diff.

654

</dd></dl>

655

656

657

658

<code class="descclassname">doctest.</code><code class="descname">REPORT_CDIFF</code><a class="headerlink" href="#doctest.REPORT_CDIFF" title="Permalink to this definition">¶</a></dt>

659

<dd>When specified, failures that involve multi-line expected and actual outputs

660

will be displayed using a context diff.

661

</dd></dl>

662

663

664

665

<code class="descclassname">doctest.</code><code class="descname">REPORT_NDIFF</code><a class="headerlink" href="#doctest.REPORT_NDIFF" title="Permalink to this definition">¶</a></dt>

666

<dd>When specified, differences are computed by <code class="docutils literal">difflib.Differ</code>, using the same

667

algorithm as the popular <code class="file docutils literal">ndiff.py</code> utility. This is the only method that

668

marks differences within lines as well as across lines. For example, if a line

669

of expected output contains digit <code class="docutils literal">1</code> where actual output contains letter

670

<code class="docutils literal">l</code>, a line is inserted with a caret marking the mismatching column positions.

671

</dd></dl>

672

673

674

675

<code class="descclassname">doctest.</code><code class="descname">REPORT_ONLY_FIRST_FAILURE</code><a class="headerlink" href="#doctest.REPORT_ONLY_FIRST_FAILURE" title="Permalink to this definition">¶</a></dt>

676

<dd>When specified, display the first failing example in each doctest, but suppress

677

output for all remaining examples. This will prevent doctest from reporting

678

correct examples that break because of earlier failures; but it might also hide

679

incorrect examples that fail independently of the first failure. When

680

681

still run, and still count towards the total number of failures reported; only

682

the output is suppressed.

683

</dd></dl>

684

685

686

687

<code class="descclassname">doctest.</code><code class="descname">REPORTING_FLAGS</code><a class="headerlink" href="#doctest.REPORTING_FLAGS" title="Permalink to this definition">¶</a></dt>

688

<dd>A bitmask or’ing together all the reporting flags above.

689

</dd></dl>

690

691

692

New in version 2.4: The constants

693

<a class="reference internal" href="#doctest.DONT_ACCEPT_BLANKLINE" title="doctest.DONT_ACCEPT_BLANKLINE"><code class="xref py py-const docutils literal">DONT_ACCEPT_BLANKLINE</code></a>, <a class="reference internal" href="#doctest.NORMALIZE_WHITESPACE" title="doctest.NORMALIZE_WHITESPACE"><code class="xref py py-const docutils literal">NORMALIZE_WHITESPACE</code></a>,

694

<a class="reference internal" href="#doctest.ELLIPSIS" title="doctest.ELLIPSIS"><code class="xref py py-const docutils literal">ELLIPSIS</code></a>, <a class="reference internal" href="#doctest.IGNORE_EXCEPTION_DETAIL" title="doctest.IGNORE_EXCEPTION_DETAIL"><code class="xref py py-const docutils literal">IGNORE_EXCEPTION_DETAIL</code></a>, <a class="reference internal" href="#doctest.REPORT_UDIFF" title="doctest.REPORT_UDIFF"><code class="xref py py-const docutils literal">REPORT_UDIFF</code></a>,

695

<a class="reference internal" href="#doctest.REPORT_CDIFF" title="doctest.REPORT_CDIFF"><code class="xref py py-const docutils literal">REPORT_CDIFF</code></a>, <a class="reference internal" href="#doctest.REPORT_NDIFF" title="doctest.REPORT_NDIFF"><code class="xref py py-const docutils literal">REPORT_NDIFF</code></a>,

696

<a class="reference internal" href="#doctest.REPORT_ONLY_FIRST_FAILURE" title="doctest.REPORT_ONLY_FIRST_FAILURE"><code class="xref py py-const docutils literal">REPORT_ONLY_FIRST_FAILURE</code></a>, <a class="reference internal" href="#doctest.COMPARISON_FLAGS" title="doctest.COMPARISON_FLAGS"><code class="xref py py-const docutils literal">COMPARISON_FLAGS</code></a> and

697

<a class="reference internal" href="#doctest.REPORTING_FLAGS" title="doctest.REPORTING_FLAGS"><code class="xref py py-const docutils literal">REPORTING_FLAGS</code></a> were added.

698

</div>

699

There’s also a way to register new option flag names, although this isn’t useful

700

unless you intend to extend <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> internals via subclassing:

701

702

703

<code class="descclassname">doctest.</code><code class="descname">register_optionflag</code>(name)<a class="headerlink" href="#doctest.register_optionflag" title="Permalink to this definition">¶</a></dt>

704

<dd>Create a new option flag with a given name, and return the new flag’s integer

705

value. <a class="reference internal" href="#doctest.register_optionflag" title="doctest.register_optionflag"><code class="xref py py-func docutils literal">register_optionflag()</code></a> can be used when subclassing

706

<a class="reference internal" href="#doctest.OutputChecker" title="doctest.OutputChecker"><code class="xref py py-class docutils literal">OutputChecker</code></a> or <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> to create new options that are

707

supported by your subclasses. <a class="reference internal" href="#doctest.register_optionflag" title="doctest.register_optionflag"><code class="xref py py-func docutils literal">register_optionflag()</code></a> should always be

708

called using the following idiom:

709

<div class="highlight-python"><div class="highlight"><pre>MY_FLAG = register_optionflag('MY_FLAG')

710

</pre></div>

711

</div>

712

713

New in version 2.4.

714

</div>

715

</dd></dl>

716

717

</div>

718

719

<h3>25.2.3.6. Directives<a class="headerlink" href="#directives" title="Permalink to this headline">¶</a></h3>

720

Doctest directives may be used to modify the <a class="reference internal" href="#doctest-options">option flags</a> for an individual example. Doctest directives are

721

special Python comments following an example’s source code:

722

<pre>

723

directive ::= "#" "doctest:" <a class="reference internal" href="#grammar-token-directive_options"><code class="xref docutils literal">directive_options</code></a>

724

directive_options ::= <a class="reference internal" href="#grammar-token-directive_option"><code class="xref docutils literal">directive_option</code></a> ("," <a class="reference internal" href="#grammar-token-directive_option"><code class="xref docutils literal">directive_option</code></a>)\*

725

directive_option ::= <a class="reference internal" href="#grammar-token-on_or_off"><code class="xref docutils literal">on_or_off</code></a> <a class="reference internal" href="#grammar-token-directive_option_name"><code class="xref docutils literal">directive_option_name</code></a>

726

on_or_off ::= "+" \| "-"

727

directive_option_name ::= "DONT_ACCEPT_BLANKLINE" \| "NORMALIZE_WHITESPACE" \| ...

728

</pre>

729

Whitespace is not allowed between the <code class="docutils literal">+</code> or <code class="docutils literal">-</code> and the directive option

730

name. The directive option name can be any of the option flag names explained

731

above.

732

An example’s doctest directives modify doctest’s behavior for that single

733

example. Use <code class="docutils literal">+</code> to enable the named behavior, or <code class="docutils literal">-</code> to disable it.

734

For example, this test passes:

735

736

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9,

737

10, 11, 12, 13, 14, 15, 16, 17, 18, 19]

738

</pre></div>

739

</div>

740

Without the directive it would fail, both because the actual output doesn’t have

741

two blanks before the single-digit list elements, and because the actual output

742

is on a single line. This test also passes, and also requires a directive to do

743

so:

744

745

[0, 1, ..., 18, 19]

746

</pre></div>

747

</div>

748

Multiple directives can be used on a single physical line, separated by

749

commas:

750

751

[0, 1, ..., 18, 19]

752

</pre></div>

753

</div>

754

If multiple directive comments are used for a single example, then they are

755

combined:

756

757

... # doctest: +NORMALIZE_WHITESPACE

758

[0, 1, ..., 18, 19]

759

</pre></div>

760

</div>

761

As the previous example shows, you can add <code class="docutils literal">...</code> lines to your example

762

containing only directives. This can be useful when an example is too long for

763

a directive to comfortably fit on the same line:

764

<div class="highlight-python"><div class="highlight"><pre>>>> print range(5) + range(10,20) + range(30,40) + range(50,60)

765

... # doctest: +ELLIPSIS

766

[0, ..., 4, 10, ..., 19, 30, ..., 39, 50, ..., 59]

767

</pre></div>

768

</div>

769

Note that since all options are disabled by default, and directives apply only

770

to the example they appear in, enabling options (via <code class="docutils literal">+</code> in a directive) is

771

usually the only meaningful choice. However, option flags can also be passed to

772

functions that run doctests, establishing different defaults. In such cases,

773

disabling an option via <code class="docutils literal">-</code> in a directive can be useful.

774

775

New in version 2.4: Support for doctest directives was added.

776

</div>

777

</div>

778

779

<h3>25.2.3.7. Warnings<a class="headerlink" href="#warnings" title="Permalink to this headline">¶</a></h3>

780

<a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> is serious about requiring exact matches in expected output. If

781

even a single character doesn’t match, the test fails. This will probably

782

surprise you a few times, as you learn exactly what Python does and doesn’t

783

guarantee about output. For example, when printing a dict, Python doesn’t

784

guarantee that the key-value pairs will be printed in any particular order, so a

785

test like

786

<div class="highlight-python"><div class="highlight"><pre>>>> foo()

787

{"Hermione": "hippogryph", "Harry": "broomstick"}

788

</pre></div>

789

</div>

790

is vulnerable! One workaround is to do

791

<div class="highlight-python"><div class="highlight"><pre>>>> foo() == {"Hermione": "hippogryph", "Harry": "broomstick"}

792

True

793

</pre></div>

794

</div>

795

instead. Another is to do

796

<div class="highlight-python"><div class="highlight"><pre>>>> d = foo().items()

797

>>> d.sort()

798

>>> d

799

[('Harry', 'broomstick'), ('Hermione', 'hippogryph')]

800

</pre></div>

801

</div>

802

There are others, but you get the idea.

803

Another bad idea is to print things that embed an object address, like

804

<div class="highlight-python"><div class="highlight"><pre>>>> id(1.0) # certain to fail some of the time

805

7948648

806

>>> class C: pass

807

>>> C() # the default repr() for instances embeds an address

808

<__main__.C instance at 0x00AC18F0>

809

</pre></div>

810

</div>

811

The <a class="reference internal" href="#doctest.ELLIPSIS" title="doctest.ELLIPSIS"><code class="xref py py-const docutils literal">ELLIPSIS</code></a> directive gives a nice approach for the last example:

812

<div class="highlight-python"><div class="highlight"><pre>>>> C() #doctest: +ELLIPSIS

813

<__main__.C instance at 0x...>

814

</pre></div>

815

</div>

816

Floating-point numbers are also subject to small output variations across

817

platforms, because Python defers to the platform C library for float formatting,

818

and C libraries vary widely in quality here.

819

<div class="highlight-python"><div class="highlight"><pre>>>> 1./7 # risky

820

0.14285714285714285

821

>>> print 1./7 # safer

822

0.142857142857

823

>>> print round(1./7, 6) # much safer

824

0.142857

825

</pre></div>

826

</div>

827

Numbers of the form <code class="docutils literal">I/2.**J</code> are safe across all platforms, and I often

828

contrive doctest examples to produce numbers of that form:

829

<div class="highlight-python"><div class="highlight"><pre>>>> 3./4 # utterly safe

830

0.75

831

</pre></div>

832

</div>

833

Simple fractions are also easier for people to understand, and that makes for

834

better documentation.

835

</div>

836

</div>

837

838

<h2>25.2.4. Basic API<a class="headerlink" href="#basic-api" title="Permalink to this headline">¶</a></h2>

839

The functions <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a> and <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a> provide a simple interface to

840

doctest that should be sufficient for most basic uses. For a less formal

841

introduction to these two functions, see sections <a class="reference internal" href="#doctest-simple-testmod">Simple Usage: Checking Examples in Docstrings</a>

842

and <a class="reference internal" href="#doctest-simple-testfile">Simple Usage: Checking Examples in a Text File</a>.

843

844

845

<code class="descclassname">doctest.</code><code class="descname">testfile</code>(filename[, module_relative][, name][, package][, globs][, verbose][, report][, optionflags][, extraglobs][, raise_on_error][, parser][, encoding])<a class="headerlink" href="#doctest.testfile" title="Permalink to this definition">¶</a></dt>

846

<dd>All arguments except filename are optional, and should be specified in keyword

847

form.

848

Test examples in the file named filename. Return <code class="docutils literal">(failure_count,

849

test_count)</code>.

850

Optional argument module_relative specifies how the filename should be

851

interpreted:

852

853

<li>If module_relative is <code class="docutils literal">True</code> (the default), then filename specifies an

854

OS-independent module-relative path. By default, this path is relative to the

855

calling module’s directory; but if the package argument is specified, then it

856

is relative to that package. To ensure OS-independence, filename should use

857

<code class="docutils literal">/</code> characters to separate path segments, and may not be an absolute path

858

(i.e., it may not begin with <code class="docutils literal">/</code>).</li>

859

<li>If module_relative is <code class="docutils literal">False</code>, then filename specifies an OS-specific

860

path. The path may be absolute or relative; relative paths are resolved with

861

respect to the current working directory.</li>

862

</ul>

863

Optional argument name gives the name of the test; by default, or if <code class="docutils literal">None</code>,

864

<code class="docutils literal">os.path.basename(filename)</code> is used.

865

Optional argument package is a Python package or the name of a Python package

866

whose directory should be used as the base directory for a module-relative

867

filename. If no package is specified, then the calling module’s directory is

868

used as the base directory for module-relative filenames. It is an error to

869

specify package if module_relative is <code class="docutils literal">False</code>.

870

Optional argument globs gives a dict to be used as the globals when executing

871

examples. A new shallow copy of this dict is created for the doctest, so its

872

examples start with a clean slate. By default, or if <code class="docutils literal">None</code>, a new empty dict

873

is used.

874

Optional argument extraglobs gives a dict merged into the globals used to

875

execute examples. This works like <a class="reference internal" href="stdtypes.html#dict.update" title="dict.update"><code class="xref py py-meth docutils literal">dict.update()</code></a>: if globs and

876

extraglobs have a common key, the associated value in extraglobs appears in

877

the combined dict. By default, or if <code class="docutils literal">None</code>, no extra globals are used. This

878

is an advanced feature that allows parameterization of doctests. For example, a

879

doctest can be written for a base class, using a generic name for the class,

880

then reused to test any number of subclasses by passing an extraglobs dict

881

mapping the generic name to the subclass to be tested.

882

Optional argument verbose prints lots of stuff if true, and prints only

883

failures if false; by default, or if <code class="docutils literal">None</code>, it’s true if and only if <code class="docutils literal">'-v'</code>

884

is in <code class="docutils literal">sys.argv</code>.

885

Optional argument report prints a summary at the end when true, else prints

886

nothing at the end. In verbose mode, the summary is detailed, else the summary

887

is very brief (in fact, empty if all tests passed).

888

Optional argument optionflags or’s together option flags. See section

889

<a class="reference internal" href="#doctest-options">Option Flags</a>.

890

Optional argument raise_on_error defaults to false. If true, an exception is

891

raised upon the first failure or unexpected exception in an example. This

892

allows failures to be post-mortem debugged. Default behavior is to continue

893

running examples.

894

Optional argument parser specifies a <a class="reference internal" href="#doctest.DocTestParser" title="doctest.DocTestParser"><code class="xref py py-class docutils literal">DocTestParser</code></a> (or subclass) that

895

should be used to extract tests from the files. It defaults to a normal parser

896

(i.e., <code class="docutils literal">DocTestParser()</code>).

897

Optional argument encoding specifies an encoding that should be used to

898

convert the file to unicode.

899

900

New in version 2.4.

901

</div>

902

903

Changed in version 2.5: The parameter encoding was added.

904

</div>

905

</dd></dl>

906

907

908

909

<code class="descclassname">doctest.</code><code class="descname">testmod</code>([m][, name][, globs][, verbose][, report][, optionflags][, extraglobs][, raise_on_error][, exclude_empty])<a class="headerlink" href="#doctest.testmod" title="Permalink to this definition">¶</a></dt>

910

<dd>All arguments are optional, and all except for m should be specified in

911

keyword form.

912

Test examples in docstrings in functions and classes reachable from module m

913

(or module <a class="reference internal" href="__main__.html#module-__main__" title="__main__: The environment where the top-level script is run."><code class="xref py py-mod docutils literal">__main__</code></a> if m is not supplied or is <code class="docutils literal">None</code>), starting with

914

<code class="docutils literal">m.__doc__</code>.

915

Also test examples reachable from dict <code class="docutils literal">m.__test__</code>, if it exists and is not

916

<code class="docutils literal">None</code>. <code class="docutils literal">m.__test__</code> maps names (strings) to functions, classes and

917

strings; function and class docstrings are searched for examples; strings are

918

searched directly, as if they were docstrings.

919

Only docstrings attached to objects belonging to module m are searched.

920

Return <code class="docutils literal">(failure_count, test_count)</code>.

921

Optional argument name gives the name of the module; by default, or if

922

<code class="docutils literal">None</code>, <code class="docutils literal">m.__name__</code> is used.

923

Optional argument exclude_empty defaults to false. If true, objects for which

924

no doctests are found are excluded from consideration. The default is a backward

925

compatibility hack, so that code still using <code class="xref py py-meth docutils literal">doctest.master.summarize()</code> in

926

conjunction with <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a> continues to get output for objects with no

927

tests. The exclude_empty argument to the newer <a class="reference internal" href="#doctest.DocTestFinder" title="doctest.DocTestFinder"><code class="xref py py-class docutils literal">DocTestFinder</code></a>

928

constructor defaults to true.

929

Optional arguments extraglobs, verbose, report, optionflags,

930

raise_on_error, and globs are the same as for function <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a>

931

above, except that globs defaults to <code class="docutils literal">m.__dict__</code>.

932

933

Changed in version 2.3: The parameter optionflags was added.

934

</div>

935

936

Changed in version 2.4: The parameters extraglobs, raise_on_error and exclude_empty were added.

937

</div>

938

939

Changed in version 2.5: The optional argument isprivate, deprecated in 2.4, was removed.

940

</div>

941

</dd></dl>

942

943

944

945

<code class="descclassname">doctest.</code><code class="descname">run_docstring_examples</code>(f, globs[, verbose][, name][, compileflags][, optionflags])<a class="headerlink" href="#doctest.run_docstring_examples" title="Permalink to this definition">¶</a></dt>

946

<dd>Test examples associated with object f; for example, f may be a string,

947

a module, a function, or a class object.

948

A shallow copy of dictionary argument globs is used for the execution context.

949

Optional argument name is used in failure messages, and defaults to

950

<code class="docutils literal">"NoName"</code>.

951

If optional argument verbose is true, output is generated even if there are no

952

failures. By default, output is generated only in case of an example failure.

953

Optional argument compileflags gives the set of flags that should be used by

954

the Python compiler when running the examples. By default, or if <code class="docutils literal">None</code>,

955

flags are deduced corresponding to the set of future features found in globs.

956

Optional argument optionflags works as for function <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a> above.

957

</dd></dl>

958

959

</div>

960

961

<h2>25.2.5. Unittest API<a class="headerlink" href="#unittest-api" title="Permalink to this headline">¶</a></h2>

962

As your collection of doctest’ed modules grows, you’ll want a way to run all

963

their doctests systematically. Prior to Python 2.4, <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> had a barely

964

documented <code class="xref py py-class docutils literal">Tester</code> class that supplied a rudimentary way to combine

965

doctests from multiple modules. <code class="xref py py-class docutils literal">Tester</code> was feeble, and in practice most

966

serious Python testing frameworks build on the <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> module, which

967

supplies many flexible ways to combine tests from multiple sources. So, in

968

Python 2.4, <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a>‘s <code class="xref py py-class docutils literal">Tester</code> class is deprecated, and

969

<a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> provides two functions that can be used to create <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a>

970

test suites from modules and text files containing doctests. To integrate with

971

<a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> test discovery, include a <code class="xref py py-func docutils literal">load_tests()</code> function in your

972

test module:

973

<div class="highlight-python"><div class="highlight"><pre>import unittest

974

import doctest

975

import my_module_with_doctests

976

977

def load_tests(loader, tests, ignore):

978

tests.addTests(doctest.DocTestSuite(my_module_with_doctests))

979

return tests

980

</pre></div>

981

</div>

982

There are two main functions for creating <a class="reference internal" href="unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">unittest.TestSuite</code></a> instances

983

from text files and modules with doctests:

984

985

986

<code class="descclassname">doctest.</code><code class="descname">DocFileSuite</code>(*paths, [module_relative][, package][, setUp][, tearDown][, globs][, optionflags][, parser][, encoding])<a class="headerlink" href="#doctest.DocFileSuite" title="Permalink to this definition">¶</a></dt>

987

<dd>Convert doctest tests from one or more text files to a

988

<a class="reference internal" href="unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">unittest.TestSuite</code></a>.

989

The returned <a class="reference internal" href="unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">unittest.TestSuite</code></a> is to be run by the unittest framework

990

and runs the interactive examples in each file. If an example in any file

991

fails, then the synthesized unit test fails, and a <code class="xref py py-exc docutils literal">failureException</code>

992

exception is raised showing the name of the file containing the test and a

993

(sometimes approximate) line number.

994

Pass one or more paths (as strings) to text files to be examined.

995

Options may be provided as keyword arguments:

996

Optional argument module_relative specifies how the filenames in paths

997

should be interpreted:

998

999

<li>If module_relative is <code class="docutils literal">True</code> (the default), then each filename in

1000

paths specifies an OS-independent module-relative path. By default, this

1001

path is relative to the calling module’s directory; but if the package

1002

argument is specified, then it is relative to that package. To ensure

1003

OS-independence, each filename should use <code class="docutils literal">/</code> characters to separate path

1004

segments, and may not be an absolute path (i.e., it may not begin with

1005

1006

<li>If module_relative is <code class="docutils literal">False</code>, then each filename in paths specifies

1007

an OS-specific path. The path may be absolute or relative; relative paths

1008

are resolved with respect to the current working directory.</li>

1009

</ul>

1010

Optional argument package is a Python package or the name of a Python

1011

package whose directory should be used as the base directory for

1012

module-relative filenames in paths. If no package is specified, then the

1013

calling module’s directory is used as the base directory for module-relative

1014

filenames. It is an error to specify package if module_relative is

1015

<code class="docutils literal">False</code>.

1016

Optional argument setUp specifies a set-up function for the test suite.

1017

This is called before running the tests in each file. The setUp function

1018

will be passed a <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> object. The setUp function can access the

1019

test globals as the globs attribute of the test passed.

1020

Optional argument tearDown specifies a tear-down function for the test

1021

suite. This is called after running the tests in each file. The tearDown

1022

function will be passed a <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> object. The setUp function can

1023

access the test globals as the globs attribute of the test passed.

1024

Optional argument globs is a dictionary containing the initial global

1025

variables for the tests. A new copy of this dictionary is created for each

1026

test. By default, globs is a new empty dictionary.

1027

Optional argument optionflags specifies the default doctest options for the

1028

tests, created by or-ing together individual option flags. See section

1029

<a class="reference internal" href="#doctest-options">Option Flags</a>. See function <a class="reference internal" href="#doctest.set_unittest_reportflags" title="doctest.set_unittest_reportflags"><code class="xref py py-func docutils literal">set_unittest_reportflags()</code></a> below

1030

for a better way to set reporting options.

1031

1032

that should be used to extract tests from the files. It defaults to a normal

1033

parser (i.e., <code class="docutils literal">DocTestParser()</code>).

1034

Optional argument encoding specifies an encoding that should be used to

1035

convert the file to unicode.

1036

1037

New in version 2.4.

1038

</div>

1039

1040

Changed in version 2.5: The global <code class="docutils literal">__file__</code> was added to the globals provided to doctests

1041

loaded from a text file using <a class="reference internal" href="#doctest.DocFileSuite" title="doctest.DocFileSuite"><code class="xref py py-func docutils literal">DocFileSuite()</code></a>.

1042

</div>

1043

1044

Changed in version 2.5: The parameter encoding was added.

1045

</div>

1046

1047

Note

1048

Unlike <a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a> and <a class="reference internal" href="#doctest.DocTestFinder" title="doctest.DocTestFinder"><code class="xref py py-class docutils literal">DocTestFinder</code></a>, this function raises

1049

a <a class="reference internal" href="exceptions.html#exceptions.ValueError" title="exceptions.ValueError"><code class="xref py py-exc docutils literal">ValueError</code></a> if module contains no docstrings. You can prevent

1050

this error by passing a <a class="reference internal" href="#doctest.DocTestFinder" title="doctest.DocTestFinder"><code class="xref py py-class docutils literal">DocTestFinder</code></a> instance as the

1051

test_finder argument with its exclude_empty keyword argument set

1052

to <code class="docutils literal">False</code>:

1053

<div class="last highlight-python"><div class="highlight"><pre>>>> finder = doctest.DocTestFinder(exclude_empty=False)

1054

>>> suite = doctest.DocTestSuite(test_finder=finder)

1055

</pre></div>

1056

</div>

1057

</div>

1058

</dd></dl>

1059

1060

1061

1062

<code class="descclassname">doctest.</code><code class="descname">DocTestSuite</code>([module][, globs][, extraglobs][, test_finder][, setUp][, tearDown][, checker])<a class="headerlink" href="#doctest.DocTestSuite" title="Permalink to this definition">¶</a></dt>

1063

<dd>Convert doctest tests for a module to a <a class="reference internal" href="unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">unittest.TestSuite</code></a>.

1064

1065

and runs each doctest in the module. If any of the doctests fail, then the

1066

synthesized unit test fails, and a <code class="xref py py-exc docutils literal">failureException</code> exception is raised

1067

showing the name of the file containing the test and a (sometimes approximate)

1068

line number.

1069

Optional argument module provides the module to be tested. It can be a module

1070

object or a (possibly dotted) module name. If not specified, the module calling

1071

this function is used.

1072

Optional argument globs is a dictionary containing the initial global

1073

variables for the tests. A new copy of this dictionary is created for each

1074

test. By default, globs is a new empty dictionary.

1075

Optional argument extraglobs specifies an extra set of global variables, which

1076

is merged into globs. By default, no extra globals are used.

1077

Optional argument test_finder is the <a class="reference internal" href="#doctest.DocTestFinder" title="doctest.DocTestFinder"><code class="xref py py-class docutils literal">DocTestFinder</code></a> object (or a

1078

drop-in replacement) that is used to extract doctests from the module.

1079

Optional arguments setUp, tearDown, and optionflags are the same as for

1080

function <a class="reference internal" href="#doctest.DocFileSuite" title="doctest.DocFileSuite"><code class="xref py py-func docutils literal">DocFileSuite()</code></a> above.

1081

1082

New in version 2.3.

1083

</div>

1084

1085

Changed in version 2.4: The parameters globs, extraglobs, test_finder, setUp, tearDown, and

1086

optionflags were added; this function now uses the same search technique as

1087

<a class="reference internal" href="#doctest.testmod" title="doctest.testmod"><code class="xref py py-func docutils literal">testmod()</code></a>.

1088

</div>

1089

</dd></dl>

1090

1091

Under the covers, <a class="reference internal" href="#doctest.DocTestSuite" title="doctest.DocTestSuite"><code class="xref py py-func docutils literal">DocTestSuite()</code></a> creates a <a class="reference internal" href="unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">unittest.TestSuite</code></a> out

1092

of <code class="xref py py-class docutils literal">doctest.DocTestCase</code> instances, and <code class="xref py py-class docutils literal">DocTestCase</code> is a

1093

subclass of <a class="reference internal" href="unittest.html#unittest.TestCase" title="unittest.TestCase"><code class="xref py py-class docutils literal">unittest.TestCase</code></a>. <code class="xref py py-class docutils literal">DocTestCase</code> isn’t documented

1094

here (it’s an internal detail), but studying its code can answer questions about

1095

the exact details of <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> integration.

1096

Similarly, <a class="reference internal" href="#doctest.DocFileSuite" title="doctest.DocFileSuite"><code class="xref py py-func docutils literal">DocFileSuite()</code></a> creates a <a class="reference internal" href="unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">unittest.TestSuite</code></a> out of

1097

<code class="xref py py-class docutils literal">doctest.DocFileCase</code> instances, and <code class="xref py py-class docutils literal">DocFileCase</code> is a subclass

1098

of <code class="xref py py-class docutils literal">DocTestCase</code>.

1099

So both ways of creating a <a class="reference internal" href="unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">unittest.TestSuite</code></a> run instances of

1100

<code class="xref py py-class docutils literal">DocTestCase</code>. This is important for a subtle reason: when you run

1101

<a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> functions yourself, you can control the <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> options in

1102

use directly, by passing option flags to <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> functions. However, if

1103

you’re writing a <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> framework, <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> ultimately controls

1104

when and how tests get run. The framework author typically wants to control

1105

1106

options), but there’s no way to pass options through <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> to

1107

1108

For this reason, <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> also supports a notion of <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a>

1109

reporting flags specific to <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> support, via this function:

1110

1111

1112

<code class="descclassname">doctest.</code><code class="descname">set_unittest_reportflags</code>(flags)<a class="headerlink" href="#doctest.set_unittest_reportflags" title="Permalink to this definition">¶</a></dt>

1113

<dd>Set the <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> reporting flags to use.

1114

Argument flags or’s together option flags. See section

1115

<a class="reference internal" href="#doctest-options">Option Flags</a>. Only “reporting flags” can be used.

1116

This is a module-global setting, and affects all future doctests run by module

1117

<a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a>: the <code class="xref py py-meth docutils literal">runTest()</code> method of <code class="xref py py-class docutils literal">DocTestCase</code> looks at

1118

the option flags specified for the test case when the <code class="xref py py-class docutils literal">DocTestCase</code>

1119

instance was constructed. If no reporting flags were specified (which is the

1120

typical and expected case), <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a>‘s <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> reporting flags are

1121

or’ed into the option flags, and the option flags so augmented are passed to the

1122

<a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> instance created to run the doctest. If any reporting

1123

flags were specified when the <code class="xref py py-class docutils literal">DocTestCase</code> instance was constructed,

1124

<a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a>‘s <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> reporting flags are ignored.

1125

The value of the <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> reporting flags in effect before the function

1126

was called is returned by the function.

1127

1128

New in version 2.4.

1129

</div>

1130

</dd></dl>

1131

1132

</div>

1133

1134

<h2>25.2.6. Advanced API<a class="headerlink" href="#advanced-api" title="Permalink to this headline">¶</a></h2>

1135

The basic API is a simple wrapper that’s intended to make doctest easy to use.

1136

It is fairly flexible, and should meet most users’ needs; however, if you

1137

require more fine-grained control over testing, or wish to extend doctest’s

1138

capabilities, then you should use the advanced API.

1139

The advanced API revolves around two container classes, which are used to store

1140

the interactive examples extracted from doctest cases:

1141

1142

<li><a class="reference internal" href="#doctest.Example" title="doctest.Example"><code class="xref py py-class docutils literal">Example</code></a>: A single Python <a class="reference internal" href="../glossary.html#term-statement">statement</a>, paired with its expected

1143

output.</li>

1144

<li><a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a>: A collection of <a class="reference internal" href="#doctest.Example" title="doctest.Example"><code class="xref py py-class docutils literal">Example</code></a>s, typically extracted

1145

from a single docstring or text file.</li>

1146

</ul>

1147

Additional processing classes are defined to find, parse, and run, and check

1148

doctest examples:

1149

1150

<li><a class="reference internal" href="#doctest.DocTestFinder" title="doctest.DocTestFinder"><code class="xref py py-class docutils literal">DocTestFinder</code></a>: Finds all docstrings in a given module, and uses a

1151

<a class="reference internal" href="#doctest.DocTestParser" title="doctest.DocTestParser"><code class="xref py py-class docutils literal">DocTestParser</code></a> to create a <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> from every docstring that

1152

contains interactive examples.</li>

1153

<li><a class="reference internal" href="#doctest.DocTestParser" title="doctest.DocTestParser"><code class="xref py py-class docutils literal">DocTestParser</code></a>: Creates a <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> object from a string (such

1154

as an object’s docstring).</li>

1155

<li><a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a>: Executes the examples in a <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a>, and uses

1156

an <a class="reference internal" href="#doctest.OutputChecker" title="doctest.OutputChecker"><code class="xref py py-class docutils literal">OutputChecker</code></a> to verify their output.</li>

1157

<li><a class="reference internal" href="#doctest.OutputChecker" title="doctest.OutputChecker"><code class="xref py py-class docutils literal">OutputChecker</code></a>: Compares the actual output from a doctest example with

1158

the expected output, and decides whether they match.</li>

1159

</ul>

1160

The relationships among these processing classes are summarized in the following

1161

diagram:

1162

<div class="highlight-python"><div class="highlight"><pre> list of:

1163

+------+ +---------+

1164

|module| --DocTestFinder-> | DocTest | --DocTestRunner-> results

1165

+------+ | ^ +---------+ | ^ (printed)

1166

1167

v | | ... | v |

1168

DocTestParser | Example | OutputChecker

1169

+---------+

1170

</pre></div>

1171

</div>

1172

1173

<h3>25.2.6.1. DocTest Objects<a class="headerlink" href="#doctest-objects" title="Permalink to this headline">¶</a></h3>

1174

1175

1176

class <code class="descclassname">doctest.</code><code class="descname">DocTest</code>(examples, globs, name, filename, lineno, docstring)<a class="headerlink" href="#doctest.DocTest" title="Permalink to this definition">¶</a></dt>

1177

<dd>A collection of doctest examples that should be run in a single namespace. The

1178

constructor arguments are used to initialize the attributes of the same names.

1179

1180

New in version 2.4.

1181

</div>

1182

<a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> defines the following attributes. They are initialized by

1183

the constructor, and should not be modified directly.

1184

1185

1186

<code class="descname">examples</code><a class="headerlink" href="#doctest.DocTest.examples" title="Permalink to this definition">¶</a></dt>

1187

<dd>A list of <a class="reference internal" href="#doctest.Example" title="doctest.Example"><code class="xref py py-class docutils literal">Example</code></a> objects encoding the individual interactive Python

1188

examples that should be run by this test.

1189

</dd></dl>

1190

1191

1192

1193

<code class="descname">globs</code><a class="headerlink" href="#doctest.DocTest.globs" title="Permalink to this definition">¶</a></dt>

1194

<dd>The namespace (aka globals) that the examples should be run in. This is a

1195

dictionary mapping names to values. Any changes to the namespace made by the

1196

examples (such as binding new variables) will be reflected in <a class="reference internal" href="#doctest.DocTest.globs" title="doctest.DocTest.globs"><code class="xref py py-attr docutils literal">globs</code></a>

1197

after the test is run.

1198

</dd></dl>

1199

1200

1201

1202

1203

<dd>A string name identifying the <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a>. Typically, this is the name

1204

of the object or file that the test was extracted from.

1205

</dd></dl>

1206

1207

1208

1209

<code class="descname">filename</code><a class="headerlink" href="#doctest.DocTest.filename" title="Permalink to this definition">¶</a></dt>

1210

<dd>The name of the file that this <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> was extracted from; or

1211

<code class="docutils literal">None</code> if the filename is unknown, or if the <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> was not

1212

extracted from a file.

1213

</dd></dl>

1214

1215

1216

1217

<code class="descname">lineno</code><a class="headerlink" href="#doctest.DocTest.lineno" title="Permalink to this definition">¶</a></dt>

1218

<dd>The line number within <a class="reference internal" href="#doctest.DocTest.filename" title="doctest.DocTest.filename"><code class="xref py py-attr docutils literal">filename</code></a> where this <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> begins, or

1219

<code class="docutils literal">None</code> if the line number is unavailable. This line number is zero-based

1220

with respect to the beginning of the file.

1221

</dd></dl>

1222

1223

1224

1225

<code class="descname">docstring</code><a class="headerlink" href="#doctest.DocTest.docstring" title="Permalink to this definition">¶</a></dt>

1226

<dd>The string that the test was extracted from, or ‘None’ if the string is

1227

unavailable, or if the test was not extracted from a string.

1228

</dd></dl>

1229

1230

</dd></dl>

1231

1232

</div>

1233

1234

<h3>25.2.6.2. Example Objects<a class="headerlink" href="#example-objects" title="Permalink to this headline">¶</a></h3>

1235

1236

1237

class <code class="descclassname">doctest.</code><code class="descname">Example</code>(source, want[, exc_msg][, lineno][, indent][, options])<a class="headerlink" href="#doctest.Example" title="Permalink to this definition">¶</a></dt>

1238

<dd>A single interactive example, consisting of a Python statement and its expected

1239

output. The constructor arguments are used to initialize the attributes of the

1240

same names.

1241

1242

New in version 2.4.

1243

</div>

1244

<a class="reference internal" href="#doctest.Example" title="doctest.Example"><code class="xref py py-class docutils literal">Example</code></a> defines the following attributes. They are initialized by

1245

the constructor, and should not be modified directly.

1246

1247

1248

<code class="descname">source</code><a class="headerlink" href="#doctest.Example.source" title="Permalink to this definition">¶</a></dt>

1249

<dd>A string containing the example’s source code. This source code consists of a

1250

single Python statement, and always ends with a newline; the constructor adds

1251

a newline when necessary.

1252

</dd></dl>

1253

1254

1255

1256

1257

<dd>The expected output from running the example’s source code (either from

1258

stdout, or a traceback in case of exception). <a class="reference internal" href="#doctest.Example.want" title="doctest.Example.want"><code class="xref py py-attr docutils literal">want</code></a> ends with a

1259

newline unless no output is expected, in which case it’s an empty string. The

1260

constructor adds a newline when necessary.

1261

</dd></dl>

1262

1263

1264

1265

1266

<dd>The exception message generated by the example, if the example is expected to

1267

generate an exception; or <code class="docutils literal">None</code> if it is not expected to generate an

1268

exception. This exception message is compared against the return value of

1269

<a class="reference internal" href="traceback.html#traceback.format_exception_only" title="traceback.format_exception_only"><code class="xref py py-func docutils literal">traceback.format_exception_only()</code></a>. <a class="reference internal" href="#doctest.Example.exc_msg" title="doctest.Example.exc_msg"><code class="xref py py-attr docutils literal">exc_msg</code></a> ends with a newline

1270

unless it’s <code class="docutils literal">None</code>. The constructor adds a newline if needed.

1271

</dd></dl>

1272

1273

1274

1275

<code class="descname">lineno</code><a class="headerlink" href="#doctest.Example.lineno" title="Permalink to this definition">¶</a></dt>

1276

<dd>The line number within the string containing this example where the example

1277

begins. This line number is zero-based with respect to the beginning of the

1278

containing string.

1279

</dd></dl>

1280

1281

1282

1283

<code class="descname">indent</code><a class="headerlink" href="#doctest.Example.indent" title="Permalink to this definition">¶</a></dt>

1284

<dd>The example’s indentation in the containing string, i.e., the number of space

1285

characters that precede the example’s first prompt.

1286

</dd></dl>

1287

1288

1289

1290

<code class="descname">options</code><a class="headerlink" href="#doctest.Example.options" title="Permalink to this definition">¶</a></dt>

1291

<dd>A dictionary mapping from option flags to <code class="docutils literal">True</code> or <code class="docutils literal">False</code>, which is used

1292

to override default options for this example. Any option flags not contained

1293

in this dictionary are left at their default value (as specified by the

1294

<a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a>‘s <code class="xref py py-attr docutils literal">optionflags</code>). By default, no options are set.

1295

</dd></dl>

1296

1297

</dd></dl>

1298

1299

</div>

1300

1301

<h3>25.2.6.3. DocTestFinder objects<a class="headerlink" href="#doctestfinder-objects" title="Permalink to this headline">¶</a></h3>

1302

1303

1304

class <code class="descclassname">doctest.</code><code class="descname">DocTestFinder</code>([verbose][, parser][, recurse][, exclude_empty])<a class="headerlink" href="#doctest.DocTestFinder" title="Permalink to this definition">¶</a></dt>

1305

<dd>A processing class used to extract the <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a>s that are relevant to

1306

a given object, from its docstring and the docstrings of its contained objects.

1307

1308

modules, functions, classes, methods, staticmethods, classmethods, and

1309

properties.

1310

The optional argument verbose can be used to display the objects searched by

1311

the finder. It defaults to <code class="docutils literal">False</code> (no output).

1312

The optional argument parser specifies the <a class="reference internal" href="#doctest.DocTestParser" title="doctest.DocTestParser"><code class="xref py py-class docutils literal">DocTestParser</code></a> object (or a

1313

drop-in replacement) that is used to extract doctests from docstrings.

1314

If the optional argument recurse is false, then <a class="reference internal" href="#doctest.DocTestFinder.find" title="doctest.DocTestFinder.find"><code class="xref py py-meth docutils literal">DocTestFinder.find()</code></a>

1315

will only examine the given object, and not any contained objects.

1316

If the optional argument exclude_empty is false, then

1317

<a class="reference internal" href="#doctest.DocTestFinder.find" title="doctest.DocTestFinder.find"><code class="xref py py-meth docutils literal">DocTestFinder.find()</code></a> will include tests for objects with empty docstrings.

1318

1319

New in version 2.4.

1320

</div>

1321

<a class="reference internal" href="#doctest.DocTestFinder" title="doctest.DocTestFinder"><code class="xref py py-class docutils literal">DocTestFinder</code></a> defines the following method:

1322

1323

1324

<code class="descname">find</code>(obj[, name][, module][, globs][, extraglobs])<a class="headerlink" href="#doctest.DocTestFinder.find" title="Permalink to this definition">¶</a></dt>

1325

<dd>Return a list of the <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a>s that are defined by obj‘s

1326

docstring, or by any of its contained objects’ docstrings.

1327

The optional argument name specifies the object’s name; this name will be

1328

used to construct names for the returned <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a>s. If name is

1329

not specified, then <code class="docutils literal">obj.__name__</code> is used.

1330

The optional parameter module is the module that contains the given object.

1331

If the module is not specified or is None, then the test finder will attempt

1332

to automatically determine the correct module. The object’s module is used:

1333

1334

<li>As a default namespace, if globs is not specified.</li>

1335

<li>To prevent the DocTestFinder from extracting DocTests from objects that are

1336

imported from other modules. (Contained objects with modules other than

1337

module are ignored.)</li>

1338

<li>To find the name of the file containing the object.</li>

1339

<li>To help find the line number of the object within its file.</li>

1340

</ul>

1341

If module is <code class="docutils literal">False</code>, no attempt to find the module will be made. This is

1342

obscure, of use mostly in testing doctest itself: if module is <code class="docutils literal">False</code>, or

1343

is <code class="docutils literal">None</code> but cannot be found automatically, then all objects are considered

1344

to belong to the (non-existent) module, so all contained objects will

1345

(recursively) be searched for doctests.

1346

The globals for each <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> is formed by combining globs and

1347

extraglobs (bindings in extraglobs override bindings in globs). A new

1348

shallow copy of the globals dictionary is created for each <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a>.

1349

If globs is not specified, then it defaults to the module’s __dict__, if

1350

specified, or <code class="docutils literal">{}</code> otherwise. If extraglobs is not specified, then it

1351

defaults to <code class="docutils literal">{}</code>.

1352

</dd></dl>

1353

1354

</dd></dl>

1355

1356

</div>

1357

1358

<h3>25.2.6.4. DocTestParser objects<a class="headerlink" href="#doctestparser-objects" title="Permalink to this headline">¶</a></h3>

1359

1360

1361

class <code class="descclassname">doctest.</code><code class="descname">DocTestParser</code><a class="headerlink" href="#doctest.DocTestParser" title="Permalink to this definition">¶</a></dt>

1362

<dd>A processing class used to extract interactive examples from a string, and use

1363

them to create a <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> object.

1364

1365

New in version 2.4.

1366

</div>

1367

<a class="reference internal" href="#doctest.DocTestParser" title="doctest.DocTestParser"><code class="xref py py-class docutils literal">DocTestParser</code></a> defines the following methods:

1368

1369

1370

<code class="descname">get_doctest</code>(string, globs, name, filename, lineno)<a class="headerlink" href="#doctest.DocTestParser.get_doctest" title="Permalink to this definition">¶</a></dt>

1371

<dd>Extract all doctest examples from the given string, and collect them into a

1372

<a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> object.

1373

globs, name, filename, and lineno are attributes for the new

1374

<a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> object. See the documentation for <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> for more

1375

information.

1376

</dd></dl>

1377

1378

1379

1380

<code class="descname">get_examples</code>(string[, name])<a class="headerlink" href="#doctest.DocTestParser.get_examples" title="Permalink to this definition">¶</a></dt>

1381

<dd>Extract all doctest examples from the given string, and return them as a list

1382

of <a class="reference internal" href="#doctest.Example" title="doctest.Example"><code class="xref py py-class docutils literal">Example</code></a> objects. Line numbers are 0-based. The optional argument

1383

name is a name identifying this string, and is only used for error messages.

1384

</dd></dl>

1385

1386

1387

1388

<code class="descname">parse</code>(string[, name])<a class="headerlink" href="#doctest.DocTestParser.parse" title="Permalink to this definition">¶</a></dt>

1389

<dd>Divide the given string into examples and intervening text, and return them as

1390

a list of alternating <a class="reference internal" href="#doctest.Example" title="doctest.Example"><code class="xref py py-class docutils literal">Example</code></a>s and strings. Line numbers for the

1391

<a class="reference internal" href="#doctest.Example" title="doctest.Example"><code class="xref py py-class docutils literal">Example</code></a>s are 0-based. The optional argument name is a name

1392

identifying this string, and is only used for error messages.

1393

</dd></dl>

1394

1395

</dd></dl>

1396

1397

</div>

1398

1399

<h3>25.2.6.5. DocTestRunner objects<a class="headerlink" href="#doctestrunner-objects" title="Permalink to this headline">¶</a></h3>

1400

1401

1402

class <code class="descclassname">doctest.</code><code class="descname">DocTestRunner</code>([checker][, verbose][, optionflags])<a class="headerlink" href="#doctest.DocTestRunner" title="Permalink to this definition">¶</a></dt>

1403

<dd>A processing class used to execute and verify the interactive examples in a

1404

<a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a>.

1405

The comparison between expected outputs and actual outputs is done by an

1406

1407

option flags; see section <a class="reference internal" href="#doctest-options">Option Flags</a> for more information. If the

1408

option flags are insufficient, then the comparison may also be customized by

1409

passing a subclass of <a class="reference internal" href="#doctest.OutputChecker" title="doctest.OutputChecker"><code class="xref py py-class docutils literal">OutputChecker</code></a> to the constructor.

1410

The test runner’s display output can be controlled in two ways. First, an output

1411

function can be passed to <code class="xref py py-meth docutils literal">TestRunner.run()</code>; this function will be called

1412

with strings that should be displayed. It defaults to <code class="docutils literal">sys.stdout.write</code>. If

1413

capturing the output is not sufficient, then the display output can be also

1414

customized by subclassing DocTestRunner, and overriding the methods

1415

<a class="reference internal" href="#doctest.DocTestRunner.report_start" title="doctest.DocTestRunner.report_start"><code class="xref py py-meth docutils literal">report_start()</code></a>, <a class="reference internal" href="#doctest.DocTestRunner.report_success" title="doctest.DocTestRunner.report_success"><code class="xref py py-meth docutils literal">report_success()</code></a>,

1416

<a class="reference internal" href="#doctest.DocTestRunner.report_unexpected_exception" title="doctest.DocTestRunner.report_unexpected_exception"><code class="xref py py-meth docutils literal">report_unexpected_exception()</code></a>, and <a class="reference internal" href="#doctest.DocTestRunner.report_failure" title="doctest.DocTestRunner.report_failure"><code class="xref py py-meth docutils literal">report_failure()</code></a>.

1417

The optional keyword argument checker specifies the <a class="reference internal" href="#doctest.OutputChecker" title="doctest.OutputChecker"><code class="xref py py-class docutils literal">OutputChecker</code></a>

1418

object (or drop-in replacement) that should be used to compare the expected

1419

outputs to the actual outputs of doctest examples.

1420

The optional keyword argument verbose controls the <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a>‘s

1421

verbosity. If verbose is <code class="docutils literal">True</code>, then information is printed about each

1422

example, as it is run. If verbose is <code class="docutils literal">False</code>, then only failures are

1423

printed. If verbose is unspecified, or <code class="docutils literal">None</code>, then verbose output is used

1424

iff the command-line switch <code class="docutils literal">-v</code> is used.

1425

The optional keyword argument optionflags can be used to control how the test

1426

runner compares expected output to actual output, and how it displays failures.

1427

For more information, see section <a class="reference internal" href="#doctest-options">Option Flags</a>.

1428

1429

New in version 2.4.

1430

</div>

1431

1432

1433

1434

<code class="descname">report_start</code>(out, test, example)<a class="headerlink" href="#doctest.DocTestRunner.report_start" title="Permalink to this definition">¶</a></dt>

1435

<dd>Report that the test runner is about to process the given example. This method

1436

is provided to allow subclasses of <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> to customize their

1437

output; it should not be called directly.

1438

example is the example about to be processed. test is the test

1439

containing example. out is the output function that was passed to

1440

<a class="reference internal" href="#doctest.DocTestRunner.run" title="doctest.DocTestRunner.run"><code class="xref py py-meth docutils literal">DocTestRunner.run()</code></a>.

1441

</dd></dl>

1442

1443

1444

1445

<code class="descname">report_success</code>(out, test, example, got)<a class="headerlink" href="#doctest.DocTestRunner.report_success" title="Permalink to this definition">¶</a></dt>

1446

<dd>Report that the given example ran successfully. This method is provided to

1447

allow subclasses of <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> to customize their output; it

1448

should not be called directly.

1449

example is the example about to be processed. got is the actual output

1450

from the example. test is the test containing example. out is the

1451

output function that was passed to <a class="reference internal" href="#doctest.DocTestRunner.run" title="doctest.DocTestRunner.run"><code class="xref py py-meth docutils literal">DocTestRunner.run()</code></a>.

1452

</dd></dl>

1453

1454

1455

1456

<code class="descname">report_failure</code>(out, test, example, got)<a class="headerlink" href="#doctest.DocTestRunner.report_failure" title="Permalink to this definition">¶</a></dt>

1457

<dd>Report that the given example failed. This method is provided to allow

1458

subclasses of <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> to customize their output; it should not

1459

be called directly.

1460

example is the example about to be processed. got is the actual output

1461

from the example. test is the test containing example. out is the

1462

1463

</dd></dl>

1464

1465

1466

1467

<code class="descname">report_unexpected_exception</code>(out, test, example, exc_info)<a class="headerlink" href="#doctest.DocTestRunner.report_unexpected_exception" title="Permalink to this definition">¶</a></dt>

1468

<dd>Report that the given example raised an unexpected exception. This method is

1469

provided to allow subclasses of <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> to customize their

1470

output; it should not be called directly.

1471

example is the example about to be processed. exc_info is a tuple

1472

containing information about the unexpected exception (as returned by

1473

<a class="reference internal" href="sys.html#sys.exc_info" title="sys.exc_info"><code class="xref py py-func docutils literal">sys.exc_info()</code></a>). test is the test containing example. out is the

1474

1475

</dd></dl>

1476

1477

1478

1479

<code class="descname">run</code>(test[, compileflags][, out][, clear_globs])<a class="headerlink" href="#doctest.DocTestRunner.run" title="Permalink to this definition">¶</a></dt>

1480

<dd>Run the examples in test (a <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> object), and display the

1481

results using the writer function out.

1482

The examples are run in the namespace <code class="docutils literal">test.globs</code>. If clear_globs is

1483

true (the default), then this namespace will be cleared after the test runs,

1484

to help with garbage collection. If you would like to examine the namespace

1485

after the test completes, then use clear_globs=False.

1486

compileflags gives the set of flags that should be used by the Python

1487

compiler when running the examples. If not specified, then it will default to

1488

the set of future-import flags that apply to globs.

1489

The output of each example is checked using the <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a>‘s

1490

output checker, and the results are formatted by the

1491

<code class="xref py py-meth docutils literal">DocTestRunner.report_*()</code> methods.

1492

</dd></dl>

1493

1494

1495

1496

<code class="descname">summarize</code>([verbose])<a class="headerlink" href="#doctest.DocTestRunner.summarize" title="Permalink to this definition">¶</a></dt>

1497

<dd>Print a summary of all the test cases that have been run by this DocTestRunner,

1498

and return a <a class="reference internal" href="../glossary.html#term-named-tuple">named tuple</a> <code class="docutils literal">TestResults(failed, attempted)</code>.

1499

The optional verbose argument controls how detailed the summary is. If the

1500

verbosity is not specified, then the <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a>‘s verbosity is

1501

used.

1502

1503

Changed in version 2.6: Use a named tuple.

1504

</div>

1505

</dd></dl>

1506

1507

</dd></dl>

1508

1509

</div>

1510

1511

<h3>25.2.6.6. OutputChecker objects<a class="headerlink" href="#outputchecker-objects" title="Permalink to this headline">¶</a></h3>

1512

1513

1514

class <code class="descclassname">doctest.</code><code class="descname">OutputChecker</code><a class="headerlink" href="#doctest.OutputChecker" title="Permalink to this definition">¶</a></dt>

1515

<dd>A class used to check the whether the actual output from a doctest example

1516

matches the expected output. <a class="reference internal" href="#doctest.OutputChecker" title="doctest.OutputChecker"><code class="xref py py-class docutils literal">OutputChecker</code></a> defines two methods:

1517

<a class="reference internal" href="#doctest.OutputChecker.check_output" title="doctest.OutputChecker.check_output"><code class="xref py py-meth docutils literal">check_output()</code></a>, which compares a given pair of outputs, and returns true

1518

if they match; and <a class="reference internal" href="#doctest.OutputChecker.output_difference" title="doctest.OutputChecker.output_difference"><code class="xref py py-meth docutils literal">output_difference()</code></a>, which returns a string describing

1519

the differences between two outputs.

1520

1521

New in version 2.4.

1522

</div>

1523

<a class="reference internal" href="#doctest.OutputChecker" title="doctest.OutputChecker"><code class="xref py py-class docutils literal">OutputChecker</code></a> defines the following methods:

1524

1525

1526

<code class="descname">check_output</code>(want, got, optionflags)<a class="headerlink" href="#doctest.OutputChecker.check_output" title="Permalink to this definition">¶</a></dt>

1527

<dd>Return <code class="docutils literal">True</code> iff the actual output from an example (got) matches the

1528

expected output (want). These strings are always considered to match if

1529

they are identical; but depending on what option flags the test runner is

1530

using, several non-exact match types are also possible. See section

1531

<a class="reference internal" href="#doctest-options">Option Flags</a> for more information about option flags.

1532

</dd></dl>

1533

1534

1535

1536

<code class="descname">output_difference</code>(example, got, optionflags)<a class="headerlink" href="#doctest.OutputChecker.output_difference" title="Permalink to this definition">¶</a></dt>

1537

<dd>Return a string describing the differences between the expected output for a

1538

given example (example) and the actual output (got). optionflags is the

1539

set of option flags used to compare want and got.

1540

</dd></dl>

1541

1542

</dd></dl>

1543

1544

</div>

1545

</div>

1546

1547

<h2>25.2.7. Debugging<a class="headerlink" href="#debugging" title="Permalink to this headline">¶</a></h2>

1548

Doctest provides several mechanisms for debugging doctest examples:

1549

<ul>

1550

<li>Several functions convert doctests to executable Python programs, which can be

1551

run under the Python debugger, <a class="reference internal" href="pdb.html#module-pdb" title="pdb: The Python debugger for interactive interpreters."><code class="xref py py-mod docutils literal">pdb</code></a>.

1552

</li>

1553

<li>The <a class="reference internal" href="#doctest.DebugRunner" title="doctest.DebugRunner"><code class="xref py py-class docutils literal">DebugRunner</code></a> class is a subclass of <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> that

1554

raises an exception for the first failing example, containing information about

1555

that example. This information can be used to perform post-mortem debugging on

1556

the example.

1557

</li>

1558

<li>The <a class="reference internal" href="unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> cases generated by <a class="reference internal" href="#doctest.DocTestSuite" title="doctest.DocTestSuite"><code class="xref py py-func docutils literal">DocTestSuite()</code></a> support the

1559

<a class="reference internal" href="#doctest.debug" title="doctest.debug"><code class="xref py py-meth docutils literal">debug()</code></a> method defined by <a class="reference internal" href="unittest.html#unittest.TestCase" title="unittest.TestCase"><code class="xref py py-class docutils literal">unittest.TestCase</code></a>.

1560

</li>

1561

<li>You can add a call to <a class="reference internal" href="pdb.html#pdb.set_trace" title="pdb.set_trace"><code class="xref py py-func docutils literal">pdb.set_trace()</code></a> in a doctest example, and you’ll

1562

drop into the Python debugger when that line is executed. Then you can inspect

1563

current values of variables, and so on. For example, suppose <code class="file docutils literal">a.py</code>

1564

contains just this module docstring:

1565

<div class="highlight-python"><div class="highlight"><pre>"""

1566

>>> def f(x):

1567

... g(x*2)

1568

>>> def g(x):

1569

... print x+3

1570

... import pdb; pdb.set_trace()

1571

>>> f(3)

1572

9

1573

"""

1574

</pre></div>

1575

</div>

1576

Then an interactive Python session may look like this:

1577

<div class="highlight-python"><div class="highlight"><pre>>>> import a, doctest

1578

>>> doctest.testmod(a)

1579

--Return--

1580

> <doctest a[1]>(3)g()->None

1581

-> import pdb; pdb.set_trace()

1582

(Pdb) list

1583

1 def g(x):

1584

2 print x+3

1585

3 -> import pdb; pdb.set_trace()

1586

[EOF]

1587

(Pdb) print x

1588

6

1589

(Pdb) step

1590

--Return--

1591

> <doctest a[0]>(2)f()->None

1592

-> g(x*2)

1593

(Pdb) list

1594

1 def f(x):

1595

2 -> g(x*2)

1596

[EOF]

1597

(Pdb) print x

1598

3

1599

(Pdb) step

1600

--Return--

1601

> <doctest a[2]>(1)?()->None

1602

-> f(3)

1603

(Pdb) cont

1604

(0, 3)

1605

>>>

1606

</pre></div>

1607

</div>

1608

1609

Changed in version 2.4: The ability to use <a class="reference internal" href="pdb.html#pdb.set_trace" title="pdb.set_trace"><code class="xref py py-func docutils literal">pdb.set_trace()</code></a> usefully inside doctests was added.

1610

</div>

1611

</li>

1612

</ul>

1613

Functions that convert doctests to Python code, and possibly run the synthesized

1614

code under the debugger:

1615

1616

1617

<code class="descclassname">doctest.</code><code class="descname">script_from_examples</code>(s)<a class="headerlink" href="#doctest.script_from_examples" title="Permalink to this definition">¶</a></dt>

1618

<dd>Convert text with examples to a script.

1619

Argument s is a string containing doctest examples. The string is converted

1620

to a Python script, where doctest examples in s are converted to regular code,

1621

and everything else is converted to Python comments. The generated script is

1622

returned as a string. For example,

1623

<div class="highlight-python"><div class="highlight"><pre>import doctest

1624

print doctest.script_from_examples(r"""

1625

Set x and y to 1 and 2.

1626

>>> x, y = 1, 2

1627

1628

Print their sum:

1629

>>> print x+y

1630

3

1631

""")

1632

</pre></div>

1633

</div>

1634

displays:

1635

<div class="highlight-python"><div class="highlight"><pre># Set x and y to 1 and 2.

1636

x, y = 1, 2

1637

#

1638

# Print their sum:

1639

print x+y

1640

# Expected:

1641

## 3

1642

</pre></div>

1643

</div>

1644

This function is used internally by other functions (see below), but can also be

1645

useful when you want to transform an interactive Python session into a Python

1646

script.

1647

1648

New in version 2.4.

1649

</div>

1650

</dd></dl>

1651

1652

1653

1654

<code class="descclassname">doctest.</code><code class="descname">testsource</code>(module, name)<a class="headerlink" href="#doctest.testsource" title="Permalink to this definition">¶</a></dt>

1655

<dd>Convert the doctest for an object to a script.

1656

Argument module is a module object, or dotted name of a module, containing the

1657

object whose doctests are of interest. Argument name is the name (within the

1658

module) of the object with the doctests of interest. The result is a string,

1659

containing the object’s docstring converted to a Python script, as described for

1660

<a class="reference internal" href="#doctest.script_from_examples" title="doctest.script_from_examples"><code class="xref py py-func docutils literal">script_from_examples()</code></a> above. For example, if module <code class="file docutils literal">a.py</code>

1661

contains a top-level function <code class="xref py py-func docutils literal">f()</code>, then

1662

<div class="highlight-python"><div class="highlight"><pre>import a, doctest

1663

print doctest.testsource(a, "a.f")

1664

</pre></div>

1665

</div>

1666

prints a script version of function <code class="xref py py-func docutils literal">f()</code>‘s docstring, with doctests

1667

converted to code, and the rest placed in comments.

1668

1669

New in version 2.3.

1670

</div>

1671

</dd></dl>

1672

1673

1674

1675

<code class="descclassname">doctest.</code><code class="descname">debug</code>(module, name[, pm])<a class="headerlink" href="#doctest.debug" title="Permalink to this definition">¶</a></dt>

1676

<dd>Debug the doctests for an object.

1677

The module and name arguments are the same as for function

1678

<a class="reference internal" href="#doctest.testsource" title="doctest.testsource"><code class="xref py py-func docutils literal">testsource()</code></a> above. The synthesized Python script for the named object’s

1679

docstring is written to a temporary file, and then that file is run under the

1680

control of the Python debugger, <a class="reference internal" href="pdb.html#module-pdb" title="pdb: The Python debugger for interactive interpreters."><code class="xref py py-mod docutils literal">pdb</code></a>.

1681

A shallow copy of <code class="docutils literal">module.__dict__</code> is used for both local and global

1682

execution context.

1683

Optional argument pm controls whether post-mortem debugging is used. If pm

1684

has a true value, the script file is run directly, and the debugger gets

1685

involved only if the script terminates via raising an unhandled exception. If

1686

it does, then post-mortem debugging is invoked, via <a class="reference internal" href="pdb.html#pdb.post_mortem" title="pdb.post_mortem"><code class="xref py py-func docutils literal">pdb.post_mortem()</code></a>,

1687

passing the traceback object from the unhandled exception. If pm is not

1688

specified, or is false, the script is run under the debugger from the start, via

1689

passing an appropriate <a class="reference internal" href="functions.html#execfile" title="execfile"><code class="xref py py-func docutils literal">execfile()</code></a> call to <a class="reference internal" href="pdb.html#pdb.run" title="pdb.run"><code class="xref py py-func docutils literal">pdb.run()</code></a>.

1690

1691

New in version 2.3.

1692

</div>

1693

1694

Changed in version 2.4: The pm argument was added.

1695

</div>

1696

</dd></dl>

1697

1698

1699

1700

<code class="descclassname">doctest.</code><code class="descname">debug_src</code>(src[, pm][, globs])<a class="headerlink" href="#doctest.debug_src" title="Permalink to this definition">¶</a></dt>

1701

<dd>Debug the doctests in a string.

1702

This is like function <a class="reference internal" href="#doctest.debug" title="doctest.debug"><code class="xref py py-func docutils literal">debug()</code></a> above, except that a string containing

1703

doctest examples is specified directly, via the src argument.

1704

Optional argument pm has the same meaning as in function <a class="reference internal" href="#doctest.debug" title="doctest.debug"><code class="xref py py-func docutils literal">debug()</code></a> above.

1705

Optional argument globs gives a dictionary to use as both local and global

1706

execution context. If not specified, or <code class="docutils literal">None</code>, an empty dictionary is used.

1707

If specified, a shallow copy of the dictionary is used.

1708

1709

New in version 2.4.

1710

</div>

1711

</dd></dl>

1712

1713

The <a class="reference internal" href="#doctest.DebugRunner" title="doctest.DebugRunner"><code class="xref py py-class docutils literal">DebugRunner</code></a> class, and the special exceptions it may raise, are of

1714

most interest to testing framework authors, and will only be sketched here. See

1715

the source code, and especially <a class="reference internal" href="#doctest.DebugRunner" title="doctest.DebugRunner"><code class="xref py py-class docutils literal">DebugRunner</code></a>‘s docstring (which is a

1716

doctest!) for more details:

1717

1718

1719

class <code class="descclassname">doctest.</code><code class="descname">DebugRunner</code>([checker][, verbose][, optionflags])<a class="headerlink" href="#doctest.DebugRunner" title="Permalink to this definition">¶</a></dt>

1720

<dd>A subclass of <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> that raises an exception as soon as a

1721

failure is encountered. If an unexpected exception occurs, an

1722

<a class="reference internal" href="#doctest.UnexpectedException" title="doctest.UnexpectedException"><code class="xref py py-exc docutils literal">UnexpectedException</code></a> exception is raised, containing the test, the

1723

example, and the original exception. If the output doesn’t match, then a

1724

<a class="reference internal" href="#doctest.DocTestFailure" title="doctest.DocTestFailure"><code class="xref py py-exc docutils literal">DocTestFailure</code></a> exception is raised, containing the test, the example, and

1725

the actual output.

1726

For information about the constructor parameters and methods, see the

1727

documentation for <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> in section <a class="reference internal" href="#doctest-advanced-api">Advanced API</a>.

1728

</dd></dl>

1729

1730

There are two exceptions that may be raised by <a class="reference internal" href="#doctest.DebugRunner" title="doctest.DebugRunner"><code class="xref py py-class docutils literal">DebugRunner</code></a> instances:

1731

1732

1733

exception <code class="descclassname">doctest.</code><code class="descname">DocTestFailure</code>(test, example, got)<a class="headerlink" href="#doctest.DocTestFailure" title="Permalink to this definition">¶</a></dt>

1734

<dd>An exception raised by <a class="reference internal" href="#doctest.DocTestRunner" title="doctest.DocTestRunner"><code class="xref py py-class docutils literal">DocTestRunner</code></a> to signal that a doctest example’s

1735

actual output did not match its expected output. The constructor arguments are

1736

used to initialize the attributes of the same names.

1737

</dd></dl>

1738

1739

<a class="reference internal" href="#doctest.DocTestFailure" title="doctest.DocTestFailure"><code class="xref py py-exc docutils literal">DocTestFailure</code></a> defines the following attributes:

1740

1741

1742

<code class="descclassname">DocTestFailure.</code><code class="descname">test</code><a class="headerlink" href="#doctest.DocTestFailure.test" title="Permalink to this definition">¶</a></dt>

1743

<dd>The <a class="reference internal" href="#doctest.DocTest" title="doctest.DocTest"><code class="xref py py-class docutils literal">DocTest</code></a> object that was being run when the example failed.

1744

</dd></dl>

1745

1746

1747

1748

<code class="descclassname">DocTestFailure.</code><code class="descname">example</code><a class="headerlink" href="#doctest.DocTestFailure.example" title="Permalink to this definition">¶</a></dt>

1749

<dd>The <a class="reference internal" href="#doctest.Example" title="doctest.Example"><code class="xref py py-class docutils literal">Example</code></a> that failed.

1750

</dd></dl>

1751

1752

1753

1754

<code class="descclassname">DocTestFailure.</code><code class="descname">got</code><a class="headerlink" href="#doctest.DocTestFailure.got" title="Permalink to this definition">¶</a></dt>

1755

<dd>The example’s actual output.

1756

</dd></dl>

1757

1758

1759

1760

exception <code class="descclassname">doctest.</code><code class="descname">UnexpectedException</code>(test, example, exc_info)<a class="headerlink" href="#doctest.UnexpectedException" title="Permalink to this definition">¶</a></dt>

1761

1762

example raised an unexpected exception. The constructor arguments are used

1763

to initialize the attributes of the same names.

1764

</dd></dl>

1765

1766

<a class="reference internal" href="#doctest.UnexpectedException" title="doctest.UnexpectedException"><code class="xref py py-exc docutils literal">UnexpectedException</code></a> defines the following attributes:

1767

1768

1769

<code class="descclassname">UnexpectedException.</code><code class="descname">test</code><a class="headerlink" href="#doctest.UnexpectedException.test" title="Permalink to this definition">¶</a></dt>

1770

1771

</dd></dl>

1772

1773

1774

1775

<code class="descclassname">UnexpectedException.</code><code class="descname">example</code><a class="headerlink" href="#doctest.UnexpectedException.example" title="Permalink to this definition">¶</a></dt>

1776

<dd>The <a class="reference internal" href="#doctest.Example" title="doctest.Example"><code class="xref py py-class docutils literal">Example</code></a> that failed.

1777

</dd></dl>

1778

1779

1780

1781

<code class="descclassname">UnexpectedException.</code><code class="descname">exc_info</code><a class="headerlink" href="#doctest.UnexpectedException.exc_info" title="Permalink to this definition">¶</a></dt>

1782

<dd>A tuple containing information about the unexpected exception, as returned by

1783

<a class="reference internal" href="sys.html#sys.exc_info" title="sys.exc_info"><code class="xref py py-func docutils literal">sys.exc_info()</code></a>.

1784

</dd></dl>

1785

1786

</div>

1787

1788

<h2>25.2.8. Soapbox<a class="headerlink" href="#soapbox" title="Permalink to this headline">¶</a></h2>

1789

As mentioned in the introduction, <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> has grown to have three primary

1790

uses:

1791

1792

<li>Checking examples in docstrings.</li>

1793

<li>Regression testing.</li>

1794

<li>Executable documentation / literate testing.</li>

1795

</ol>

1796

These uses have different requirements, and it is important to distinguish them.

1797

In particular, filling your docstrings with obscure test cases makes for bad

1798

documentation.

1799

When writing a docstring, choose docstring examples with care. There’s an art to

1800

this that needs to be learned—it may not be natural at first. Examples should

1801

add genuine value to the documentation. A good example can often be worth many

1802

words. If done with care, the examples will be invaluable for your users, and

1803

will pay back the time it takes to collect them many times over as the years go

1804

by and things change. I’m still amazed at how often one of my <a class="reference internal" href="#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a>

1805

examples stops working after a “harmless” change.

1806

Doctest also makes an excellent tool for regression testing, especially if you

1807

don’t skimp on explanatory text. By interleaving prose and examples, it becomes

1808

much easier to keep track of what’s actually being tested, and why. When a test

1809

fails, good prose can make it much easier to figure out what the problem is, and

1810

how it should be fixed. It’s true that you could write extensive comments in

1811

code-based testing, but few programmers do. Many have found that using doctest

1812

approaches instead leads to much clearer tests. Perhaps this is simply because

1813

doctest makes writing prose a little easier than writing code, while writing

1814

comments in code is a little harder. I think it goes deeper than just that:

1815

the natural attitude when writing a doctest-based test is that you want to

1816

explain the fine points of your software, and illustrate them with examples.

1817

This in turn naturally leads to test files that start with the simplest

1818

features, and logically progress to complications and edge cases. A coherent

1819

narrative is the result, instead of a collection of isolated functions that test

1820

isolated bits of functionality seemingly at random. It’s a different attitude,

1821

and produces different results, blurring the distinction between testing and

1822

explaining.

1823

Regression testing is best confined to dedicated objects or files. There are

1824

several options for organizing tests:

1825

1826

<li>Write text files containing test cases as interactive examples, and test the

1827

files using <a class="reference internal" href="#doctest.testfile" title="doctest.testfile"><code class="xref py py-func docutils literal">testfile()</code></a> or <a class="reference internal" href="#doctest.DocFileSuite" title="doctest.DocFileSuite"><code class="xref py py-func docutils literal">DocFileSuite()</code></a>. This is recommended,

1828

although is easiest to do for new projects, designed from the start to use

1829

doctest.</li>

1830

<li>Define functions named <code class="docutils literal">_regrtest_topic</code> that consist of single docstrings,

1831

containing test cases for the named topics. These functions can be included in

1832

the same file as the module, or separated out into a separate test file.</li>

1833

<li>Define a <code class="docutils literal">__test__</code> dictionary mapping from regression test topics to

1834

docstrings containing test cases.</li>

1835

</ul>

1836

When you have placed your tests in a module, the module can itself be the test

1837

runner. When a test fails, you can arrange for your test runner to re-run only

1838

the failing doctest while you debug the problem. Here is a minimal example of

1839

such a test runner:

1840

<div class="highlight-python"><div class="highlight"><pre>if __name__ == '__main__':

1841

import doctest

1842

flags = doctest.REPORT_NDIFF|doctest.REPORT_ONLY_FIRST_FAILURE

1843

if len(sys.argv) > 1:

1844

name = sys.argv[1]

1845

if name in globals():

1846

obj = globals()[name]

1847

else:

1848

obj = __test__[name]

1849

doctest.run_docstring_examples(obj, globals(), name=name,

1850

optionflags=flags)

1851

else:

1852

fail, total = doctest.testmod(optionflags=flags)

1853

print("{} failures out of {} tests".format(fail, total))

1854

</pre></div>

1855

</div>

1856

Footnotes

1857

1858

1859

1860

<tr><td class="label"><a class="fn-backref" href="#id1">[1]</a></td><td>Examples containing both expected output and an exception are not supported.

1861

Trying to guess where one ends and the other begins is too error-prone, and that

1862

also makes for a confusing test.</td></tr>

1863

</tbody>

1864

</table>

1865

</div>

1866

</div>

1867

1868

1869

</div>

1870

</div>

1871

</div>

1872

1873

1874

<h3><a href="../contents.html">Table Of Contents</a></h3>

1875

<ul>

1876

<li><a class="reference internal" href="#">25.2. <code class="docutils literal">doctest</code> — Test interactive Python examples</a><ul>

1877

<li><a class="reference internal" href="#simple-usage-checking-examples-in-docstrings">25.2.1. Simple Usage: Checking Examples in Docstrings</a></li>

1878

<li><a class="reference internal" href="#simple-usage-checking-examples-in-a-text-file">25.2.2. Simple Usage: Checking Examples in a Text File</a></li>

1879

<li><a class="reference internal" href="#how-it-works">25.2.3. How It Works</a><ul>

1880

<li><a class="reference internal" href="#which-docstrings-are-examined">25.2.3.1. Which Docstrings Are Examined?</a></li>

1881

<li><a class="reference internal" href="#how-are-docstring-examples-recognized">25.2.3.2. How are Docstring Examples Recognized?</a></li>

1882

<li><a class="reference internal" href="#what-s-the-execution-context">25.2.3.3. What’s the Execution Context?</a></li>

1883

<li><a class="reference internal" href="#what-about-exceptions">25.2.3.4. What About Exceptions?</a></li>

1884

<li><a class="reference internal" href="#option-flags">25.2.3.5. Option Flags</a></li>

1885

<li><a class="reference internal" href="#directives">25.2.3.6. Directives</a></li>

1886

<li><a class="reference internal" href="#warnings">25.2.3.7. Warnings</a></li>

1887

</ul>

1888

</li>

1889

<li><a class="reference internal" href="#basic-api">25.2.4. Basic API</a></li>

1890

<li><a class="reference internal" href="#unittest-api">25.2.5. Unittest API</a></li>

1891

<li><a class="reference internal" href="#advanced-api">25.2.6. Advanced API</a><ul>

1892

<li><a class="reference internal" href="#doctest-objects">25.2.6.1. DocTest Objects</a></li>

1893

<li><a class="reference internal" href="#example-objects">25.2.6.2. Example Objects</a></li>

1894

<li><a class="reference internal" href="#doctestfinder-objects">25.2.6.3. DocTestFinder objects</a></li>

1895

<li><a class="reference internal" href="#doctestparser-objects">25.2.6.4. DocTestParser objects</a></li>

1896

<li><a class="reference internal" href="#doctestrunner-objects">25.2.6.5. DocTestRunner objects</a></li>

1897

<li><a class="reference internal" href="#outputchecker-objects">25.2.6.6. OutputChecker objects</a></li>

1898

</ul>

1899

</li>

1900

<li><a class="reference internal" href="#debugging">25.2.7. Debugging</a></li>

1901

<li><a class="reference internal" href="#soapbox">25.2.8. Soapbox</a></li>

1902

</ul>

1903

</li>

1904

</ul>

1905

1906

<h4>Previous topic</h4>

1907

<a href="pydoc.html"

1908

title="previous chapter">25.1. <code class="docutils literal">pydoc</code> — Documentation generator and online help system</a>

1909

<h4>Next topic</h4>

1910

<a href="unittest.html"

1911

title="next chapter">25.3. <code class="docutils literal">unittest</code> — Unit testing framework</a>

1912

1913

1914

<li><a href="../bugs.html">Report a Bug</a></li>

1915

<li><a href="../_sources/library/doctest.txt"

1916

rel="nofollow">Show Source</a></li>

1917

</ul>

1918

1919

1920

<h3>Quick search</h3>

1921

1922

1923

1924

1925

1926

</form>

1927

1928

Enter search terms or a module, class or function name.

1929

1930

</div>

1931

1932

</div>

1933

</div>

1934

1935

</div>

1936

1937

<h3>Navigation</h3>

1938

<ul>

1939

1940

<a href="../genindex.html" title="General Index"

1941

>index</a></li>

1942

1943

<a href="../py-modindex.html" title="Python Module Index"

1944

>modules</a> |</li>

1945

1946

<a href="unittest.html" title="25.3. unittest — Unit testing framework"

1947

>next</a> |</li>

1948

1949

<a href="pydoc.html" title="25.1. pydoc — Documentation generator and online help system"

1950

>previous</a> |</li>

1951

<li><img src="../_static/py.png" alt=""

1952

style="vertical-align: middle; margin-top: -1px"/></li>

1953

<li><a href="https://www.python.org/">Python</a> »</li>

1954

<li>

1955

2.7.11

1956

<a href="../index.html">Documentation</a> »

1957

</li>

1958

1959

<li class="nav-item nav-item-1"><a href="index.html" >The Python Standard Library</a> »</li>

1960

<li class="nav-item nav-item-2"><a href="development.html" >25. Development Tools</a> »</li>

1961

</ul>

1962

</div>

1963

1964

1965

1966

The Python Software Foundation is a non-profit corporation.

1967

<a href="https://www.python.org/psf/donations/">Please donate.</a>

1968

1969

Last updated on Jan 23, 2016.

1970

<a href="../bugs.html">Found a bug</a>?

1971

1972

Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.3.3.

1973

</div>

1974

1975

</body>

1976

</html>

b'\\ No newline at end of file'

Older »