~dkuhlman/python-training-materials/Materials

are <a class="reference internal" href="../library/logging.html#logging.debug" title="logging.debug"><code class="xref py py-func docutils literal">debug()</code></a>, <a class="reference internal" href="../library/logging.html#logging.info" title="logging.info"><code class="xref py py-func docutils literal">info()</code></a>, <a class="reference internal" href="../library/logging.html#logging.warning" title="logging.warning"><code class="xref py py-func docutils literal">warning()</code></a>, <a class="reference internal" href="../library/logging.html#logging.error" title="logging.error"><code class="xref py py-func docutils literal">error()</code></a> and

116

<a class="reference internal" href="../library/logging.html#logging.critical" title="logging.critical"><code class="xref py py-func docutils literal">critical()</code></a>. To determine when to use logging, see the table below, which

117

states, for each of a set of common tasks, the best tool to use for it.

118

119

120

121

122

</colgroup>

123

124

<tr class="row-odd"><th class="head">Task you want to perform</th>

125

126

</tr>

127

</thead>

128

129

<tr class="row-even"><td>Display console output for ordinary

130

usage of a command line script or

131

program</td>

132

<td><a class="reference internal" href="../library/functions.html#print" title="print"><code class="xref py py-func docutils literal">print()</code></a></td>

133

</tr>

134

<tr class="row-odd"><td>Report events that occur during

135

normal operation of a program (e.g.

136

for status monitoring or fault

137

investigation)</td>

138

<td><a class="reference internal" href="../library/logging.html#logging.info" title="logging.info"><code class="xref py py-func docutils literal">logging.info()</code></a> (or

139

<a class="reference internal" href="../library/logging.html#logging.debug" title="logging.debug"><code class="xref py py-func docutils literal">logging.debug()</code></a> for very

140

detailed output for diagnostic

141

purposes)</td>

142

</tr>

143

<tr class="row-even"><td>Issue a warning regarding a

144

particular runtime event</td>

145

<td><a class="reference internal" href="../library/warnings.html#warnings.warn" title="warnings.warn"><code class="xref py py-func docutils literal">warnings.warn()</code></a> in library

146

code if the issue is avoidable and

147

the client application should be

148

modified to eliminate the warning

149

<a class="reference internal" href="../library/logging.html#logging.warning" title="logging.warning"><code class="xref py py-func docutils literal">logging.warning()</code></a> if there is

150

nothing the client application can do

151

about the situation, but the event

152

should still be noted

153

</td>

154

</tr>

155

<tr class="row-odd"><td>Report an error regarding a

156

particular runtime event</td>

157

<td>Raise an exception</td>

158

</tr>

159

<tr class="row-even"><td>Report suppression of an error

160

without raising an exception (e.g.

161

error handler in a long-running

162

server process)</td>

163

<td><a class="reference internal" href="../library/logging.html#logging.error" title="logging.error"><code class="xref py py-func docutils literal">logging.error()</code></a>,

164

<a class="reference internal" href="../library/logging.html#logging.exception" title="logging.exception"><code class="xref py py-func docutils literal">logging.exception()</code></a> or

165

166

appropriate for the specific error

167

and application domain</td>

168

</tr>

169

</tbody>

170

</table>

171

The logging functions are named after the level or severity of the events

172

they are used to track. The standard levels and their applicability are

173

described below (in increasing order of severity):

174

175

176

177

178

</colgroup>

179

180

<tr class="row-odd"><th class="head">Level</th>

181

182

</tr>

183

</thead>

184

185

<tr class="row-even"><td><code class="docutils literal">DEBUG</code></td>

186

<td>Detailed information, typically of interest

187

only when diagnosing problems.</td>

188

</tr>

189

190

<td>Confirmation that things are working as

191

expected.</td>

192

</tr>

193

<tr class="row-even"><td><code class="docutils literal">WARNING</code></td>

194

<td>An indication that something unexpected

195

happened, or indicative of some problem in

196

the near future (e.g. ‘disk space low’).

197

The software is still working as expected.</td>

198

</tr>

199

<tr class="row-odd"><td><code class="docutils literal">ERROR</code></td>

200

<td>Due to a more serious problem, the software

201

has not been able to perform some function.</td>

202

</tr>

203

<tr class="row-even"><td><code class="docutils literal">CRITICAL</code></td>

204

<td>A serious error, indicating that the program

205

itself may be unable to continue running.</td>

206

</tr>

207

</tbody>

208

</table>

209

The default level is <code class="docutils literal">WARNING</code>, which means that only events of this level

210

and above will be tracked, unless the logging package is configured to do

211

otherwise.

212

Events that are tracked can be handled in different ways. The simplest way of

213

handling tracked events is to print them to the console. Another common way

214

is to write them to a disk file.

215

</div>

216

217

<h3>A simple example<a class="headerlink" href="#a-simple-example" title="Permalink to this headline">¶</a></h3>

218

A very simple example is:

219

<div class="highlight-python3"><div class="highlight"><pre>import logging

220

logging.warning('Watch out!') # will print a message to the console

221

logging.info('I told you so') # will not print anything

222

</pre></div>

223

</div>

224

If you type these lines into a script and run it, you’ll see:

225

<div class="highlight-none"><div class="highlight"><pre>WARNING:root:Watch out!

226

</pre></div>

227

</div>

228

printed out on the console. The <code class="docutils literal">INFO</code> message doesn’t appear because the

229

default level is <code class="docutils literal">WARNING</code>. The printed message includes the indication of

230

the level and the description of the event provided in the logging call, i.e.

231

‘Watch out!’. Don’t worry about the ‘root’ part for now: it will be explained

232

later. The actual output can be formatted quite flexibly if you need that;

233

formatting options will also be explained later.

234

</div>

235

236

<h3>Logging to a file<a class="headerlink" href="#logging-to-a-file" title="Permalink to this headline">¶</a></h3>

237

A very common situation is that of recording logging events in a file, so let’s

238

look at that next. Be sure to try the following in a newly-started Python

239

interpreter, and don’t just continue from the session described above:

240

<div class="highlight-python3"><div class="highlight"><pre>import logging

241

logging.basicConfig(filename='example.log',level=logging.DEBUG)

242

logging.debug('This message should go to the log file')

243

logging.info('So should this')

244

logging.warning('And this, too')

245

</pre></div>

246

</div>

247

And now if we open the file and look at what we have, we should find the log

248

messages:

249

<div class="highlight-python3"><div class="highlight"><pre>DEBUG:root:This message should go to the log file

250

INFO:root:So should this

251

WARNING:root:And this, too

252

</pre></div>

253

</div>

254

This example also shows how you can set the logging level which acts as the

255

threshold for tracking. In this case, because we set the threshold to

256

<code class="docutils literal">DEBUG</code>, all of the messages were printed.

257

If you want to set the logging level from a command-line option such as:

258

<div class="highlight-python3"><div class="highlight"><pre>--log=INFO

259

</pre></div>

260

</div>

261

and you have the value of the parameter passed for <code class="docutils literal">--log</code> in some variable

262

loglevel, you can use:

263

<div class="highlight-python3"><div class="highlight"><pre>getattr(logging, loglevel.upper())

264

</pre></div>

265

</div>

266

to get the value which you’ll pass to <a class="reference internal" href="../library/logging.html#logging.basicConfig" title="logging.basicConfig"><code class="xref py py-func docutils literal">basicConfig()</code></a> via the level

267

argument. You may want to error check any user input value, perhaps as in the

268

following example:

269

<div class="highlight-python3"><div class="highlight"><pre># assuming loglevel is bound to the string value obtained from the

270

# command line argument. Convert to upper case to allow the user to

271

# specify --log=DEBUG or --log=debug

272

numeric_level = getattr(logging, loglevel.upper(), None)

273

if not isinstance(numeric_level, int):

274

raise ValueError('Invalid log level: %s' % loglevel)

275

logging.basicConfig(level=numeric_level, ...)

276

</pre></div>

277

</div>

278

The call to <a class="reference internal" href="../library/logging.html#logging.basicConfig" title="logging.basicConfig"><code class="xref py py-func docutils literal">basicConfig()</code></a> should come before any calls to <a class="reference internal" href="../library/logging.html#logging.debug" title="logging.debug"><code class="xref py py-func docutils literal">debug()</code></a>,

279

<a class="reference internal" href="../library/logging.html#logging.info" title="logging.info"><code class="xref py py-func docutils literal">info()</code></a> etc. As it’s intended as a one-off simple configuration facility,

280

only the first call will actually do anything: subsequent calls are effectively

281

no-ops.

282

If you run the above script several times, the messages from successive runs

283

are appended to the file example.log. If you want each run to start afresh,

284

not remembering the messages from earlier runs, you can specify the filemode

285

argument, by changing the call in the above example to:

286

<div class="highlight-python3"><div class="highlight"><pre>logging.basicConfig(filename='example.log', filemode='w', level=logging.DEBUG)

287

</pre></div>

288

</div>

289

The output will be the same as before, but the log file is no longer appended

290

to, so the messages from earlier runs are lost.

291

</div>

292

293

<h3>Logging from multiple modules<a class="headerlink" href="#logging-from-multiple-modules" title="Permalink to this headline">¶</a></h3>

294

If your program consists of multiple modules, here’s an example of how you

295

could organize logging in it:

296

<div class="highlight-python3"><div class="highlight"><pre># myapp.py

297

import logging

298

import mylib

299

300

def main():

301

logging.basicConfig(filename='myapp.log', level=logging.INFO)

302

logging.info('Started')

303

mylib.do_something()

304

logging.info('Finished')

305

306

if __name__ == '__main__':

307

main()

308

</pre></div>

309

</div>

310

<div class="highlight-python3"><div class="highlight"><pre># mylib.py

311

import logging

312

313

def do_something():

314

logging.info('Doing something')

315

</pre></div>

316

</div>

317

If you run myapp.py, you should see this in myapp.log:

318

<div class="highlight-python3"><div class="highlight"><pre>INFO:root:Started

319

INFO:root:Doing something

320

INFO:root:Finished

321

</pre></div>

322

</div>

323

which is hopefully what you were expecting to see. You can generalize this to

324

multiple modules, using the pattern in mylib.py. Note that for this simple

325

usage pattern, you won’t know, by looking in the log file, where in your

326

application your messages came from, apart from looking at the event

327

description. If you want to track the location of your messages, you’ll need

328

to refer to the documentation beyond the tutorial level – see

329

<a class="reference internal" href="#logging-advanced-tutorial">Advanced Logging Tutorial</a>.

330

</div>

331

332

<h3>Logging variable data<a class="headerlink" href="#logging-variable-data" title="Permalink to this headline">¶</a></h3>

333

To log variable data, use a format string for the event description message and

334

append the variable data as arguments. For example:

335

<div class="highlight-python3"><div class="highlight"><pre>import logging

336

logging.warning('%s before you %s', 'Look', 'leap!')

337

</pre></div>

338

</div>

339

will display:

340

<div class="highlight-none"><div class="highlight"><pre>WARNING:root:Look before you leap!

341

</pre></div>

342

</div>

343

As you can see, merging of variable data into the event description message

344

uses the old, %-style of string formatting. This is for backwards

345

compatibility: the logging package pre-dates newer formatting options such as

346

<a class="reference internal" href="../library/stdtypes.html#str.format" title="str.format"><code class="xref py py-meth docutils literal">str.format()</code></a> and <a class="reference internal" href="../library/string.html#string.Template" title="string.Template"><code class="xref py py-class docutils literal">string.Template</code></a>. These newer formatting

347

options are supported, but exploring them is outside the scope of this

348

tutorial: see <a class="reference internal" href="logging-cookbook.html#formatting-styles">Using particular formatting styles throughout your application</a> for more information.

349

</div>

350

351

<h3>Changing the format of displayed messages<a class="headerlink" href="#changing-the-format-of-displayed-messages" title="Permalink to this headline">¶</a></h3>

352

To change the format which is used to display messages, you need to

353

specify the format you want to use:

354

<div class="highlight-python3"><div class="highlight"><pre>import logging

355

logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG)

356

logging.debug('This message should appear on the console')

357

logging.info('So should this')

358

logging.warning('And this, too')

359

</pre></div>

360

</div>

361

which would print:

362

<div class="highlight-python3"><div class="highlight"><pre>DEBUG:This message should appear on the console

363

INFO:So should this

364

WARNING:And this, too

365

</pre></div>

366

</div>

367

Notice that the ‘root’ which appeared in earlier examples has disappeared. For

368

a full set of things that can appear in format strings, you can refer to the

369

documentation for <a class="reference internal" href="../library/logging.html#logrecord-attributes">LogRecord attributes</a>, but for simple usage, you just

370

need the levelname (severity), message (event description, including

371

variable data) and perhaps to display when the event occurred. This is

372

described in the next section.

373

</div>

374

375

<h3>Displaying the date/time in messages<a class="headerlink" href="#displaying-the-date-time-in-messages" title="Permalink to this headline">¶</a></h3>

376

To display the date and time of an event, you would place ‘%(asctime)s’ in

377

your format string:

378

<div class="highlight-python3"><div class="highlight"><pre>import logging

379

380

logging.warning('is when this event was logged.')

381

</pre></div>

382

</div>

383

which should print something like this:

384

<div class="highlight-python3"><div class="highlight"><pre>2010-12-12 11:41:42,612 is when this event was logged.

385

</pre></div>

386

</div>

387

The default format for date/time display (shown above) is ISO8601. If you need

388

more control over the formatting of the date/time, provide a datefmt

389

argument to <code class="docutils literal">basicConfig</code>, as in this example:

390

<div class="highlight-python3"><div class="highlight"><pre>import logging

391

logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p')

392

logging.warning('is when this event was logged.')

393

</pre></div>

394

</div>

395

which would display something like this:

396

<div class="highlight-python3"><div class="highlight"><pre>12/12/2010 11:46:36 AM is when this event was logged.

397

</pre></div>

398

</div>

399

The format of the datefmt argument is the same as supported by

400

<a class="reference internal" href="../library/time.html#time.strftime" title="time.strftime"><code class="xref py py-func docutils literal">time.strftime()</code></a>.

401

</div>

402

403

<h3>Next Steps<a class="headerlink" href="#next-steps" title="Permalink to this headline">¶</a></h3>

404

That concludes the basic tutorial. It should be enough to get you up and

405

running with logging. There’s a lot more that the logging package offers, but

406

to get the best out of it, you’ll need to invest a little more of your time in

407

reading the following sections. If you’re ready for that, grab some of your

408

favourite beverage and carry on.

409

If your logging needs are simple, then use the above examples to incorporate

410

logging into your own scripts, and if you run into problems or don’t

411

understand something, please post a question on the comp.lang.python Usenet

412

group (available at <a class="reference external" href="https://groups.google.com/group/comp.lang.python">https://groups.google.com/group/comp.lang.python</a>) and you

413

should receive help before too long.

414

Still here? You can carry on reading the next few sections, which provide a

415

slightly more advanced/in-depth tutorial than the basic one above. After that,

416

you can take a look at the <a class="reference internal" href="logging-cookbook.html#logging-cookbook">Logging Cookbook</a>.

417

</div>

418

</div>

419

420

<h2>Advanced Logging Tutorial<a class="headerlink" href="#advanced-logging-tutorial" title="Permalink to this headline">¶</a></h2>

421

The logging library takes a modular approach and offers several categories

422

of components: loggers, handlers, filters, and formatters.

423

424

<li>Loggers expose the interface that application code directly uses.</li>

425

<li>Handlers send the log records (created by loggers) to the appropriate

426

destination.</li>

427

<li>Filters provide a finer grained facility for determining which log records

428

to output.</li>

429

<li>Formatters specify the layout of log records in the final output.</li>

430

</ul>

431

Log event information is passed between loggers, handlers, filters and

432

formatters in a <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal">LogRecord</code></a> instance.

433

Logging is performed by calling methods on instances of the <a class="reference internal" href="../library/logging.html#logging.Logger" title="logging.Logger"><code class="xref py py-class docutils literal">Logger</code></a>

434

class (hereafter called loggers). Each instance has a name, and they are

435

conceptually arranged in a namespace hierarchy using dots (periods) as

436

separators. For example, a logger named ‘scan’ is the parent of loggers

437

‘scan.text’, ‘scan.html’ and ‘scan.pdf’. Logger names can be anything you want,

438

and indicate the area of an application in which a logged message originates.

439

A good convention to use when naming loggers is to use a module-level logger,

440

in each module which uses logging, named as follows:

441

<div class="highlight-python3"><div class="highlight"><pre>logger = logging.getLogger(__name__)

442

</pre></div>

443

</div>

444

This means that logger names track the package/module hierarchy, and it’s

445

intuitively obvious where events are logged just from the logger name.

446

The root of the hierarchy of loggers is called the root logger. That’s the

447

logger used by the functions <a class="reference internal" href="../library/logging.html#logging.debug" title="logging.debug"><code class="xref py py-func docutils literal">debug()</code></a>, <a class="reference internal" href="../library/logging.html#logging.info" title="logging.info"><code class="xref py py-func docutils literal">info()</code></a>, <a class="reference internal" href="../library/logging.html#logging.warning" title="logging.warning"><code class="xref py py-func docutils literal">warning()</code></a>,

448

<a class="reference internal" href="../library/logging.html#logging.error" title="logging.error"><code class="xref py py-func docutils literal">error()</code></a> and <a class="reference internal" href="../library/logging.html#logging.critical" title="logging.critical"><code class="xref py py-func docutils literal">critical()</code></a>, which just call the same-named method of

449

the root logger. The functions and the methods have the same signatures. The

450

root logger’s name is printed as ‘root’ in the logged output.

451

It is, of course, possible to log messages to different destinations. Support

452

is included in the package for writing log messages to files, HTTP GET/POST

453

locations, email via SMTP, generic sockets, queues, or OS-specific logging

454

mechanisms such as syslog or the Windows NT event log. Destinations are served

455

by handler classes. You can create your own log destination class if

456

you have special requirements not met by any of the built-in handler classes.

457

By default, no destination is set for any logging messages. You can specify

458

a destination (such as console or file) by using <a class="reference internal" href="../library/logging.html#logging.basicConfig" title="logging.basicConfig"><code class="xref py py-func docutils literal">basicConfig()</code></a> as in the

459

tutorial examples. If you call the functions <a class="reference internal" href="../library/logging.html#logging.debug" title="logging.debug"><code class="xref py py-func docutils literal">debug()</code></a>, <a class="reference internal" href="../library/logging.html#logging.info" title="logging.info"><code class="xref py py-func docutils literal">info()</code></a>,

460

<a class="reference internal" href="../library/logging.html#logging.warning" title="logging.warning"><code class="xref py py-func docutils literal">warning()</code></a>, <a class="reference internal" href="../library/logging.html#logging.error" title="logging.error"><code class="xref py py-func docutils literal">error()</code></a> and <a class="reference internal" href="../library/logging.html#logging.critical" title="logging.critical"><code class="xref py py-func docutils literal">critical()</code></a>, they will check to see

461

if no destination is set; and if one is not set, they will set a destination

462

of the console (<code class="docutils literal">sys.stderr</code>) and a default format for the displayed

463

message before delegating to the root logger to do the actual message output.

464

The default format set by <a class="reference internal" href="../library/logging.html#logging.basicConfig" title="logging.basicConfig"><code class="xref py py-func docutils literal">basicConfig()</code></a> for messages is:

465

<div class="highlight-python3"><div class="highlight"><pre>severity:logger name:message

466

</pre></div>

467

</div>

468

You can change this by passing a format string to <a class="reference internal" href="../library/logging.html#logging.basicConfig" title="logging.basicConfig"><code class="xref py py-func docutils literal">basicConfig()</code></a> with the

469

format keyword argument. For all options regarding how a format string is

470

constructed, see <a class="reference internal" href="../library/logging.html#formatter-objects">Formatter Objects</a>.

471

472

<h3>Logging Flow<a class="headerlink" href="#logging-flow" title="Permalink to this headline">¶</a></h3>

473

The flow of log event information in loggers and handlers is illustrated in the

474

following diagram.

475

476

</div>

477

478

<h3>Loggers<a class="headerlink" href="#loggers" title="Permalink to this headline">¶</a></h3>

479

<a class="reference internal" href="../library/logging.html#logging.Logger" title="logging.Logger"><code class="xref py py-class docutils literal">Logger</code></a> objects have a threefold job. First, they expose several

480

methods to application code so that applications can log messages at runtime.

481

Second, logger objects determine which log messages to act upon based upon

482

severity (the default filtering facility) or filter objects. Third, logger

483

objects pass along relevant log messages to all interested log handlers.

484

The most widely used methods on logger objects fall into two categories:

485

configuration and message sending.

486

These are the most common configuration methods:

487

488

<li><a class="reference internal" href="../library/logging.html#logging.Logger.setLevel" title="logging.Logger.setLevel"><code class="xref py py-meth docutils literal">Logger.setLevel()</code></a> specifies the lowest-severity log message a logger

489

will handle, where debug is the lowest built-in severity level and critical

490

is the highest built-in severity. For example, if the severity level is

491

INFO, the logger will handle only INFO, WARNING, ERROR, and CRITICAL messages

492

and will ignore DEBUG messages.</li>

493

<li><a class="reference internal" href="../library/logging.html#logging.Logger.addHandler" title="logging.Logger.addHandler"><code class="xref py py-meth docutils literal">Logger.addHandler()</code></a> and <a class="reference internal" href="../library/logging.html#logging.Logger.removeHandler" title="logging.Logger.removeHandler"><code class="xref py py-meth docutils literal">Logger.removeHandler()</code></a> add and remove

494

handler objects from the logger object. Handlers are covered in more detail

495

in <a class="reference internal" href="#handler-basic">Handlers</a>.</li>

496

<li><a class="reference internal" href="../library/logging.html#logging.Logger.addFilter" title="logging.Logger.addFilter"><code class="xref py py-meth docutils literal">Logger.addFilter()</code></a> and <a class="reference internal" href="../library/logging.html#logging.Logger.removeFilter" title="logging.Logger.removeFilter"><code class="xref py py-meth docutils literal">Logger.removeFilter()</code></a> add and remove filter

497

objects from the logger object. Filters are covered in more detail in

498

<a class="reference internal" href="../library/logging.html#filter">Filter Objects</a>.</li>

499

</ul>

500

You don’t need to always call these methods on every logger you create. See the

501

last two paragraphs in this section.

502

With the logger object configured, the following methods create log messages:

503

504

<li><a class="reference internal" href="../library/logging.html#logging.Logger.debug" title="logging.Logger.debug"><code class="xref py py-meth docutils literal">Logger.debug()</code></a>, <a class="reference internal" href="../library/logging.html#logging.Logger.info" title="logging.Logger.info"><code class="xref py py-meth docutils literal">Logger.info()</code></a>, <a class="reference internal" href="../library/logging.html#logging.Logger.warning" title="logging.Logger.warning"><code class="xref py py-meth docutils literal">Logger.warning()</code></a>,

505

<a class="reference internal" href="../library/logging.html#logging.Logger.error" title="logging.Logger.error"><code class="xref py py-meth docutils literal">Logger.error()</code></a>, and <a class="reference internal" href="../library/logging.html#logging.Logger.critical" title="logging.Logger.critical"><code class="xref py py-meth docutils literal">Logger.critical()</code></a> all create log records with

506

a message and a level that corresponds to their respective method names. The

507

message is actually a format string, which may contain the standard string

508

substitution syntax of <code class="docutils literal">%s</code>, <code class="docutils literal">%d</code>, <code class="docutils literal">%f</code>, and so on. The

509

rest of their arguments is a list of objects that correspond with the

510

substitution fields in the message. With regard to <code class="docutils literal">**kwargs</code>, the

511

logging methods care only about a keyword of <code class="docutils literal">exc_info</code> and use it to

512

determine whether to log exception information.</li>

513

<li><a class="reference internal" href="../library/logging.html#logging.Logger.exception" title="logging.Logger.exception"><code class="xref py py-meth docutils literal">Logger.exception()</code></a> creates a log message similar to

514

<a class="reference internal" href="../library/logging.html#logging.Logger.error" title="logging.Logger.error"><code class="xref py py-meth docutils literal">Logger.error()</code></a>. The difference is that <a class="reference internal" href="../library/logging.html#logging.Logger.exception" title="logging.Logger.exception"><code class="xref py py-meth docutils literal">Logger.exception()</code></a> dumps a

515

stack trace along with it. Call this method only from an exception handler.</li>

516

<li><a class="reference internal" href="../library/logging.html#logging.Logger.log" title="logging.Logger.log"><code class="xref py py-meth docutils literal">Logger.log()</code></a> takes a log level as an explicit argument. This is a

517

little more verbose for logging messages than using the log level convenience

518

methods listed above, but this is how to log at custom log levels.</li>

519

</ul>

520

<a class="reference internal" href="../library/logging.html#logging.getLogger" title="logging.getLogger"><code class="xref py py-func docutils literal">getLogger()</code></a> returns a reference to a logger instance with the specified

521

name if it is provided, or <code class="docutils literal">root</code> if not. The names are period-separated

522

hierarchical structures. Multiple calls to <a class="reference internal" href="../library/logging.html#logging.getLogger" title="logging.getLogger"><code class="xref py py-func docutils literal">getLogger()</code></a> with the same name

523

will return a reference to the same logger object. Loggers that are further

524

down in the hierarchical list are children of loggers higher up in the list.

525

For example, given a logger with a name of <code class="docutils literal">foo</code>, loggers with names of

526

<code class="docutils literal">foo.bar</code>, <code class="docutils literal">foo.bar.baz</code>, and <code class="docutils literal">foo.bam</code> are all descendants of <code class="docutils literal">foo</code>.

527

Loggers have a concept of effective level. If a level is not explicitly set

528

on a logger, the level of its parent is used instead as its effective level.

529

If the parent has no explicit level set, its parent is examined, and so on -

530

all ancestors are searched until an explicitly set level is found. The root

531

logger always has an explicit level set (<code class="docutils literal">WARNING</code> by default). When deciding

532

whether to process an event, the effective level of the logger is used to

533

determine whether the event is passed to the logger’s handlers.

534

Child loggers propagate messages up to the handlers associated with their

535

ancestor loggers. Because of this, it is unnecessary to define and configure

536

handlers for all the loggers an application uses. It is sufficient to

537

configure handlers for a top-level logger and create child loggers as needed.

538

(You can, however, turn off propagation by setting the propagate

539

attribute of a logger to <code class="docutils literal">False</code>.)

540

</div>

541

542

<h3>Handlers<a class="headerlink" href="#handlers" title="Permalink to this headline">¶</a></h3>

543

<code class="xref py py-class docutils literal">Handler</code> objects are responsible for dispatching the

544

appropriate log messages (based on the log messages’ severity) to the handler’s

545

specified destination. <a class="reference internal" href="../library/logging.html#logging.Logger" title="logging.Logger"><code class="xref py py-class docutils literal">Logger</code></a> objects can add zero or more handler

546

objects to themselves with an <a class="reference internal" href="../library/logging.html#logging.Logger.addHandler" title="logging.Logger.addHandler"><code class="xref py py-meth docutils literal">addHandler()</code></a> method. As an example

547

scenario, an application may want to send all log messages to a log file, all

548

log messages of error or higher to stdout, and all messages of critical to an

549

email address. This scenario requires three individual handlers where each

550

handler is responsible for sending messages of a specific severity to a specific

551

location.

552

The standard library includes quite a few handler types (see

553

<a class="reference internal" href="#useful-handlers">Useful Handlers</a>); the tutorials use mainly <a class="reference internal" href="../library/logging.handlers.html#logging.StreamHandler" title="logging.StreamHandler"><code class="xref py py-class docutils literal">StreamHandler</code></a> and

554

<a class="reference internal" href="../library/logging.handlers.html#logging.FileHandler" title="logging.FileHandler"><code class="xref py py-class docutils literal">FileHandler</code></a> in its examples.

555

There are very few methods in a handler for application developers to concern

556

themselves with. The only handler methods that seem relevant for application

557

developers who are using the built-in handler objects (that is, not creating

558

custom handlers) are the following configuration methods:

559

560

<li>The <a class="reference internal" href="../library/logging.html#logging.Handler.setLevel" title="logging.Handler.setLevel"><code class="xref py py-meth docutils literal">setLevel()</code></a> method, just as in logger objects, specifies the

561

lowest severity that will be dispatched to the appropriate destination. Why

562

are there two <code class="xref py py-func docutils literal">setLevel()</code> methods? The level set in the logger

563

determines which severity of messages it will pass to its handlers. The level

564

set in each handler determines which messages that handler will send on.</li>

565

<li><a class="reference internal" href="../library/logging.html#logging.Handler.setFormatter" title="logging.Handler.setFormatter"><code class="xref py py-meth docutils literal">setFormatter()</code></a> selects a Formatter object for this handler to

566

use.</li>

567

<li><a class="reference internal" href="../library/logging.html#logging.Handler.addFilter" title="logging.Handler.addFilter"><code class="xref py py-meth docutils literal">addFilter()</code></a> and <a class="reference internal" href="../library/logging.html#logging.Handler.removeFilter" title="logging.Handler.removeFilter"><code class="xref py py-meth docutils literal">removeFilter()</code></a> respectively

568

configure and deconfigure filter objects on handlers.</li>

569

</ul>

570

Application code should not directly instantiate and use instances of

571

<code class="xref py py-class docutils literal">Handler</code>. Instead, the <code class="xref py py-class docutils literal">Handler</code> class is a base class that

572

defines the interface that all handlers should have and establishes some

573

default behavior that child classes can use (or override).

574

</div>

575

576

<h3>Formatters<a class="headerlink" href="#formatters" title="Permalink to this headline">¶</a></h3>

577

Formatter objects configure the final order, structure, and contents of the log

578

message. Unlike the base <code class="xref py py-class docutils literal">logging.Handler</code> class, application code may

579

instantiate formatter classes, although you could likely subclass the formatter

580

if your application needs special behavior. The constructor takes three

581

optional arguments – a message format string, a date format string and a style

582

indicator.

583

584

585

<code class="descclassname">logging.Formatter.</code><code class="descname">__init__</code>(fmt=None, datefmt=None, style='%')<a class="headerlink" href="#logging.logging.Formatter.__init__" title="Permalink to this definition">¶</a></dt>

586

587

588

If there is no message format string, the default is to use the

589

raw message. If there is no date format string, the default date format is:

590

<div class="highlight-python3"><div class="highlight"><pre>%Y-%m-%d %H:%M:%S

591

</pre></div>

592

</div>

593

with the milliseconds tacked on at the end. The <code class="docutils literal">style</code> is one of <cite>%</cite>, ‘{‘

594

or ‘$’. If one of these is not specified, then ‘%’ will be used.

595

If the <code class="docutils literal">style</code> is ‘%’, the message format string uses

596

<code class="docutils literal">%(<dictionary key>)s</code> styled string substitution; the possible keys are

597

documented in <a class="reference internal" href="../library/logging.html#logrecord-attributes">LogRecord attributes</a>. If the style is ‘{‘, the message

598

format string is assumed to be compatible with <a class="reference internal" href="../library/stdtypes.html#str.format" title="str.format"><code class="xref py py-meth docutils literal">str.format()</code></a> (using

599

keyword arguments), while if the style is ‘$’ then the message format string

600

should conform to what is expected by <a class="reference internal" href="../library/string.html#string.Template.substitute" title="string.Template.substitute"><code class="xref py py-meth docutils literal">string.Template.substitute()</code></a>.

601

602

Changed in version 3.2: Added the <code class="docutils literal">style</code> parameter.

603

</div>

604

The following message format string will log the time in a human-readable

605

format, the severity of the message, and the contents of the message, in that

606

order:

607

<div class="highlight-python3"><div class="highlight"><pre>'%(asctime)s - %(levelname)s - %(message)s'

608

</pre></div>

609

</div>

610

Formatters use a user-configurable function to convert the creation time of a

611

record to a tuple. By default, <a class="reference internal" href="../library/time.html#time.localtime" title="time.localtime"><code class="xref py py-func docutils literal">time.localtime()</code></a> is used; to change this

612

for a particular formatter instance, set the <code class="docutils literal">converter</code> attribute of the

613

instance to a function with the same signature as <a class="reference internal" href="../library/time.html#time.localtime" title="time.localtime"><code class="xref py py-func docutils literal">time.localtime()</code></a> or

614

<a class="reference internal" href="../library/time.html#time.gmtime" title="time.gmtime"><code class="xref py py-func docutils literal">time.gmtime()</code></a>. To change it for all formatters, for example if you want

615

all logging times to be shown in GMT, set the <code class="docutils literal">converter</code> attribute in the

616

Formatter class (to <code class="docutils literal">time.gmtime</code> for GMT display).

617

</div>

618

619

<h3>Configuring Logging<a class="headerlink" href="#configuring-logging" title="Permalink to this headline">¶</a></h3>

620

Programmers can configure logging in three ways:

621

622

<li>Creating loggers, handlers, and formatters explicitly using Python

623

code that calls the configuration methods listed above.</li>

624

<li>Creating a logging config file and reading it using the <a class="reference internal" href="../library/logging.config.html#logging.config.fileConfig" title="logging.config.fileConfig"><code class="xref py py-func docutils literal">fileConfig()</code></a>

625

function.</li>

626

<li>Creating a dictionary of configuration information and passing it

627

to the <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal">dictConfig()</code></a> function.</li>

628

</ol>

629

For the reference documentation on the last two options, see

630

<a class="reference internal" href="../library/logging.config.html#logging-config-api">Configuration functions</a>. The following example configures a very simple

631

logger, a console handler, and a simple formatter using Python code:

632

<div class="highlight-python3"><div class="highlight"><pre>import logging

633

634

# create logger

635

logger = logging.getLogger('simple_example')

636

logger.setLevel(logging.DEBUG)

637

638

# create console handler and set level to debug

639

ch = logging.StreamHandler()

640

ch.setLevel(logging.DEBUG)

641

642

# create formatter

643

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

644

645

# add formatter to ch

646

ch.setFormatter(formatter)

647

648

# add ch to logger

649

logger.addHandler(ch)

650

651

# 'application' code

652

logger.debug('debug message')

653

logger.info('info message')

654

logger.warn('warn message')

655

logger.error('error message')

656

logger.critical('critical message')

657

</pre></div>

658

</div>

659

Running this module from the command line produces the following output:

660

<div class="highlight-shell-session"><div class="highlight"><pre>$ python simple_logging_module.py

661

2005-03-19 15:10:26,618 - simple_example - DEBUG - debug message

662

2005-03-19 15:10:26,620 - simple_example - INFO - info message

663

2005-03-19 15:10:26,695 - simple_example - WARNING - warn message

664

2005-03-19 15:10:26,697 - simple_example - ERROR - error message

665

2005-03-19 15:10:26,773 - simple_example - CRITICAL - critical message

666

</pre></div>

667

</div>

668

The following Python module creates a logger, handler, and formatter nearly

669

identical to those in the example listed above, with the only difference being

670

the names of the objects:

671

<div class="highlight-python3"><div class="highlight"><pre>import logging

672

import logging.config

673

674

logging.config.fileConfig('logging.conf')

675

676

# create logger

677

678

679

# 'application' code

680

logger.debug('debug message')

681

logger.info('info message')

682

logger.warn('warn message')

683

logger.error('error message')

684

logger.critical('critical message')

685

</pre></div>

686

</div>

687

Here is the logging.conf file:

688

<div class="highlight-python3"><div class="highlight"><pre>[loggers]

689

keys=root,simpleExample

690

691

[handlers]

692

keys=consoleHandler

693

694

[formatters]

695

keys=simpleFormatter

696

697

[logger_root]

698

level=DEBUG

699

handlers=consoleHandler

700

701

[logger_simpleExample]

702

level=DEBUG

703

handlers=consoleHandler

704

qualname=simpleExample

705

propagate=0

706

707

[handler_consoleHandler]

708

class=StreamHandler

709

level=DEBUG

710

formatter=simpleFormatter

711

args=(sys.stdout,)

712

713

[formatter_simpleFormatter]

714

format=%(asctime)s - %(name)s - %(levelname)s - %(message)s

715

datefmt=

716

</pre></div>

717

</div>

718

The output is nearly identical to that of the non-config-file-based example:

719

<div class="highlight-shell-session"><div class="highlight"><pre>$ python simple_logging_config.py

720

2005-03-19 15:38:55,977 - simpleExample - DEBUG - debug message

721

2005-03-19 15:38:55,979 - simpleExample - INFO - info message

722

2005-03-19 15:38:56,054 - simpleExample - WARNING - warn message

723

2005-03-19 15:38:56,055 - simpleExample - ERROR - error message

724

2005-03-19 15:38:56,130 - simpleExample - CRITICAL - critical message

725

</pre></div>

726

</div>

727

You can see that the config file approach has a few advantages over the Python

728

code approach, mainly separation of configuration and code and the ability of

729

noncoders to easily modify the logging properties.

730

731

Warning

732

The <a class="reference internal" href="../library/logging.config.html#logging.config.fileConfig" title="logging.config.fileConfig"><code class="xref py py-func docutils literal">fileConfig()</code></a> function takes a default parameter,

733

<code class="docutils literal">disable_existing_loggers</code>, which defaults to <code class="docutils literal">True</code> for reasons of

734

backward compatibility. This may or may not be what you want, since it

735

will cause any loggers existing before the <a class="reference internal" href="../library/logging.config.html#logging.config.fileConfig" title="logging.config.fileConfig"><code class="xref py py-func docutils literal">fileConfig()</code></a> call to

736

be disabled unless they (or an ancestor) are explicitly named in the

737

configuration. Please refer to the reference documentation for more

738

information, and specify <code class="docutils literal">False</code> for this parameter if you wish.

739

The dictionary passed to <a class="reference internal" href="../library/logging.config.html#logging.config.dictConfig" title="logging.config.dictConfig"><code class="xref py py-func docutils literal">dictConfig()</code></a> can also specify a Boolean

740

value with key <code class="docutils literal">disable_existing_loggers</code>, which if not specified

741

explicitly in the dictionary also defaults to being interpreted as

742

<code class="docutils literal">True</code>. This leads to the logger-disabling behaviour described above,

743

which may not be what you want - in which case, provide the key

744

explicitly with a value of <code class="docutils literal">False</code>.

745

</div>

746

Note that the class names referenced in config files need to be either relative

747

to the logging module, or absolute values which can be resolved using normal

748

import mechanisms. Thus, you could use either

749

<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.WatchedFileHandler" title="logging.handlers.WatchedFileHandler"><code class="xref py py-class docutils literal">WatchedFileHandler</code></a> (relative to the logging module) or

750

<code class="docutils literal">mypackage.mymodule.MyHandler</code> (for a class defined in package <code class="docutils literal">mypackage</code>

751

and module <code class="docutils literal">mymodule</code>, where <code class="docutils literal">mypackage</code> is available on the Python import

752

path).

753

In Python 3.2, a new means of configuring logging has been introduced, using

754

dictionaries to hold configuration information. This provides a superset of the

755

functionality of the config-file-based approach outlined above, and is the

756

recommended configuration method for new applications and deployments. Because

757

a Python dictionary is used to hold configuration information, and since you

758

can populate that dictionary using different means, you have more options for

759

configuration. For example, you can use a configuration file in JSON format,

760

or, if you have access to YAML processing functionality, a file in YAML

761

format, to populate the configuration dictionary. Or, of course, you can

762

construct the dictionary in Python code, receive it in pickled form over a

763

socket, or use whatever approach makes sense for your application.

764

Here’s an example of the same configuration as above, in YAML format for

765

the new dictionary-based approach:

766

<div class="highlight-python3"><div class="highlight"><pre>version: 1

767

formatters:

768

simple:

769

format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'

770

handlers:

771

console:

772

class: logging.StreamHandler

773

level: DEBUG

774

formatter: simple

775

stream: ext://sys.stdout

776

loggers:

777

simpleExample:

778

level: DEBUG

779

handlers: [console]

780

propagate: no

781

root:

782

level: DEBUG

783

handlers: [console]

784

</pre></div>

785

</div>

786

For more information about logging using a dictionary, see

787

<a class="reference internal" href="../library/logging.config.html#logging-config-api">Configuration functions</a>.

788

</div>

789

790

<h3>What happens if no configuration is provided<a class="headerlink" href="#what-happens-if-no-configuration-is-provided" title="Permalink to this headline">¶</a></h3>

791

If no logging configuration is provided, it is possible to have a situation

792

where a logging event needs to be output, but no handlers can be found to

793

output the event. The behaviour of the logging package in these

794

circumstances is dependent on the Python version.

795

For versions of Python prior to 3.2, the behaviour is as follows:

796

797

<li>If logging.raiseExceptions is <code class="docutils literal">False</code> (production mode), the event is

798

silently dropped.</li>

799

<li>If logging.raiseExceptions is <code class="docutils literal">True</code> (development mode), a message

800

‘No handlers could be found for logger X.Y.Z’ is printed once.</li>

801

</ul>

802

In Python 3.2 and later, the behaviour is as follows:

803

804

<li>The event is output using a ‘handler of last resort’, stored in

805

<code class="docutils literal">logging.lastResort</code>. This internal handler is not associated with any

806

logger, and acts like a <a class="reference internal" href="../library/logging.handlers.html#logging.StreamHandler" title="logging.StreamHandler"><code class="xref py py-class docutils literal">StreamHandler</code></a> which writes the

807

event description message to the current value of <code class="docutils literal">sys.stderr</code> (therefore

808

respecting any redirections which may be in effect). No formatting is

809

done on the message - just the bare event description message is printed.

810

The handler’s level is set to <code class="docutils literal">WARNING</code>, so all events at this and

811

greater severities will be output.</li>

812

</ul>

813

To obtain the pre-3.2 behaviour, <code class="docutils literal">logging.lastResort</code> can be set to <code class="docutils literal">None</code>.

814

</div>

815

816

<h3>Configuring Logging for a Library<a class="headerlink" href="#configuring-logging-for-a-library" title="Permalink to this headline">¶</a></h3>

817

When developing a library which uses logging, you should take care to

818

document how the library uses logging - for example, the names of loggers

819

used. Some consideration also needs to be given to its logging configuration.

820

If the using application does not use logging, and library code makes logging

821

calls, then (as described in the previous section) events of severity

822

<code class="docutils literal">WARNING</code> and greater will be printed to <code class="docutils literal">sys.stderr</code>. This is regarded as

823

the best default behaviour.

824

If for some reason you don’t want these messages printed in the absence of

825

any logging configuration, you can attach a do-nothing handler to the top-level

826

logger for your library. This avoids the message being printed, since a handler

827

will be always be found for the library’s events: it just doesn’t produce any

828

output. If the library user configures logging for application use, presumably

829

that configuration will add some handlers, and if levels are suitably

830

configured then logging calls made in library code will send output to those

831

handlers, as normal.

832

A do-nothing handler is included in the logging package:

833

<a class="reference internal" href="../library/logging.handlers.html#logging.NullHandler" title="logging.NullHandler"><code class="xref py py-class docutils literal">NullHandler</code></a> (since Python 3.1). An instance of this handler

834

could be added to the top-level logger of the logging namespace used by the

835

library (if you want to prevent your library’s logged events being output to

836

<code class="docutils literal">sys.stderr</code> in the absence of logging configuration). If all logging by a

837

library foo is done using loggers with names matching ‘foo.x’, ‘foo.x.y’,

838

etc. then the code:

839

<div class="highlight-python3"><div class="highlight"><pre>import logging

840

logging.getLogger('foo').addHandler(logging.NullHandler())

841

</pre></div>

842

</div>

843

should have the desired effect. If an organisation produces a number of

844

libraries, then the logger name specified can be ‘orgname.foo’ rather than

845

just ‘foo’.

846

847

Note

848

It is strongly advised that you do not add any handlers other

849

than <a class="reference internal" href="../library/logging.handlers.html#logging.NullHandler" title="logging.NullHandler"><code class="xref py py-class docutils literal">NullHandler</code></a> to your library’s loggers. This is

850

because the configuration of handlers is the prerogative of the application

851

developer who uses your library. The application developer knows their

852

target audience and what handlers are most appropriate for their

853

application: if you add handlers ‘under the hood’, you might well interfere

854

with their ability to carry out unit tests and deliver logs which suit their

855

requirements.

856

</div>

857

</div>

858

</div>

859

860

<h2>Logging Levels<a class="headerlink" href="#logging-levels" title="Permalink to this headline">¶</a></h2>

861

The numeric values of logging levels are given in the following table. These are

862

primarily of interest if you want to define your own levels, and need them to

863

have specific values relative to the predefined levels. If you define a level

864

with the same numeric value, it overwrites the predefined value; the predefined

865

name is lost.

866

867

868

869

870

</colgroup>

871

872

<tr class="row-odd"><th class="head">Level</th>

873

<th class="head">Numeric value</th>

874

</tr>

875

</thead>

876

877

<tr class="row-even"><td><code class="docutils literal">CRITICAL</code></td>

878

879

</tr>

880

<tr class="row-odd"><td><code class="docutils literal">ERROR</code></td>

881

882

</tr>

883

<tr class="row-even"><td><code class="docutils literal">WARNING</code></td>

884

885

</tr>

886

887

888

</tr>

889

<tr class="row-even"><td><code class="docutils literal">DEBUG</code></td>

890

891

</tr>

892

<tr class="row-odd"><td><code class="docutils literal">NOTSET</code></td>

893

894

</tr>

895

</tbody>

896

</table>

897

Levels can also be associated with loggers, being set either by the developer or

898

through loading a saved logging configuration. When a logging method is called

899

on a logger, the logger compares its own level with the level associated with

900

the method call. If the logger’s level is higher than the method call’s, no

901

logging message is actually generated. This is the basic mechanism controlling

902

the verbosity of logging output.

903

Logging messages are encoded as instances of the <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal">LogRecord</code></a>

904

class. When a logger decides to actually log an event, a

905

<a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal">LogRecord</code></a> instance is created from the logging message.

906

Logging messages are subjected to a dispatch mechanism through the use of

907

handlers, which are instances of subclasses of the <code class="xref py py-class docutils literal">Handler</code>

908

class. Handlers are responsible for ensuring that a logged message (in the form

909

of a <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal">LogRecord</code></a>) ends up in a particular location (or set of locations)

910

which is useful for the target audience for that message (such as end users,

911

support desk staff, system administrators, developers). Handlers are passed

912

913

can have zero, one or more handlers associated with it (via the

914

<a class="reference internal" href="../library/logging.html#logging.Logger.addHandler" title="logging.Logger.addHandler"><code class="xref py py-meth docutils literal">addHandler()</code></a> method of <a class="reference internal" href="../library/logging.html#logging.Logger" title="logging.Logger"><code class="xref py py-class docutils literal">Logger</code></a>). In addition to any

915

handlers directly associated with a logger, all handlers associated with all

916

ancestors of the logger are called to dispatch the message (unless the

917

propagate flag for a logger is set to a false value, at which point the

918

passing to ancestor handlers stops).

919

Just as for loggers, handlers can have levels associated with them. A handler’s

920

level acts as a filter in the same way as a logger’s level does. If a handler

921

decides to actually dispatch an event, the <a class="reference internal" href="../library/logging.html#logging.Handler.emit" title="logging.Handler.emit"><code class="xref py py-meth docutils literal">emit()</code></a> method is used

922

to send the message to its destination. Most user-defined subclasses of

923

<code class="xref py py-class docutils literal">Handler</code> will need to override this <a class="reference internal" href="../library/logging.html#logging.Handler.emit" title="logging.Handler.emit"><code class="xref py py-meth docutils literal">emit()</code></a>.

924

925

<h3>Custom Levels<a class="headerlink" href="#custom-levels" title="Permalink to this headline">¶</a></h3>

926

Defining your own levels is possible, but should not be necessary, as the

927

existing levels have been chosen on the basis of practical experience.

928

However, if you are convinced that you need custom levels, great care should

929

be exercised when doing this, and it is possibly a very bad idea to define

930

custom levels if you are developing a library. That’s because if multiple

931

library authors all define their own custom levels, there is a chance that

932

the logging output from such multiple libraries used together will be

933

difficult for the using developer to control and/or interpret, because a

934

given numeric value might mean different things for different libraries.

935

</div>

936

</div>

937

938

<h2>Useful Handlers<a class="headerlink" href="#useful-handlers" title="Permalink to this headline">¶</a></h2>

939

In addition to the base <code class="xref py py-class docutils literal">Handler</code> class, many useful subclasses are

940

provided:

941

942

<li><a class="reference internal" href="../library/logging.handlers.html#logging.StreamHandler" title="logging.StreamHandler"><code class="xref py py-class docutils literal">StreamHandler</code></a> instances send messages to streams (file-like

943

objects).</li>

944

<li><a class="reference internal" href="../library/logging.handlers.html#logging.FileHandler" title="logging.FileHandler"><code class="xref py py-class docutils literal">FileHandler</code></a> instances send messages to disk files.</li>

945

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.BaseRotatingHandler" title="logging.handlers.BaseRotatingHandler"><code class="xref py py-class docutils literal">BaseRotatingHandler</code></a> is the base class for handlers that

946

rotate log files at a certain point. It is not meant to be instantiated

947

directly. Instead, use <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.RotatingFileHandler" title="logging.handlers.RotatingFileHandler"><code class="xref py py-class docutils literal">RotatingFileHandler</code></a> or

948

<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.TimedRotatingFileHandler" title="logging.handlers.TimedRotatingFileHandler"><code class="xref py py-class docutils literal">TimedRotatingFileHandler</code></a>.</li>

949

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.RotatingFileHandler" title="logging.handlers.RotatingFileHandler"><code class="xref py py-class docutils literal">RotatingFileHandler</code></a> instances send messages to disk

950

files, with support for maximum log file sizes and log file rotation.</li>

951

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.TimedRotatingFileHandler" title="logging.handlers.TimedRotatingFileHandler"><code class="xref py py-class docutils literal">TimedRotatingFileHandler</code></a> instances send messages to

952

disk files, rotating the log file at certain timed intervals.</li>

953

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SocketHandler" title="logging.handlers.SocketHandler"><code class="xref py py-class docutils literal">SocketHandler</code></a> instances send messages to TCP/IP

954

sockets. Since 3.4, Unix domain sockets are also supported.</li>

955

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.DatagramHandler" title="logging.handlers.DatagramHandler"><code class="xref py py-class docutils literal">DatagramHandler</code></a> instances send messages to UDP

956

sockets. Since 3.4, Unix domain sockets are also supported.</li>

957

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SMTPHandler" title="logging.handlers.SMTPHandler"><code class="xref py py-class docutils literal">SMTPHandler</code></a> instances send messages to a designated

958

email address.</li>

959

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SysLogHandler" title="logging.handlers.SysLogHandler"><code class="xref py py-class docutils literal">SysLogHandler</code></a> instances send messages to a Unix

960

syslog daemon, possibly on a remote machine.</li>

961

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.NTEventLogHandler" title="logging.handlers.NTEventLogHandler"><code class="xref py py-class docutils literal">NTEventLogHandler</code></a> instances send messages to a

962

Windows NT/2000/XP event log.</li>

963

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.MemoryHandler" title="logging.handlers.MemoryHandler"><code class="xref py py-class docutils literal">MemoryHandler</code></a> instances send messages to a buffer

964

in memory, which is flushed whenever specific criteria are met.</li>

965

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.HTTPHandler" title="logging.handlers.HTTPHandler"><code class="xref py py-class docutils literal">HTTPHandler</code></a> instances send messages to an HTTP

966

server using either <code class="docutils literal">GET</code> or <code class="docutils literal">POST</code> semantics.</li>

967

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.WatchedFileHandler" title="logging.handlers.WatchedFileHandler"><code class="xref py py-class docutils literal">WatchedFileHandler</code></a> instances watch the file they are

968

logging to. If the file changes, it is closed and reopened using the file

969

name. This handler is only useful on Unix-like systems; Windows does not

970

support the underlying mechanism used.</li>

971

<li><a class="reference internal" href="../library/logging.handlers.html#logging.handlers.QueueHandler" title="logging.handlers.QueueHandler"><code class="xref py py-class docutils literal">QueueHandler</code></a> instances send messages to a queue, such as

972

those implemented in the <a class="reference internal" href="../library/queue.html#module-queue" title="queue: A synchronized queue class."><code class="xref py py-mod docutils literal">queue</code></a> or <a class="reference internal" href="../library/multiprocessing.html#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal">multiprocessing</code></a> modules.</li>

973

<li><a class="reference internal" href="../library/logging.handlers.html#logging.NullHandler" title="logging.NullHandler"><code class="xref py py-class docutils literal">NullHandler</code></a> instances do nothing with error messages. They are used

974

by library developers who want to use logging, but want to avoid the ‘No

975

handlers could be found for logger XXX’ message which can be displayed if

976

the library user has not configured logging. See <a class="reference internal" href="#library-config">Configuring Logging for a Library</a> for

977

more information.</li>

978

</ol>

979

980

New in version 3.1: The <a class="reference internal" href="../library/logging.handlers.html#logging.NullHandler" title="logging.NullHandler"><code class="xref py py-class docutils literal">NullHandler</code></a> class.

981

</div>

982

983

New in version 3.2: The <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.QueueHandler" title="logging.handlers.QueueHandler"><code class="xref py py-class docutils literal">QueueHandler</code></a> class.

984

</div>

985

The <a class="reference internal" href="../library/logging.handlers.html#logging.NullHandler" title="logging.NullHandler"><code class="xref py py-class docutils literal">NullHandler</code></a>, <a class="reference internal" href="../library/logging.handlers.html#logging.StreamHandler" title="logging.StreamHandler"><code class="xref py py-class docutils literal">StreamHandler</code></a> and <a class="reference internal" href="../library/logging.handlers.html#logging.FileHandler" title="logging.FileHandler"><code class="xref py py-class docutils literal">FileHandler</code></a>

986

classes are defined in the core logging package. The other handlers are

987

defined in a sub- module, <a class="reference internal" href="../library/logging.handlers.html#module-logging.handlers" title="logging.handlers: Handlers for the logging module."><code class="xref py py-mod docutils literal">logging.handlers</code></a>. (There is also another

988

sub-module, <a class="reference internal" href="../library/logging.config.html#module-logging.config" title="logging.config: Configuration of the logging module."><code class="xref py py-mod docutils literal">logging.config</code></a>, for configuration functionality.)

989

Logged messages are formatted for presentation through instances of the

990

<a class="reference internal" href="../library/logging.html#logging.Formatter" title="logging.Formatter"><code class="xref py py-class docutils literal">Formatter</code></a> class. They are initialized with a format string suitable for

991

use with the % operator and a dictionary.

992

For formatting multiple messages in a batch, instances of

993

<code class="xref py py-class docutils literal">BufferingFormatter</code> can be used. In addition to the format

994

string (which is applied to each message in the batch), there is provision for

995

header and trailer format strings.

996

When filtering based on logger level and/or handler level is not enough,

997

instances of <a class="reference internal" href="../library/logging.html#logging.Filter" title="logging.Filter"><code class="xref py py-class docutils literal">Filter</code></a> can be added to both <a class="reference internal" href="../library/logging.html#logging.Logger" title="logging.Logger"><code class="xref py py-class docutils literal">Logger</code></a> and

998

<code class="xref py py-class docutils literal">Handler</code> instances (through their <a class="reference internal" href="../library/logging.html#logging.Handler.addFilter" title="logging.Handler.addFilter"><code class="xref py py-meth docutils literal">addFilter()</code></a> method).

999

Before deciding to process a message further, both loggers and handlers consult

1000

all their filters for permission. If any filter returns a false value, the

1001

message is not processed further.

1002

The basic <a class="reference internal" href="../library/logging.html#logging.Filter" title="logging.Filter"><code class="xref py py-class docutils literal">Filter</code></a> functionality allows filtering by specific logger

1003

name. If this feature is used, messages sent to the named logger and its

1004

children are allowed through the filter, and all others dropped.

1005

</div>

1006

1007

<h2>Exceptions raised during logging<a class="headerlink" href="#exceptions-raised-during-logging" title="Permalink to this headline">¶</a></h2>

1008

The logging package is designed to swallow exceptions which occur while logging

1009

in production. This is so that errors which occur while handling logging events

1010

- such as logging misconfiguration, network or other similar errors - do not

1011

cause the application using logging to terminate prematurely.

1012

<a class="reference internal" href="../library/exceptions.html#SystemExit" title="SystemExit"><code class="xref py py-class docutils literal">SystemExit</code></a> and <a class="reference internal" href="../library/exceptions.html#KeyboardInterrupt" title="KeyboardInterrupt"><code class="xref py py-class docutils literal">KeyboardInterrupt</code></a> exceptions are never

1013

swallowed. Other exceptions which occur during the <a class="reference internal" href="../library/logging.html#logging.Handler.emit" title="logging.Handler.emit"><code class="xref py py-meth docutils literal">emit()</code></a> method

1014

of a <code class="xref py py-class docutils literal">Handler</code> subclass are passed to its <a class="reference internal" href="../library/logging.html#logging.Handler.handleError" title="logging.Handler.handleError"><code class="xref py py-meth docutils literal">handleError()</code></a>

1015

method.

1016

The default implementation of <a class="reference internal" href="../library/logging.html#logging.Handler.handleError" title="logging.Handler.handleError"><code class="xref py py-meth docutils literal">handleError()</code></a> in <code class="xref py py-class docutils literal">Handler</code>

1017

checks to see if a module-level variable, <code class="xref py py-data docutils literal">raiseExceptions</code>, is set. If

1018

set, a traceback is printed to <a class="reference internal" href="../library/sys.html#sys.stderr" title="sys.stderr"><code class="xref py py-data docutils literal">sys.stderr</code></a>. If not set, the exception is

1019

swallowed.

1020

1021

Note

1022

The default value of <code class="xref py py-data docutils literal">raiseExceptions</code> is <code class="docutils literal">True</code>. This is

1023

because during development, you typically want to be notified of any

1024

exceptions that occur. It’s advised that you set <code class="xref py py-data docutils literal">raiseExceptions</code> to

1025

<code class="docutils literal">False</code> for production usage.

1026

</div>

1027

</div>

1028

1029

<h2>Using arbitrary objects as messages<a class="headerlink" href="#using-arbitrary-objects-as-messages" title="Permalink to this headline">¶</a></h2>

1030

In the preceding sections and examples, it has been assumed that the message

1031

passed when logging the event is a string. However, this is not the only

1032

possibility. You can pass an arbitrary object as a message, and its

1033

<a class="reference internal" href="../reference/datamodel.html#object.__str__" title="object.__str__"><code class="xref py py-meth docutils literal">__str__()</code></a> method will be called when the logging system needs to

1034

convert it to a string representation. In fact, if you want to, you can avoid

1035

computing a string representation altogether - for example, the

1036

<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SocketHandler" title="logging.handlers.SocketHandler"><code class="xref py py-class docutils literal">SocketHandler</code></a> emits an event by pickling it and sending it

1037

over the wire.

1038

</div>

1039

1040

<h2>Optimization<a class="headerlink" href="#optimization" title="Permalink to this headline">¶</a></h2>

1041

Formatting of message arguments is deferred until it cannot be avoided.

1042

However, computing the arguments passed to the logging method can also be

1043

expensive, and you may want to avoid doing it if the logger will just throw

1044

away your event. To decide what to do, you can call the

1045

<a class="reference internal" href="../library/logging.html#logging.Logger.isEnabledFor" title="logging.Logger.isEnabledFor"><code class="xref py py-meth docutils literal">isEnabledFor()</code></a> method which takes a level argument and returns

1046

true if the event would be created by the Logger for that level of call.

1047

You can write code like this:

1048

<div class="highlight-python3"><div class="highlight"><pre>if logger.isEnabledFor(logging.DEBUG):

1049

logger.debug('Message with %s, %s', expensive_func1(),

1050

expensive_func2())

1051

</pre></div>

1052

</div>

1053

so that if the logger’s threshold is set above <code class="docutils literal">DEBUG</code>, the calls to

1054

<code class="xref py py-func docutils literal">expensive_func1()</code> and <code class="xref py py-func docutils literal">expensive_func2()</code> are never made.

1055

1056

Note

1057

In some cases, <a class="reference internal" href="../library/logging.html#logging.Logger.isEnabledFor" title="logging.Logger.isEnabledFor"><code class="xref py py-meth docutils literal">isEnabledFor()</code></a> can itself be more

1058

expensive than you’d like (e.g. for deeply nested loggers where an explicit

1059

level is only set high up in the logger hierarchy). In such cases (or if you

1060

want to avoid calling a method in tight loops), you can cache the result of a

1061

call to <a class="reference internal" href="../library/logging.html#logging.Logger.isEnabledFor" title="logging.Logger.isEnabledFor"><code class="xref py py-meth docutils literal">isEnabledFor()</code></a> in a local or instance variable, and use

1062

that instead of calling the method each time. Such a cached value would only

1063

need to be recomputed when the logging configuration changes dynamically

1064

while the application is running (which is not all that common).

1065

</div>

1066

There are other optimizations which can be made for specific applications which

1067

need more precise control over what logging information is collected. Here’s a

1068

list of things you can do to avoid processing during logging which you don’t

1069

need:

1070

1071

1072

1073

1074

</colgroup>

1075

1076

<tr class="row-odd"><th class="head">What you don’t want to collect</th>

1077

<th class="head">How to avoid collecting it</th>

1078

</tr>

1079

</thead>

1080

1081

<tr class="row-even"><td>Information about where calls were made from.</td>

1082

<td>Set <code class="docutils literal">logging._srcfile</code> to <code class="docutils literal">None</code>.

1083

This avoids calling

1084

<a class="reference internal" href="../library/sys.html#sys._getframe" title="sys._getframe"><code class="xref py py-func docutils literal">sys._getframe()</code></a>, which may help

1085

to speed up your code in environments

1086

like PyPy (which can’t speed up code

1087

that uses <a class="reference internal" href="../library/sys.html#sys._getframe" title="sys._getframe"><code class="xref py py-func docutils literal">sys._getframe()</code></a>), if

1088

and when PyPy supports Python 3.x.</td>

1089

</tr>

1090

<tr class="row-odd"><td>Threading information.</td>

1091

<td>Set <code class="docutils literal">logging.logThreads</code> to <code class="docutils literal">0</code>.</td>

1092

</tr>

1093

<tr class="row-even"><td>Process information.</td>

1094

<td>Set <code class="docutils literal">logging.logProcesses</code> to <code class="docutils literal">0</code>.</td>

1095

</tr>

1096

</tbody>

1097

</table>

1098

Also note that the core logging module only includes the basic handlers. If

1099

you don’t import <a class="reference internal" href="../library/logging.handlers.html#module-logging.handlers" title="logging.handlers: Handlers for the logging module."><code class="xref py py-mod docutils literal">logging.handlers</code></a> and <a class="reference internal" href="../library/logging.config.html#module-logging.config" title="logging.config: Configuration of the logging module."><code class="xref py py-mod docutils literal">logging.config</code></a>, they won’t

1100

take up any memory.

1101

1102

See also

1103

1104

<dt>Module <a class="reference internal" href="../library/logging.html#module-logging" title="logging: Flexible event logging system for applications."><code class="xref py py-mod docutils literal">logging</code></a></dt>

1105

<dd>API reference for the logging module.</dd>

1106

<dt>Module <a class="reference internal" href="../library/logging.config.html#module-logging.config" title="logging.config: Configuration of the logging module."><code class="xref py py-mod docutils literal">logging.config</code></a></dt>

1107

<dd>Configuration API for the logging module.</dd>

1108

<dt>Module <a class="reference internal" href="../library/logging.handlers.html#module-logging.handlers" title="logging.handlers: Handlers for the logging module."><code class="xref py py-mod docutils literal">logging.handlers</code></a></dt>

1109

<dd>Useful handlers included with the logging module.</dd>

1110

</dl>

1111

<a class="reference internal" href="logging-cookbook.html#logging-cookbook">A logging cookbook</a>

1112

</div>

1113

</div>

1114

</div>

1115

1116

1117

</div>

1118

</div>

1119

</div>

1120

1121

1122

<h3><a href="../contents.html">Table Of Contents</a></h3>

1123

<ul>

1124

<li><a class="reference internal" href="#">Logging HOWTO</a><ul>

1125

<li><a class="reference internal" href="#basic-logging-tutorial">Basic Logging Tutorial</a><ul>

1126

<li><a class="reference internal" href="#when-to-use-logging">When to use logging</a></li>

1127

<li><a class="reference internal" href="#a-simple-example">A simple example</a></li>

1128

<li><a class="reference internal" href="#logging-to-a-file">Logging to a file</a></li>

1129

<li><a class="reference internal" href="#logging-from-multiple-modules">Logging from multiple modules</a></li>

1130

<li><a class="reference internal" href="#logging-variable-data">Logging variable data</a></li>

1131

<li><a class="reference internal" href="#changing-the-format-of-displayed-messages">Changing the format of displayed messages</a></li>

1132

<li><a class="reference internal" href="#displaying-the-date-time-in-messages">Displaying the date/time in messages</a></li>

1133

<li><a class="reference internal" href="#next-steps">Next Steps</a></li>

1134

</ul>

1135

</li>

1136

<li><a class="reference internal" href="#advanced-logging-tutorial">Advanced Logging Tutorial</a><ul>

1137

<li><a class="reference internal" href="#logging-flow">Logging Flow</a></li>

1138

<li><a class="reference internal" href="#loggers">Loggers</a></li>

1139

<li><a class="reference internal" href="#handlers">Handlers</a></li>

1140

<li><a class="reference internal" href="#formatters">Formatters</a></li>

1141

<li><a class="reference internal" href="#configuring-logging">Configuring Logging</a></li>

1142

<li><a class="reference internal" href="#what-happens-if-no-configuration-is-provided">What happens if no configuration is provided</a></li>

1143

<li><a class="reference internal" href="#configuring-logging-for-a-library">Configuring Logging for a Library</a></li>

1144

</ul>

1145

</li>

1146

<li><a class="reference internal" href="#logging-levels">Logging Levels</a><ul>

1147

<li><a class="reference internal" href="#custom-levels">Custom Levels</a></li>

1148

</ul>

1149

</li>

1150

<li><a class="reference internal" href="#useful-handlers">Useful Handlers</a></li>

1151

<li><a class="reference internal" href="#exceptions-raised-during-logging">Exceptions raised during logging</a></li>

1152

<li><a class="reference internal" href="#using-arbitrary-objects-as-messages">Using arbitrary objects as messages</a></li>

1153

<li><a class="reference internal" href="#optimization">Optimization</a></li>

1154

</ul>

1155

</li>

1156

</ul>

1157

1158

<h4>Previous topic</h4>

1159

<a href="functional.html"

1160

title="previous chapter">Functional Programming HOWTO</a>

1161

<h4>Next topic</h4>

1162

<a href="logging-cookbook.html"

1163

title="next chapter">Logging Cookbook</a>

1164

1165

1166

1167

<li><a href="../bugs.html">Report a Bug</a></li>

1168

<li><a href="../_sources/howto/logging.txt"

1169

rel="nofollow">Show Source</a></li>

1170

</ul>

1171

</div>

1172

</div>

1173

</div>

1174

1175

</div>

1176

1177

<h3>Navigation</h3>

1178

<ul>

1179

1180

<a href="../genindex.html" title="General Index"

1181

>index</a></li>

1182

1183

<a href="../py-modindex.html" title="Python Module Index"

1184

>modules</a> |</li>

1185

1186

<a href="logging-cookbook.html" title="Logging Cookbook"

1187

>next</a> |</li>

1188

1189

<a href="functional.html" title="Functional Programming HOWTO"

1190

>previous</a> |</li>

1191

<li><img src="../_static/py.png" alt=""

1192

style="vertical-align: middle; margin-top: -1px"/></li>

1193

<li><a href="https://www.python.org/">Python</a> »</li>

1194

<li>

1195

3.5.2

1196

<a href="../index.html">Documentation </a> »

1197

</li>

1198

1199

<li class="nav-item nav-item-1"><a href="index.html" >Python HOWTOs</a> »</li>

1200

1201

1202

1203

1204

1205

1206

1207

1208

1209

</form>

1210

</div>

1211

1212

1213

</li>

1214

1215

</ul>

1216

</div>

1217

1218

1219

1220

The Python Software Foundation is a non-profit corporation.

1221

<a href="https://www.python.org/psf/donations/">Please donate.</a>

1222

1223

Last updated on Oct 19, 2016.

1224

<a href="../bugs.html">Found a bug</a>?

1225

1226

Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.3.3.

1227

</div>

1228

1229

</body>

1230

</html>

b'\\ No newline at end of file'

Older »