~dkuhlman/python-training-materials/Materials

have been added such as <a class="reference internal" href="../library/functions.html#sum" title="sum"><code class="xref py py-func docutils literal">sum()</code></a> and <a class="reference internal" href="../library/functions.html#enumerate" title="enumerate"><code class="xref py py-func docutils literal">enumerate()</code></a>. The <a class="reference internal" href="../reference/expressions.html#in"><code class="xref std std-keyword docutils literal">in</code></a>

operator can now be used for substring searches (e.g. <code class="docutils literal">"ab" in "abc"</code> returns

<a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a>).

Some of the many new library features include Boolean, set, heap, and date/time

data types, the ability to import modules from ZIP-format archives, metadata

support for the long-awaited Python catalog, an updated version of IDLE, and

modules for logging messages, wrapping text, parsing CSV files, processing

100

command-line options, using BerkeleyDB databases... the list of new and

101

enhanced modules is lengthy.

102

This article doesn’t attempt to provide a complete specification of the new

103

features, but instead provides a convenient overview. For full details, you

104

should refer to the documentation for Python 2.3, such as the Python Library

105

Reference and the Python Reference Manual. If you want to understand the

106

complete implementation and design rationale, refer to the PEP for a particular

107

new feature.

108

109

<h2>PEP 218: A Standard Set Datatype<a class="headerlink" href="#pep-218-a-standard-set-datatype" title="Permalink to this headline">¶</a></h2>

110

The new <a class="reference internal" href="../library/sets.html#module-sets" title="sets: Implementation of sets of unique elements. (deprecated)"><code class="xref py py-mod docutils literal">sets</code></a> module contains an implementation of a set datatype. The

111

<code class="xref py py-class docutils literal">Set</code> class is for mutable sets, sets that can have members added and

112

removed. The <code class="xref py py-class docutils literal">ImmutableSet</code> class is for sets that can’t be modified,

113

and instances of <code class="xref py py-class docutils literal">ImmutableSet</code> can therefore be used as dictionary keys.

114

Sets are built on top of dictionaries, so the elements within a set must be

115

hashable.

116

Here’s a simple example:

117

<div class="highlight-python"><div class="highlight"><pre>>>> import sets

118

>>> S = sets.Set([1,2,3])

119

>>> S

120

Set([1, 2, 3])

121

>>> 1 in S

122

True

123

>>> 0 in S

124

False

125

>>> S.add(5)

126

>>> S.remove(3)

127

>>> S

128

Set([1, 2, 5])

129

>>>

130

</pre></div>

131

</div>

132

The union and intersection of sets can be computed with the <code class="xref py py-meth docutils literal">union()</code> and

133

<code class="xref py py-meth docutils literal">intersection()</code> methods; an alternative notation uses the bitwise operators

134

<code class="docutils literal">&</code> and <code class="docutils literal">|</code>. Mutable sets also have in-place versions of these methods,

135

<code class="xref py py-meth docutils literal">union_update()</code> and <code class="xref py py-meth docutils literal">intersection_update()</code>.

136

<div class="highlight-python"><div class="highlight"><pre>>>> S1 = sets.Set([1,2,3])

137

>>> S2 = sets.Set([4,5,6])

138

>>> S1.union(S2)

139

Set([1, 2, 3, 4, 5, 6])

140

>>> S1 | S2 # Alternative notation

141

Set([1, 2, 3, 4, 5, 6])

142

>>> S1.intersection(S2)

143

Set([])

144

>>> S1 & S2 # Alternative notation

145

Set([])

146

>>> S1.union_update(S2)

147

>>> S1

148

Set([1, 2, 3, 4, 5, 6])

149

>>>

150

</pre></div>

151

</div>

152

It’s also possible to take the symmetric difference of two sets. This is the

153

set of all elements in the union that aren’t in the intersection. Another way

154

of putting it is that the symmetric difference contains all elements that are in

155

exactly one set. Again, there’s an alternative notation (<code class="docutils literal">^</code>), and an in-

156

place version with the ungainly name <code class="xref py py-meth docutils literal">symmetric_difference_update()</code>.

157

158

>>> S2 = sets.Set([3,4,5,6])

159

>>> S1.symmetric_difference(S2)

160

Set([1, 2, 5, 6])

161

>>> S1 ^ S2

162

Set([1, 2, 5, 6])

163

>>>

164

</pre></div>

165

</div>

166

There are also <code class="xref py py-meth docutils literal">issubset()</code> and <code class="xref py py-meth docutils literal">issuperset()</code> methods for checking

167

whether one set is a subset or superset of another:

168

169

>>> S2 = sets.Set([2,3])

170

>>> S2.issubset(S1)

171

True

172

>>> S1.issubset(S2)

173

False

174

>>> S1.issuperset(S2)

175

True

176

>>>

177

</pre></div>

178

</div>

179

180

See also

181

182

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0218">PEP 218</a> - Adding a Built-In Set Object Type</dt>

183

<dd>PEP written by Greg V. Wilson. Implemented by Greg V. Wilson, Alex Martelli, and

184

GvR.</dd>

185

</dl>

186

</div>

187

</div>

188

189

<h2>PEP 255: Simple Generators<a class="headerlink" href="#pep-255-simple-generators" title="Permalink to this headline">¶</a></h2>

190

In Python 2.2, generators were added as an optional feature, to be enabled by a

191

<code class="docutils literal">from __future__ import generators</code> directive. In 2.3 generators no longer

192

need to be specially enabled, and are now always present; this means that

193

194

the description of generators from the “What’s New in Python 2.2” document; if

195

you read it back when Python 2.2 came out, you can skip the rest of this

196

section.

197

You’re doubtless familiar with how function calls work in Python or C. When you

198

call a function, it gets a private namespace where its local variables are

199

created. When the function reaches a <a class="reference internal" href="../reference/simple_stmts.html#return"><code class="xref std std-keyword docutils literal">return</code></a> statement, the local

200

variables are destroyed and the resulting value is returned to the caller. A

201

later call to the same function will get a fresh new set of local variables.

202

But, what if the local variables weren’t thrown away on exiting a function?

203

What if you could later resume the function where it left off? This is what

204

generators provide; they can be thought of as resumable functions.

205

Here’s the simplest example of a generator function:

206

<div class="highlight-python"><div class="highlight"><pre>def generate_ints(N):

207

for i in range(N):

208

yield i

209

</pre></div>

210

</div>

211

A new keyword, <a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a>, was introduced for generators. Any function

212

containing a <a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a> statement is a generator function; this is

213

detected by Python’s bytecode compiler which compiles the function specially as

214

a result.

215

When you call a generator function, it doesn’t return a single value; instead it

216

returns a generator object that supports the iterator protocol. On executing

217

the <a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a> statement, the generator outputs the value of <code class="docutils literal">i</code>,

218

similar to a <a class="reference internal" href="../reference/simple_stmts.html#return"><code class="xref std std-keyword docutils literal">return</code></a> statement. The big difference between

219

<a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a> and a <a class="reference internal" href="../reference/simple_stmts.html#return"><code class="xref std std-keyword docutils literal">return</code></a> statement is that on reaching a

220

221

variables are preserved. On the next call to the generator’s <code class="docutils literal">.next()</code>

222

method, the function will resume executing immediately after the

223

<a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a> statement. (For complicated reasons, the <a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a>

224

statement isn’t allowed inside the <a class="reference internal" href="../reference/compound_stmts.html#try"><code class="xref std std-keyword docutils literal">try</code></a> block of a <a class="reference internal" href="../reference/compound_stmts.html#try"><code class="xref std std-keyword docutils literal">try</code></a>...<a class="reference internal" href="../reference/compound_stmts.html#finally"><code class="xref std std-keyword docutils literal">finally</code></a> statement; read <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0255">PEP 255</a> for a full explanation of the

225

interaction between <a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a> and exceptions.)

226

Here’s a sample usage of the <code class="xref py py-func docutils literal">generate_ints()</code> generator:

227

<div class="highlight-python"><div class="highlight"><pre>>>> gen = generate_ints(3)

228

>>> gen

229

230

>>> gen.next()

231

0

232

>>> gen.next()

233

1

234

>>> gen.next()

235

2

236

>>> gen.next()

237

Traceback (most recent call last):

238

File "stdin", line 1, in ?

239

File "stdin", line 2, in generate_ints

240

StopIteration

241

</pre></div>

242

</div>

243

You could equally write <code class="docutils literal">for i in generate_ints(5)</code>, or <code class="docutils literal">a,b,c =

244

generate_ints(3)</code>.

245

Inside a generator function, the <a class="reference internal" href="../reference/simple_stmts.html#return"><code class="xref std std-keyword docutils literal">return</code></a> statement can only be used

246

without a value, and signals the end of the procession of values; afterwards the

247

generator cannot return any further values. <a class="reference internal" href="../reference/simple_stmts.html#return"><code class="xref std std-keyword docutils literal">return</code></a> with a value, such

248

as <code class="docutils literal">return 5</code>, is a syntax error inside a generator function. The end of the

249

generator’s results can also be indicated by raising <a class="reference internal" href="../library/exceptions.html#exceptions.StopIteration" title="exceptions.StopIteration"><code class="xref py py-exc docutils literal">StopIteration</code></a>

250

manually, or by just letting the flow of execution fall off the bottom of the

251

function.

252

You could achieve the effect of generators manually by writing your own class

253

and storing all the local variables of the generator as instance variables. For

254

example, returning a list of integers could be done by setting <code class="docutils literal">self.count</code> to

255

0, and having the <a class="reference internal" href="../library/functions.html#next" title="next"><code class="xref py py-meth docutils literal">next()</code></a> method increment <code class="docutils literal">self.count</code> and return it.

256

However, for a moderately complicated generator, writing a corresponding class

257

would be much messier. <code class="file docutils literal">Lib/test/test_generators.py</code> contains a number of

258

more interesting examples. The simplest one implements an in-order traversal of

259

a tree using generators recursively.

260

<div class="highlight-python"><div class="highlight"><pre># A recursive generator that generates Tree leaves in in-order.

261

def inorder(t):

262

if t:

263

for x in inorder(t.left):

264

yield x

265

yield t.label

266

267

yield x

268

</pre></div>

269

</div>

270

Two other examples in <code class="file docutils literal">Lib/test/test_generators.py</code> produce solutions for

271

the N-Queens problem (placing $N$ queens on an $NxN$ chess board so that no

272

queen threatens another) and the Knight’s Tour (a route that takes a knight to

273

every square of an $NxN$ chessboard without visiting any square twice).

274

The idea of generators comes from other programming languages, especially Icon

275

(<a class="reference external" href="http://www.cs.arizona.edu/icon/">http://www.cs.arizona.edu/icon/</a>), where the idea of generators is central. In

276

Icon, every expression and function call behaves like a generator. One example

277

from “An Overview of the Icon Programming Language” at

278

<a class="reference external" href="http://www.cs.arizona.edu/icon/docs/ipd266.htm">http://www.cs.arizona.edu/icon/docs/ipd266.htm</a> gives an idea of what this looks

279

like:

280

<div class="highlight-python"><div class="highlight"><pre>sentence := "Store it in the neighboring harbor"

281

if (i := find("or", sentence)) > 5 then write(i)

282

</pre></div>

283

</div>

284

In Icon the <code class="xref py py-func docutils literal">find()</code> function returns the indexes at which the substring

285

“or” is found: 3, 23, 33. In the <a class="reference internal" href="../reference/compound_stmts.html#if"><code class="xref std std-keyword docutils literal">if</code></a> statement, <code class="docutils literal">i</code> is first

286

assigned a value of 3, but 3 is less than 5, so the comparison fails, and Icon

287

retries it with the second value of 23. 23 is greater than 5, so the comparison

288

now succeeds, and the code prints the value 23 to the screen.

289

Python doesn’t go nearly as far as Icon in adopting generators as a central

290

concept. Generators are considered part of the core Python language, but

291

learning or using them isn’t compulsory; if they don’t solve any problems that

292

you have, feel free to ignore them. One novel feature of Python’s interface as

293

compared to Icon’s is that a generator’s state is represented as a concrete

294

object (the iterator) that can be passed around to other functions or stored in

295

a data structure.

296

297

See also

298

299

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0255">PEP 255</a> - Simple Generators</dt>

300

<dd>Written by Neil Schemenauer, Tim Peters, Magnus Lie Hetland. Implemented mostly

301

by Neil Schemenauer and Tim Peters, with other fixes from the Python Labs crew.</dd>

302

</dl>

303

</div>

304

</div>

305

306

<h2>PEP 263: Source Code Encodings<a class="headerlink" href="#pep-263-source-code-encodings" title="Permalink to this headline">¶</a></h2>

307

Python source files can now be declared as being in different character set

308

encodings. Encodings are declared by including a specially formatted comment in

309

the first or second line of the source file. For example, a UTF-8 file can be

310

declared with:

311

<div class="highlight-python"><div class="highlight"><pre>#!/usr/bin/env python

312

# -*- coding: UTF-8 -*-

313

</pre></div>

314

</div>

315

Without such an encoding declaration, the default encoding used is 7-bit ASCII.

316

Executing or importing modules that contain string literals with 8-bit

317

characters and have no encoding declaration will result in a

318

<a class="reference internal" href="../library/exceptions.html#exceptions.DeprecationWarning" title="exceptions.DeprecationWarning"><code class="xref py py-exc docutils literal">DeprecationWarning</code></a> being signalled by Python 2.3; in 2.4 this will be a

319

syntax error.

320

The encoding declaration only affects Unicode string literals, which will be

321

converted to Unicode using the specified encoding. Note that Python identifiers

322

are still restricted to ASCII characters, so you can’t have variable names that

323

use characters outside of the usual alphanumerics.

324

325

See also

326

327

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0263">PEP 263</a> - Defining Python Source Code Encodings</dt>

328

<dd>Written by Marc-André Lemburg and Martin von Löwis; implemented by Suzuki Hisao

329

and Martin von Löwis.</dd>

330

</dl>

331

</div>

332

</div>

333

334

<h2>PEP 273: Importing Modules from ZIP Archives<a class="headerlink" href="#pep-273-importing-modules-from-zip-archives" title="Permalink to this headline">¶</a></h2>

335

The new <a class="reference internal" href="../library/zipimport.html#module-zipimport" title="zipimport: support for importing Python modules from ZIP archives."><code class="xref py py-mod docutils literal">zipimport</code></a> module adds support for importing modules from a ZIP-

336

format archive. You don’t need to import the module explicitly; it will be

337

automatically imported if a ZIP archive’s filename is added to <code class="docutils literal">sys.path</code>.

338

For example:

339

<div class="highlight-python"><div class="highlight"><pre>amk@nyman:~/src/python$ unzip -l /tmp/example.zip

340

Archive: /tmp/example.zip

341

Length Date Time Name

342

-------- ---- ---- ----

343

8467 11-26-02 22:30 jwzthreading.py

344

-------- -------

345

8467 1 file

346

amk@nyman:~/src/python$ ./python

347

Python 2.3 (#1, Aug 1 2003, 19:54:32)

348

>>> import sys

349

>>> sys.path.insert(0, '/tmp/example.zip') # Add .zip file to front of path

350

>>> import jwzthreading

351

>>> jwzthreading.__file__

352

'/tmp/example.zip/jwzthreading.py'

353

>>>

354

</pre></div>

355

</div>

356

An entry in <code class="docutils literal">sys.path</code> can now be the filename of a ZIP archive. The ZIP

357

archive can contain any kind of files, but only files named <code class="file docutils literal">*.py</code>,

358

<code class="file docutils literal">*.pyc</code>, or <code class="file docutils literal">*.pyo</code> can be imported. If an archive only contains

359

<code class="file docutils literal">*.py</code> files, Python will not attempt to modify the archive by adding the

360

corresponding <code class="file docutils literal">*.pyc</code> file, meaning that if a ZIP archive doesn’t contain

361

<code class="file docutils literal">*.pyc</code> files, importing may be rather slow.

362

A path within the archive can also be specified to only import from a

363

subdirectory; for example, the path <code class="file docutils literal">/tmp/example.zip/lib/</code> would only

364

import from the <code class="file docutils literal">lib/</code> subdirectory within the archive.

365

366

See also

367

368

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0273">PEP 273</a> - Import Modules from Zip Archives</dt>

369

<dd>Written by James C. Ahlstrom, who also provided an implementation. Python 2.3

370

follows the specification in <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0273">PEP 273</a>, but uses an implementation written by

371

Just van Rossum that uses the import hooks described in <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0302">PEP 302</a>. See section

372

<a class="reference internal" href="#section-pep302">PEP 302: New Import Hooks</a> for a description of the new import hooks.</dd>

373

</dl>

374

</div>

375

</div>

376

377

<h2>PEP 277: Unicode file name support for Windows NT<a class="headerlink" href="#pep-277-unicode-file-name-support-for-windows-nt" title="Permalink to this headline">¶</a></h2>

378

On Windows NT, 2000, and XP, the system stores file names as Unicode strings.

379

Traditionally, Python has represented file names as byte strings, which is

380

inadequate because it renders some file names inaccessible.

381

Python now allows using arbitrary Unicode strings (within the limitations of the

382

file system) for all functions that expect file names, most notably the

383

<a class="reference internal" href="../library/functions.html#open" title="open"><code class="xref py py-func docutils literal">open()</code></a> built-in function. If a Unicode string is passed to

384

<a class="reference internal" href="../library/os.html#os.listdir" title="os.listdir"><code class="xref py py-func docutils literal">os.listdir()</code></a>, Python now returns a list of Unicode strings. A new

385

function, <a class="reference internal" href="../library/os.html#os.getcwdu" title="os.getcwdu"><code class="xref py py-func docutils literal">os.getcwdu()</code></a>, returns the current directory as a Unicode string.

386

Byte strings still work as file names, and on Windows Python will transparently

387

convert them to Unicode using the <code class="docutils literal">mbcs</code> encoding.

388

Other systems also allow Unicode strings as file names but convert them to byte

389

strings before passing them to the system, which can cause a <a class="reference internal" href="../library/exceptions.html#exceptions.UnicodeError" title="exceptions.UnicodeError"><code class="xref py py-exc docutils literal">UnicodeError</code></a>

390

to be raised. Applications can test whether arbitrary Unicode strings are

391

supported as file names by checking <a class="reference internal" href="../library/os.path.html#os.path.supports_unicode_filenames" title="os.path.supports_unicode_filenames"><code class="xref py py-attr docutils literal">os.path.supports_unicode_filenames</code></a>,

392

a Boolean value.

393

Under MacOS, <a class="reference internal" href="../library/os.html#os.listdir" title="os.listdir"><code class="xref py py-func docutils literal">os.listdir()</code></a> may now return Unicode filenames.

394

395

See also

396

397

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0277">PEP 277</a> - Unicode file name support for Windows NT</dt>

398

<dd>Written by Neil Hodgson; implemented by Neil Hodgson, Martin von Löwis, and Mark

399

Hammond.</dd>

400

</dl>

401

</div>

402

</div>

403

404

<h2>PEP 278: Universal Newline Support<a class="headerlink" href="#pep-278-universal-newline-support" title="Permalink to this headline">¶</a></h2>

405

The three major operating systems used today are Microsoft Windows, Apple’s

406

Macintosh OS, and the various Unix derivatives. A minor irritation of cross-

407

platform work is that these three platforms all use different characters to

408

mark the ends of lines in text files. Unix uses the linefeed (ASCII character

409

10), MacOS uses the carriage return (ASCII character 13), and Windows uses a

410

two-character sequence of a carriage return plus a newline.

411

Python’s file objects can now support end of line conventions other than the

412

one followed by the platform on which Python is running. Opening a file with

413

the mode <code class="docutils literal">'U'</code> or <code class="docutils literal">'rU'</code> will open a file for reading in <a class="reference internal" href="../glossary.html#term-universal-newlines">universal

414

newlines</a> mode. All three line ending conventions will be translated to a

415

<code class="docutils literal">'\n'</code> in the strings returned by the various file methods such as

416

<code class="xref py py-meth docutils literal">read()</code> and <a class="reference internal" href="../library/readline.html#module-readline" title="readline: GNU readline support for Python. (Unix)"><code class="xref py py-meth docutils literal">readline()</code></a>.

417

Universal newline support is also used when importing modules and when executing

418

a file with the <a class="reference internal" href="../library/functions.html#execfile" title="execfile"><code class="xref py py-func docutils literal">execfile()</code></a> function. This means that Python modules can

419

be shared between all three operating systems without needing to convert the

420

line-endings.

421

This feature can be disabled when compiling Python by specifying the

422

<code class="xref std std-option docutils literal">--without-universal-newlines</code> switch when running Python’s

423

configure script.

424

425

See also

426

427

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0278">PEP 278</a> - Universal Newline Support</dt>

428

<dd>Written and implemented by Jack Jansen.</dd>

429

</dl>

430

</div>

431

</div>

432

433

<h2>PEP 279: enumerate()<a class="headerlink" href="#pep-279-enumerate" title="Permalink to this headline">¶</a></h2>

434

A new built-in function, <a class="reference internal" href="../library/functions.html#enumerate" title="enumerate"><code class="xref py py-func docutils literal">enumerate()</code></a>, will make certain loops a bit

435

clearer. <code class="docutils literal">enumerate(thing)</code>, where thing is either an iterator or a

436

sequence, returns a iterator that will return <code class="docutils literal">(0, thing[0])</code>, <code class="docutils literal">(1,

437

thing[1])</code>, <code class="docutils literal">(2, thing[2])</code>, and so forth.

438

A common idiom to change every element of a list looks like this:

439

<div class="highlight-python"><div class="highlight"><pre>for i in range(len(L)):

440

item = L[i]

441

# ... compute some result based on item ...

442

L[i] = result

443

</pre></div>

444

</div>

445

This can be rewritten using <a class="reference internal" href="../library/functions.html#enumerate" title="enumerate"><code class="xref py py-func docutils literal">enumerate()</code></a> as:

446

<div class="highlight-python"><div class="highlight"><pre>for i, item in enumerate(L):

447

# ... compute some result based on item ...

448

L[i] = result

449

</pre></div>

450

</div>

451

452

See also

453

454

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0279">PEP 279</a> - The enumerate() built-in function</dt>

455

<dd>Written and implemented by Raymond D. Hettinger.</dd>

456

</dl>

457

</div>

458

</div>

459

460

<h2>PEP 282: The logging Package<a class="headerlink" href="#pep-282-the-logging-package" title="Permalink to this headline">¶</a></h2>

461

A standard package for writing logs, <a class="reference internal" href="../library/logging.html#module-logging" title="logging: Flexible event logging system for applications."><code class="xref py py-mod docutils literal">logging</code></a>, has been added to Python

462

2.3. It provides a powerful and flexible mechanism for generating logging

463

output which can then be filtered and processed in various ways. A

464

configuration file written in a standard format can be used to control the

465

logging behavior of a program. Python includes handlers that will write log

466

records to standard error or to a file or socket, send them to the system log,

467

or even e-mail them to a particular address; of course, it’s also possible to

468

write your own handler classes.

469

The <code class="xref py py-class docutils literal">Logger</code> class is the primary class. Most application code will deal

470

with one or more <code class="xref py py-class docutils literal">Logger</code> objects, each one used by a particular

471

subsystem of the application. Each <code class="xref py py-class docutils literal">Logger</code> is identified by a name, and

472

names are organized into a hierarchy using <code class="docutils literal">.</code> as the component separator.

473

For example, you might have <code class="xref py py-class docutils literal">Logger</code> instances named <code class="docutils literal">server</code>,

474

<code class="docutils literal">server.auth</code> and <code class="docutils literal">server.network</code>. The latter two instances are below

475

<code class="docutils literal">server</code> in the hierarchy. This means that if you turn up the verbosity for

476

<code class="docutils literal">server</code> or direct <code class="docutils literal">server</code> messages to a different handler, the changes

477

will also apply to records logged to <code class="docutils literal">server.auth</code> and <code class="docutils literal">server.network</code>.

478

There’s also a root <code class="xref py py-class docutils literal">Logger</code> that’s the parent of all other loggers.

479

For simple uses, the <a class="reference internal" href="../library/logging.html#module-logging" title="logging: Flexible event logging system for applications."><code class="xref py py-mod docutils literal">logging</code></a> package contains some convenience functions

480

that always use the root log:

481

<div class="highlight-python"><div class="highlight"><pre>import logging

482

483

logging.debug('Debugging information')

484

logging.info('Informational message')

485

logging.warning('Warning:config file %s not found', 'server.conf')

486

logging.error('Error occurred')

487

logging.critical('Critical error -- shutting down')

488

</pre></div>

489

</div>

490

This produces the following output:

491

<div class="highlight-python"><div class="highlight"><pre>WARNING:root:Warning:config file server.conf not found

492

ERROR:root:Error occurred

493

CRITICAL:root:Critical error -- shutting down

494

</pre></div>

495

</div>

496

In the default configuration, informational and debugging messages are

497

suppressed and the output is sent to standard error. You can enable the display

498

of informational and debugging messages by calling the <code class="xref py py-meth docutils literal">setLevel()</code> method

499

on the root logger.

500

Notice the <code class="xref py py-func docutils literal">warning()</code> call’s use of string formatting operators; all of the

501

functions for logging messages take the arguments <code class="docutils literal">(msg, arg1, arg2, ...)</code> and

502

log the string resulting from <code class="docutils literal">msg % (arg1, arg2, ...)</code>.

503

There’s also an <code class="xref py py-func docutils literal">exception()</code> function that records the most recent

504

traceback. Any of the other functions will also record the traceback if you

505

specify a true value for the keyword argument exc_info.

506

<div class="highlight-python"><div class="highlight"><pre>def f():

507

try: 1/0

508

except: logging.exception('Problem recorded')

509

510

f()

511

</pre></div>

512

</div>

513

This produces the following output:

514

<div class="highlight-python"><div class="highlight"><pre>ERROR:root:Problem recorded

515

Traceback (most recent call last):

516

File "t.py", line 6, in f

517

1/0

518

ZeroDivisionError: integer division or modulo by zero

519

</pre></div>

520

</div>

521

Slightly more advanced programs will use a logger other than the root logger.

522

The <code class="xref py py-func docutils literal">getLogger(name)()</code> function is used to get a particular log, creating

523

it if it doesn’t exist yet. <code class="xref py py-func docutils literal">getLogger(None)()</code> returns the root logger.

524

<div class="highlight-python"><div class="highlight"><pre>log = logging.getLogger('server')

525

...

526

log.info('Listening on port %i', port)

527

...

528

log.critical('Disk full')

529

...

530

</pre></div>

531

</div>

532

Log records are usually propagated up the hierarchy, so a message logged to

533

<code class="docutils literal">server.auth</code> is also seen by <code class="docutils literal">server</code> and <code class="docutils literal">root</code>, but a <code class="xref py py-class docutils literal">Logger</code>

534

can prevent this by setting its <code class="xref py py-attr docutils literal">propagate</code> attribute to <a class="reference internal" href="../library/constants.html#False" title="False"><code class="xref py py-const docutils literal">False</code></a>.

535

There are more classes provided by the <a class="reference internal" href="../library/logging.html#module-logging" title="logging: Flexible event logging system for applications."><code class="xref py py-mod docutils literal">logging</code></a> package that can be

536

customized. When a <code class="xref py py-class docutils literal">Logger</code> instance is told to log a message, it

537

creates a <code class="xref py py-class docutils literal">LogRecord</code> instance that is sent to any number of different

538

<code class="xref py py-class docutils literal">Handler</code> instances. Loggers and handlers can also have an attached list

539

of filters, and each filter can cause the <code class="xref py py-class docutils literal">LogRecord</code> to be ignored or

540

can modify the record before passing it along. When they’re finally output,

541

<code class="xref py py-class docutils literal">LogRecord</code> instances are converted to text by a <code class="xref py py-class docutils literal">Formatter</code>

542

class. All of these classes can be replaced by your own specially-written

543

classes.

544

With all of these features the <a class="reference internal" href="../library/logging.html#module-logging" title="logging: Flexible event logging system for applications."><code class="xref py py-mod docutils literal">logging</code></a> package should provide enough

545

flexibility for even the most complicated applications. This is only an

546

incomplete overview of its features, so please see the package’s reference

547

documentation for all of the details. Reading <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0282">PEP 282</a> will also be helpful.

548

549

See also

550

551

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0282">PEP 282</a> - A Logging System</dt>

552

<dd>Written by Vinay Sajip and Trent Mick; implemented by Vinay Sajip.</dd>

553

</dl>

554

</div>

555

</div>

556

557

<h2>PEP 285: A Boolean Type<a class="headerlink" href="#pep-285-a-boolean-type" title="Permalink to this headline">¶</a></h2>

558

A Boolean type was added to Python 2.3. Two new constants were added to the

559

<a class="reference internal" href="../library/__builtin__.html#module-__builtin__" title="__builtin__: The module that provides the built-in namespace."><code class="xref py py-mod docutils literal">__builtin__</code></a> module, <a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a> and <a class="reference internal" href="../library/constants.html#False" title="False"><code class="xref py py-const docutils literal">False</code></a>. (<a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a> and

560

<a class="reference internal" href="../library/constants.html#False" title="False"><code class="xref py py-const docutils literal">False</code></a> constants were added to the built-ins in Python 2.2.1, but the

561

2.2.1 versions are simply set to integer values of 1 and 0 and aren’t a

562

different type.)

563

The type object for this new type is named <a class="reference internal" href="../library/functions.html#bool" title="bool"><code class="xref py py-class docutils literal">bool</code></a>; the constructor for it

564

takes any Python value and converts it to <a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a> or <a class="reference internal" href="../library/constants.html#False" title="False"><code class="xref py py-const docutils literal">False</code></a>.

565

<div class="highlight-python"><div class="highlight"><pre>>>> bool(1)

566

True

567

>>> bool(0)

568

False

569

>>> bool([])

570

False

571

>>> bool( (1,) )

572

True

573

</pre></div>

574

</div>

575

Most of the standard library modules and built-in functions have been changed to

576

return Booleans.

577

<div class="highlight-python"><div class="highlight"><pre>>>> obj = []

578

>>> hasattr(obj, 'append')

579

True

580

>>> isinstance(obj, list)

581

True

582

>>> isinstance(obj, tuple)

583

False

584

</pre></div>

585

</div>

586

Python’s Booleans were added with the primary goal of making code clearer. For

587

example, if you’re reading a function and encounter the statement <code class="docutils literal">return 1</code>,

588

you might wonder whether the <code class="docutils literal">1</code> represents a Boolean truth value, an index,

589

or a coefficient that multiplies some other quantity. If the statement is

590

<code class="docutils literal">return True</code>, however, the meaning of the return value is quite clear.

591

Python’s Booleans were not added for the sake of strict type-checking. A very

592

strict language such as Pascal would also prevent you performing arithmetic with

593

Booleans, and would require that the expression in an <a class="reference internal" href="../reference/compound_stmts.html#if"><code class="xref std std-keyword docutils literal">if</code></a> statement

594

always evaluate to a Boolean result. Python is not this strict and never will

595

be, as <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0285">PEP 285</a> explicitly says. This means you can still use any expression

596

in an <a class="reference internal" href="../reference/compound_stmts.html#if"><code class="xref std std-keyword docutils literal">if</code></a> statement, even ones that evaluate to a list or tuple or

597

some random object. The Boolean type is a subclass of the <a class="reference internal" href="../library/functions.html#int" title="int"><code class="xref py py-class docutils literal">int</code></a> class so

598

that arithmetic using a Boolean still works.

599

<div class="highlight-python"><div class="highlight"><pre>>>> True + 1

600

2

601

>>> False + 1

602

1

603

>>> False * 75

604

0

605

>>> True * 75

606

75

607

</pre></div>

608

</div>

609

To sum up <a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a> and <a class="reference internal" href="../library/constants.html#False" title="False"><code class="xref py py-const docutils literal">False</code></a> in a sentence: they’re alternative

610

ways to spell the integer values 1 and 0, with the single difference that

611

<a class="reference internal" href="../library/functions.html#str" title="str"><code class="xref py py-func docutils literal">str()</code></a> and <a class="reference internal" href="../library/functions.html#repr" title="repr"><code class="xref py py-func docutils literal">repr()</code></a> return the strings <code class="docutils literal">'True'</code> and <code class="docutils literal">'False'</code>

612

instead of <code class="docutils literal">'1'</code> and <code class="docutils literal">'0'</code>.

613

614

See also

615

616

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0285">PEP 285</a> - Adding a bool type</dt>

617

<dd>Written and implemented by GvR.</dd>

618

</dl>

619

</div>

620

</div>

621

622

<h2>PEP 293: Codec Error Handling Callbacks<a class="headerlink" href="#pep-293-codec-error-handling-callbacks" title="Permalink to this headline">¶</a></h2>

623

When encoding a Unicode string into a byte string, unencodable characters may be

624

encountered. So far, Python has allowed specifying the error processing as

625

either “strict” (raising <a class="reference internal" href="../library/exceptions.html#exceptions.UnicodeError" title="exceptions.UnicodeError"><code class="xref py py-exc docutils literal">UnicodeError</code></a>), “ignore” (skipping the

626

character), or “replace” (using a question mark in the output string), with

627

“strict” being the default behavior. It may be desirable to specify alternative

628

processing of such errors, such as inserting an XML character reference or HTML

629

entity reference into the converted string.

630

Python now has a flexible framework to add different processing strategies. New

631

error handlers can be added with <a class="reference internal" href="../library/codecs.html#codecs.register_error" title="codecs.register_error"><code class="xref py py-func docutils literal">codecs.register_error()</code></a>, and codecs then

632

can access the error handler with <a class="reference internal" href="../library/codecs.html#codecs.lookup_error" title="codecs.lookup_error"><code class="xref py py-func docutils literal">codecs.lookup_error()</code></a>. An equivalent C

633

API has been added for codecs written in C. The error handler gets the necessary

634

state information such as the string being converted, the position in the string

635

where the error was detected, and the target encoding. The handler can then

636

either raise an exception or return a replacement string.

637

Two additional error handlers have been implemented using this framework:

638

“backslashreplace” uses Python backslash quoting to represent unencodable

639

characters and “xmlcharrefreplace” emits XML character references.

640

641

See also

642

643

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0293">PEP 293</a> - Codec Error Handling Callbacks</dt>

644

<dd>Written and implemented by Walter Dörwald.</dd>

645

</dl>

646

</div>

647

</div>

648

649

<h2>PEP 301: Package Index and Metadata for Distutils<a class="headerlink" href="#pep-301-package-index-and-metadata-for-distutils" title="Permalink to this headline">¶</a></h2>

650

Support for the long-requested Python catalog makes its first appearance in 2.3.

651

The heart of the catalog is the new Distutils register command.

652

Running <code class="docutils literal">python setup.py register</code> will collect the metadata describing a

653

package, such as its name, version, maintainer, description, &c., and send it to

654

a central catalog server. The resulting catalog is available from

655

<a class="reference external" href="https://pypi.python.org/pypi">https://pypi.python.org/pypi</a>.

656

To make the catalog a bit more useful, a new optional classifiers keyword

657

argument has been added to the Distutils <code class="xref py py-func docutils literal">setup()</code> function. A list of

658

<a class="reference external" href="http://catb.org/~esr/trove/">Trove</a>-style strings can be supplied to help

659

classify the software.

660

Here’s an example <code class="file docutils literal">setup.py</code> with classifiers, written to be compatible

661

with older versions of the Distutils:

662

<div class="highlight-python"><div class="highlight"><pre>from distutils import core

663

kw = {'name': "Quixote",

664

'version': "0.5.1",

665

'description': "A highly Pythonic Web application framework",

666

# ...

667

}

668

669

if (hasattr(core, 'setup_keywords') and

670

'classifiers' in core.setup_keywords):

671

kw['classifiers'] = \

672

['Topic :: Internet :: WWW/HTTP :: Dynamic Content',

673

'Environment :: No Input/Output (Daemon)',

674

'Intended Audience :: Developers'],

675

676

core.setup(**kw)

677

</pre></div>

678

</div>

679

The full list of classifiers can be obtained by running <code class="docutils literal">python setup.py

680

register --list-classifiers</code>.

681

682

See also

683

684

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0301">PEP 301</a> - Package Index and Metadata for Distutils</dt>

685

<dd>Written and implemented by Richard Jones.</dd>

686

</dl>

687

</div>

688

</div>

689

690

<h2>PEP 302: New Import Hooks<a class="headerlink" href="#pep-302-new-import-hooks" title="Permalink to this headline">¶</a></h2>

691

While it’s been possible to write custom import hooks ever since the

692

<code class="xref py py-mod docutils literal">ihooks</code> module was introduced in Python 1.3, no one has ever been really

693

happy with it because writing new import hooks is difficult and messy. There

694

have been various proposed alternatives such as the <a class="reference internal" href="../library/imputil.html#module-imputil" title="imputil: Manage and augment the import process. (deprecated)"><code class="xref py py-mod docutils literal">imputil</code></a> and <code class="xref py py-mod docutils literal">iu</code>

695

modules, but none of them has ever gained much acceptance, and none of them were

696

easily usable from C code.

697

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0302">PEP 302</a> borrows ideas from its predecessors, especially from Gordon

698

McMillan’s <code class="xref py py-mod docutils literal">iu</code> module. Three new items are added to the <a class="reference internal" href="../library/sys.html#module-sys" title="sys: Access system-specific parameters and functions."><code class="xref py py-mod docutils literal">sys</code></a>

699

module:

700

701

<li><code class="docutils literal">sys.path_hooks</code> is a list of callable objects; most often they’ll be

702

classes. Each callable takes a string containing a path and either returns an

703

importer object that will handle imports from this path or raises an

704

<a class="reference internal" href="../library/exceptions.html#exceptions.ImportError" title="exceptions.ImportError"><code class="xref py py-exc docutils literal">ImportError</code></a> exception if it can’t handle this path.</li>

705

<li><code class="docutils literal">sys.path_importer_cache</code> caches importer objects for each path, so

706

<code class="docutils literal">sys.path_hooks</code> will only need to be traversed once for each path.</li>

707

<li><code class="docutils literal">sys.meta_path</code> is a list of importer objects that will be traversed before

708

<code class="docutils literal">sys.path</code> is checked. This list is initially empty, but user code can add

709

objects to it. Additional built-in and frozen modules can be imported by an

710

object added to this list.</li>

711

</ul>

712

Importer objects must have a single method, <code class="xref py py-meth docutils literal">find_module(fullname,

713

path=None)()</code>. fullname will be a module or package name, e.g. <code class="docutils literal">string</code> or

714

<code class="docutils literal">distutils.core</code>. <code class="xref py py-meth docutils literal">find_module()</code> must return a loader object that has a

715

single method, <code class="xref py py-meth docutils literal">load_module(fullname)()</code>, that creates and returns the

716

corresponding module object.

717

Pseudo-code for Python’s new import logic, therefore, looks something like this

718

(simplified a bit; see <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0302">PEP 302</a> for the full details):

719

<div class="highlight-python"><div class="highlight"><pre>for mp in sys.meta_path:

720

loader = mp(fullname)

721

if loader is not None:

722

<module> = loader.load_module(fullname)

723

724

for path in sys.path:

725

for hook in sys.path_hooks:

726

try:

727

importer = hook(path)

728

except ImportError:

729

# ImportError, so try the other path hooks

730

pass

731

else:

732

loader = importer.find_module(fullname)

733

<module> = loader.load_module(fullname)

734

735

# Not found!

736

raise ImportError

737

</pre></div>

738

</div>

739

740

See also

741

742

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0302">PEP 302</a> - New Import Hooks</dt>

743

<dd>Written by Just van Rossum and Paul Moore. Implemented by Just van Rossum.</dd>

744

</dl>

745

</div>

746

</div>

747

748

<h2>PEP 305: Comma-separated Files<a class="headerlink" href="#pep-305-comma-separated-files" title="Permalink to this headline">¶</a></h2>

749

Comma-separated files are a format frequently used for exporting data from

750

databases and spreadsheets. Python 2.3 adds a parser for comma-separated files.

751

Comma-separated format is deceptively simple at first glance:

752

<div class="highlight-python"><div class="highlight"><pre>Costs,150,200,3.95

753

</pre></div>

754

</div>

755

Read a line and call <code class="docutils literal">line.split(',')</code>: what could be simpler? But toss in

756

string data that can contain commas, and things get more complicated:

757

<div class="highlight-python"><div class="highlight"><pre>"Costs",150,200,3.95,"Includes taxes, shipping, and sundry items"

758

</pre></div>

759

</div>

760

A big ugly regular expression can parse this, but using the new <a class="reference internal" href="../library/csv.html#module-csv" title="csv: Write and read tabular data to and from delimited files."><code class="xref py py-mod docutils literal">csv</code></a>

761

package is much simpler:

762

<div class="highlight-python"><div class="highlight"><pre>import csv

763

764

input = open('datafile', 'rb')

765

reader = csv.reader(input)

766

for line in reader:

767

print line

768

</pre></div>

769

</div>

770

The <code class="xref py py-func docutils literal">reader()</code> function takes a number of different options. The field

771

separator isn’t limited to the comma and can be changed to any character, and so

772

can the quoting and line-ending characters.

773

Different dialects of comma-separated files can be defined and registered;

774

currently there are two dialects, both used by Microsoft Excel. A separate

775

<a class="reference internal" href="../library/csv.html#csv.writer" title="csv.writer"><code class="xref py py-class docutils literal">csv.writer</code></a> class will generate comma-separated files from a succession

776

of tuples or lists, quoting strings that contain the delimiter.

777

778

See also

779

780

781

<dd>Written and implemented by Kevin Altis, Dave Cole, Andrew McNamara, Skip

782

Montanaro, Cliff Wells.</dd>

783

</dl>

784

</div>

785

</div>

786

787

<h2>PEP 307: Pickle Enhancements<a class="headerlink" href="#pep-307-pickle-enhancements" title="Permalink to this headline">¶</a></h2>

788

The <a class="reference internal" href="../library/pickle.html#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal">pickle</code></a> and <a class="reference internal" href="../library/pickle.html#module-cPickle" title="cPickle: Faster version of pickle, but not subclassable."><code class="xref py py-mod docutils literal">cPickle</code></a> modules received some attention during the

789

2.3 development cycle. In 2.2, new-style classes could be pickled without

790

difficulty, but they weren’t pickled very compactly; <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0307">PEP 307</a> quotes a trivial

791

example where a new-style class results in a pickled string three times longer

792

than that for a classic class.

793

The solution was to invent a new pickle protocol. The <a class="reference internal" href="../library/pickle.html#pickle.dumps" title="pickle.dumps"><code class="xref py py-func docutils literal">pickle.dumps()</code></a>

794

function has supported a text-or-binary flag for a long time. In 2.3, this

795

flag is redefined from a Boolean to an integer: 0 is the old text-mode pickle

796

format, 1 is the old binary format, and now 2 is a new 2.3-specific format. A

797

new constant, <a class="reference internal" href="../library/pickle.html#pickle.HIGHEST_PROTOCOL" title="pickle.HIGHEST_PROTOCOL"><code class="xref py py-const docutils literal">pickle.HIGHEST_PROTOCOL</code></a>, can be used to select the

798

fanciest protocol available.

799

Unpickling is no longer considered a safe operation. 2.2’s <a class="reference internal" href="../library/pickle.html#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal">pickle</code></a>

800

provided hooks for trying to prevent unsafe classes from being unpickled

801

(specifically, a <code class="xref py py-attr docutils literal">__safe_for_unpickling__</code> attribute), but none of this

802

code was ever audited and therefore it’s all been ripped out in 2.3. You should

803

not unpickle untrusted data in any version of Python.

804

To reduce the pickling overhead for new-style classes, a new interface for

805

customizing pickling was added using three special methods:

806

<a class="reference internal" href="../library/pickle.html#object.__getstate__" title="object.__getstate__"><code class="xref py py-meth docutils literal">__getstate__()</code></a>, <a class="reference internal" href="../library/pickle.html#object.__setstate__" title="object.__setstate__"><code class="xref py py-meth docutils literal">__setstate__()</code></a>, and <a class="reference internal" href="../library/pickle.html#object.__getnewargs__" title="object.__getnewargs__"><code class="xref py py-meth docutils literal">__getnewargs__()</code></a>. Consult

807

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0307">PEP 307</a> for the full semantics of these methods.

808

As a way to compress pickles yet further, it’s now possible to use integer codes

809

instead of long strings to identify pickled classes. The Python Software

810

Foundation will maintain a list of standardized codes; there’s also a range of

811

codes for private use. Currently no codes have been specified.

812

813

See also

814

815

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0307">PEP 307</a> - Extensions to the pickle protocol</dt>

816

<dd>Written and implemented by Guido van Rossum and Tim Peters.</dd>

817

</dl>

818

</div>

819

</div>

820

821

<h2>Extended Slices<a class="headerlink" href="#extended-slices" title="Permalink to this headline">¶</a></h2>

822

Ever since Python 1.4, the slicing syntax has supported an optional third “step”

823

or “stride” argument. For example, these are all legal Python syntax:

824

<code class="docutils literal">L[1:10:2]</code>, <code class="docutils literal">L[:-1:1]</code>, <code class="docutils literal">L[::-1]</code>. This was added to Python at the

825

request of the developers of Numerical Python, which uses the third argument

826

extensively. However, Python’s built-in list, tuple, and string sequence types

827

have never supported this feature, raising a <a class="reference internal" href="../library/exceptions.html#exceptions.TypeError" title="exceptions.TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a> if you tried it.

828

Michael Hudson contributed a patch to fix this shortcoming.

829

For example, you can now easily extract the elements of a list that have even

830

indexes:

831

<div class="highlight-python"><div class="highlight"><pre>>>> L = range(10)

832

>>> L[::2]

833

[0, 2, 4, 6, 8]

834

</pre></div>

835

</div>

836

Negative values also work to make a copy of the same list in reverse order:

837

<div class="highlight-python"><div class="highlight"><pre>>>> L[::-1]

838

[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

839

</pre></div>

840

</div>

841

This also works for tuples, arrays, and strings:

842

<div class="highlight-python"><div class="highlight"><pre>>>> s='abcd'

843

>>> s[::2]

844

'ac'

845

>>> s[::-1]

846

'dcba'

847

</pre></div>

848

</div>

849

If you have a mutable sequence such as a list or an array you can assign to or

850

delete an extended slice, but there are some differences between assignment to

851

extended and regular slices. Assignment to a regular slice can be used to

852

change the length of the sequence:

853

<div class="highlight-python"><div class="highlight"><pre>>>> a = range(3)

854

>>> a

855

[0, 1, 2]

856

>>> a[1:3] = [4, 5, 6]

857

>>> a

858

[0, 4, 5, 6]

859

</pre></div>

860

</div>

861

Extended slices aren’t this flexible. When assigning to an extended slice, the

862

list on the right hand side of the statement must contain the same number of

863

items as the slice it is replacing:

864

865

>>> a

866

[0, 1, 2, 3]

867

>>> a[::2]

868

[0, 2]

869

>>> a[::2] = [0, -1]

870

>>> a

871

[0, 1, -1, 3]

872

>>> a[::2] = [0,1,2]

873

Traceback (most recent call last):

874

File "<stdin>", line 1, in ?

875

ValueError: attempt to assign sequence of size 3 to extended slice of size 2

876

</pre></div>

877

</div>

878

Deletion is more straightforward:

879

880

>>> a

881

[0, 1, 2, 3]

882

>>> a[::2]

883

[0, 2]

884

>>> del a[::2]

885

>>> a

886

[1, 3]

887

</pre></div>

888

</div>

889

One can also now pass slice objects to the <a class="reference internal" href="../reference/datamodel.html#object.__getitem__" title="object.__getitem__"><code class="xref py py-meth docutils literal">__getitem__()</code></a> methods of the

890

built-in sequences:

891

<div class="highlight-python"><div class="highlight"><pre>>>> range(10).__getitem__(slice(0, 5, 2))

892

[0, 2, 4]

893

</pre></div>

894

</div>

895

Or use slice objects directly in subscripts:

896

<div class="highlight-python"><div class="highlight"><pre>>>> range(10)[slice(0, 5, 2)]

897

[0, 2, 4]

898

</pre></div>

899

</div>

900

To simplify implementing sequences that support extended slicing, slice objects

901

now have a method <code class="xref py py-meth docutils literal">indices(length)()</code> which, given the length of a sequence,

902

returns a <code class="docutils literal">(start, stop, step)</code> tuple that can be passed directly to

903

<a class="reference internal" href="../library/functions.html#range" title="range"><code class="xref py py-func docutils literal">range()</code></a>. <code class="xref py py-meth docutils literal">indices()</code> handles omitted and out-of-bounds indices in a

904

manner consistent with regular slices (and this innocuous phrase hides a welter

905

of confusing details!). The method is intended to be used like this:

906

<div class="highlight-python"><div class="highlight"><pre>class FakeSeq:

907

...

908

def calc_item(self, i):

909

...

910

def __getitem__(self, item):

911

if isinstance(item, slice):

912

indices = item.indices(len(self))

913

return FakeSeq([self.calc_item(i) for i in range(*indices)])

914

else:

915

return self.calc_item(i)

916

</pre></div>

917

</div>

918

From this example you can also see that the built-in <a class="reference internal" href="../library/functions.html#slice" title="slice"><code class="xref py py-class docutils literal">slice</code></a> object is

919

now the type object for the slice type, and is no longer a function. This is

920

consistent with Python 2.2, where <a class="reference internal" href="../library/functions.html#int" title="int"><code class="xref py py-class docutils literal">int</code></a>, <a class="reference internal" href="../library/functions.html#str" title="str"><code class="xref py py-class docutils literal">str</code></a>, etc., underwent

921

the same change.

922

</div>

923

924

<h2>Other Language Changes<a class="headerlink" href="#other-language-changes" title="Permalink to this headline">¶</a></h2>

925

Here are all of the changes that Python 2.3 makes to the core Python language.

926

<ul>

927

<li>The <a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a> statement is now always a keyword, as described in

928

section <a class="reference internal" href="#section-generators">PEP 255: Simple Generators</a> of this document.

929

</li>

930

<li>A new built-in function <a class="reference internal" href="../library/functions.html#enumerate" title="enumerate"><code class="xref py py-func docutils literal">enumerate()</code></a> was added, as described in section

931

<a class="reference internal" href="#section-enumerate">PEP 279: enumerate()</a> of this document.

932

</li>

933

<li>Two new constants, <a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a> and <a class="reference internal" href="../library/constants.html#False" title="False"><code class="xref py py-const docutils literal">False</code></a> were added along with the

934

built-in <a class="reference internal" href="../library/functions.html#bool" title="bool"><code class="xref py py-class docutils literal">bool</code></a> type, as described in section <a class="reference internal" href="#section-bool">PEP 285: A Boolean Type</a> of this

935

document.

936

</li>

937

<li>The <a class="reference internal" href="../library/functions.html#int" title="int"><code class="xref py py-func docutils literal">int()</code></a> type constructor will now return a long integer instead of

938

raising an <a class="reference internal" href="../library/exceptions.html#exceptions.OverflowError" title="exceptions.OverflowError"><code class="xref py py-exc docutils literal">OverflowError</code></a> when a string or floating-point number is too

939

large to fit into an integer. This can lead to the paradoxical result that

940

<code class="docutils literal">isinstance(int(expression), int)</code> is false, but that seems unlikely to cause

941

problems in practice.

942

</li>

943

<li>Built-in types now support the extended slicing syntax, as described in

944

section <a class="reference internal" href="#section-slices">Extended Slices</a> of this document.

945

</li>

946

<li>A new built-in function, <code class="xref py py-func docutils literal">sum(iterable, start=0)()</code>, adds up the numeric

947

items in the iterable object and returns their sum. <a class="reference internal" href="../library/functions.html#sum" title="sum"><code class="xref py py-func docutils literal">sum()</code></a> only accepts

948

numbers, meaning that you can’t use it to concatenate a bunch of strings.

949

(Contributed by Alex Martelli.)

950

</li>

951

<li><code class="docutils literal">list.insert(pos, value)</code> used to insert value at the front of the list

952

when pos was negative. The behaviour has now been changed to be consistent

953

with slice indexing, so when pos is -1 the value will be inserted before the

954

last element, and so forth.

955

</li>

956

<li><code class="docutils literal">list.index(value)</code>, which searches for value within the list and returns

957

its index, now takes optional start and stop arguments to limit the search

958

to only part of the list.

959

</li>

960

<li>Dictionaries have a new method, <code class="xref py py-meth docutils literal">pop(key[, *default*])()</code>, that returns

961

the value corresponding to key and removes that key/value pair from the

962

dictionary. If the requested key isn’t present in the dictionary, default is

963

returned if it’s specified and <a class="reference internal" href="../library/exceptions.html#exceptions.KeyError" title="exceptions.KeyError"><code class="xref py py-exc docutils literal">KeyError</code></a> raised if it isn’t.

964

<div class="highlight-python"><div class="highlight"><pre>>>> d = {1:2}

965

>>> d

966

{1: 2}

967

>>> d.pop(4)

968

Traceback (most recent call last):

969

File "stdin", line 1, in ?

970

KeyError: 4

971

>>> d.pop(1)

972

2

973

>>> d.pop(1)

974

Traceback (most recent call last):

975

File "stdin", line 1, in ?

976

KeyError: 'pop(): dictionary is empty'

977

>>> d

978

{}

979

>>>

980

</pre></div>

981

</div>

982

There’s also a new class method, <code class="xref py py-meth docutils literal">dict.fromkeys(iterable, value)()</code>, that

983

creates a dictionary with keys taken from the supplied iterator iterable and

984

all values set to value, defaulting to <code class="docutils literal">None</code>.

985

(Patches contributed by Raymond Hettinger.)

986

Also, the <a class="reference internal" href="../library/stdtypes.html#dict" title="dict"><code class="xref py py-func docutils literal">dict()</code></a> constructor now accepts keyword arguments to simplify

987

creating small dictionaries:

988

<div class="highlight-python"><div class="highlight"><pre>>>> dict(red=1, blue=2, green=3, black=4)

989

{'blue': 2, 'black': 4, 'green': 3, 'red': 1}

990

</pre></div>

991

</div>

992

(Contributed by Just van Rossum.)

993

</li>

994

<li>The <a class="reference internal" href="../reference/simple_stmts.html#assert"><code class="xref std std-keyword docutils literal">assert</code></a> statement no longer checks the <code class="docutils literal">__debug__</code> flag, so

995

you can no longer disable assertions by assigning to <code class="docutils literal">__debug__</code>. Running

996

Python with the <a class="reference internal" href="../using/cmdline.html#cmdoption-O"><code class="xref std std-option docutils literal">-O</code></a> switch will still generate code that doesn’t

997

execute any assertions.

998

</li>

999

<li>Most type objects are now callable, so you can use them to create new objects

1000

such as functions, classes, and modules. (This means that the <a class="reference internal" href="../library/new.html#module-new" title="new: Interface to the creation of runtime implementation objects. (deprecated)"><code class="xref py py-mod docutils literal">new</code></a> module

1001

can be deprecated in a future Python version, because you can now use the type

1002

objects available in the <a class="reference internal" href="../library/types.html#module-types" title="types: Names for built-in types."><code class="xref py py-mod docutils literal">types</code></a> module.) For example, you can create a new

1003

module object with the following code:

1004

<div class="highlight-python"><div class="highlight"><pre>>>> import types

1005

>>> m = types.ModuleType('abc','docstring')

1006

>>> m

1007

1008

>>> m.__doc__

1009

'docstring'

1010

</pre></div>

1011

</div>

1012

</li>

1013

<li>A new warning, <a class="reference internal" href="../library/exceptions.html#exceptions.PendingDeprecationWarning" title="exceptions.PendingDeprecationWarning"><code class="xref py py-exc docutils literal">PendingDeprecationWarning</code></a> was added to indicate features

1014

which are in the process of being deprecated. The warning will not be printed

1015

by default. To check for use of features that will be deprecated in the future,

1016

supply <code class="xref std std-option docutils literal">-Walways::PendingDeprecationWarning::</code> on the command line or

1017

use <a class="reference internal" href="../library/warnings.html#warnings.filterwarnings" title="warnings.filterwarnings"><code class="xref py py-func docutils literal">warnings.filterwarnings()</code></a>.

1018

</li>

1019

<li>The process of deprecating string-based exceptions, as in <code class="docutils literal">raise "Error

1020

occurred"</code>, has begun. Raising a string will now trigger

1021

<a class="reference internal" href="../library/exceptions.html#exceptions.PendingDeprecationWarning" title="exceptions.PendingDeprecationWarning"><code class="xref py py-exc docutils literal">PendingDeprecationWarning</code></a>.

1022

</li>

1023

<li>Using <code class="docutils literal">None</code> as a variable name will now result in a <a class="reference internal" href="../library/exceptions.html#exceptions.SyntaxWarning" title="exceptions.SyntaxWarning"><code class="xref py py-exc docutils literal">SyntaxWarning</code></a>

1024

warning. In a future version of Python, <code class="docutils literal">None</code> may finally become a keyword.

1025

</li>

1026

<li>The <code class="xref py py-meth docutils literal">xreadlines()</code> method of file objects, introduced in Python 2.1, is no

1027

longer necessary because files now behave as their own iterator.

1028

<code class="xref py py-meth docutils literal">xreadlines()</code> was originally introduced as a faster way to loop over all

1029

the lines in a file, but now you can simply write <code class="docutils literal">for line in file_obj</code>.

1030

File objects also have a new read-only <code class="xref py py-attr docutils literal">encoding</code> attribute that gives the

1031

encoding used by the file; Unicode strings written to the file will be

1032

automatically converted to bytes using the given encoding.

1033

</li>

1034

<li>The method resolution order used by new-style classes has changed, though

1035

you’ll only notice the difference if you have a really complicated inheritance

1036

hierarchy. Classic classes are unaffected by this change. Python 2.2

1037

originally used a topological sort of a class’s ancestors, but 2.3 now uses the

1038

C3 algorithm as described in the paper <a class="reference external" href="http://www.webcom.com/haahr/dylan/linearization-oopsla96.html">“A Monotonic Superclass Linearization

1039

for Dylan”</a>. To

1040

understand the motivation for this change, read Michele Simionato’s article

1041

<a class="reference external" href="https://www.python.org/2.3/mro.html">“Python 2.3 Method Resolution Order”</a>, or

1042

read the thread on python-dev starting with the message at

1043

<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2002-October/029035.html">https://mail.python.org/pipermail/python-dev/2002-October/029035.html</a>. Samuele

1044

Pedroni first pointed out the problem and also implemented the fix by coding the

1045

C3 algorithm.

1046

</li>

1047

<li>Python runs multithreaded programs by switching between threads after

1048

executing N bytecodes. The default value for N has been increased from 10 to

1049

100 bytecodes, speeding up single-threaded applications by reducing the

1050

switching overhead. Some multithreaded applications may suffer slower response

1051

time, but that’s easily fixed by setting the limit back to a lower number using

1052

<code class="xref py py-func docutils literal">sys.setcheckinterval(N)()</code>. The limit can be retrieved with the new

1053

<a class="reference internal" href="../library/sys.html#sys.getcheckinterval" title="sys.getcheckinterval"><code class="xref py py-func docutils literal">sys.getcheckinterval()</code></a> function.

1054

</li>

1055

<li>One minor but far-reaching change is that the names of extension types defined

1056

by the modules included with Python now contain the module and a <code class="docutils literal">'.'</code> in

1057

front of the type name. For example, in Python 2.2, if you created a socket and

1058

printed its <code class="xref py py-attr docutils literal">__class__</code>, you’d get this output:

1059

<div class="highlight-python"><div class="highlight"><pre>>>> s = socket.socket()

1060

>>> s.__class__

1061

1062

</pre></div>

1063

</div>

1064

In 2.3, you get this:

1065

<div class="highlight-python"><div class="highlight"><pre>>>> s.__class__

1066

1067

</pre></div>

1068

</div>

1069

</li>

1070

<li>One of the noted incompatibilities between old- and new-style classes has been

1071

removed: you can now assign to the <code class="xref py py-attr docutils literal">__name__</code> and <code class="xref py py-attr docutils literal">__bases__</code>

1072

attributes of new-style classes. There are some restrictions on what can be

1073

assigned to <code class="xref py py-attr docutils literal">__bases__</code> along the lines of those relating to assigning to

1074

an instance’s <code class="xref py py-attr docutils literal">__class__</code> attribute.

1075

</li>

1076

</ul>

1077

1078

<h3>String Changes<a class="headerlink" href="#string-changes" title="Permalink to this headline">¶</a></h3>

1079

<ul>

1080

<li>The <a class="reference internal" href="../reference/expressions.html#in"><code class="xref std std-keyword docutils literal">in</code></a> operator now works differently for strings. Previously, when

1081

evaluating <code class="docutils literal">X in Y</code> where X and Y are strings, X could only be a single

1082

character. That’s now changed; X can be a string of any length, and <code class="docutils literal">X in Y</code>

1083

will return <a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a> if X is a substring of Y. If X is the empty

1084

string, the result is always <a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a>.

1085

<div class="highlight-python"><div class="highlight"><pre>>>> 'ab' in 'abcd'

1086

True

1087

>>> 'ad' in 'abcd'

1088

False

1089

>>> '' in 'abcd'

1090

True

1091

</pre></div>

1092

</div>

1093

Note that this doesn’t tell you where the substring starts; if you need that

1094

information, use the <code class="xref py py-meth docutils literal">find()</code> string method.

1095

</li>

1096

<li>The <code class="xref py py-meth docutils literal">strip()</code>, <code class="xref py py-meth docutils literal">lstrip()</code>, and <code class="xref py py-meth docutils literal">rstrip()</code> string methods now have

1097

an optional argument for specifying the characters to strip. The default is

1098

still to remove all whitespace characters:

1099

<div class="highlight-python"><div class="highlight"><pre>>>> ' abc '.strip()

1100

'abc'

1101

>>> '><><abc<><><>'.strip('<>')

1102

'abc'

1103

>>> '><><abc<><><>\n'.strip('<>')

1104

'abc<><><>\n'

1105

>>> u'\u4000\u4001abc\u4000'.strip(u'\u4000')

1106

u'\u4001abc'

1107

>>>

1108

</pre></div>

1109

</div>

1110

(Suggested by Simon Brunning and implemented by Walter Dörwald.)

1111

</li>

1112

<li>The <code class="xref py py-meth docutils literal">startswith()</code> and <code class="xref py py-meth docutils literal">endswith()</code> string methods now accept negative

1113

numbers for the start and end parameters.

1114

</li>

1115

<li>Another new string method is <code class="xref py py-meth docutils literal">zfill()</code>, originally a function in the

1116

<a class="reference internal" href="../library/string.html#module-string" title="string: Common string operations."><code class="xref py py-mod docutils literal">string</code></a> module. <code class="xref py py-meth docutils literal">zfill()</code> pads a numeric string with zeros on the

1117

left until it’s the specified width. Note that the <code class="docutils literal">%</code> operator is still more

1118

flexible and powerful than <code class="xref py py-meth docutils literal">zfill()</code>.

1119

<div class="highlight-python"><div class="highlight"><pre>>>> '45'.zfill(4)

1120

'0045'

1121

>>> '12345'.zfill(4)

1122

'12345'

1123

>>> 'goofy'.zfill(6)

1124

'0goofy'

1125

</pre></div>

1126

</div>

1127

(Contributed by Walter Dörwald.)

1128

</li>

1129

<li>A new type object, <a class="reference internal" href="../library/functions.html#basestring" title="basestring"><code class="xref py py-class docutils literal">basestring</code></a>, has been added. Both 8-bit strings and

1130

Unicode strings inherit from this type, so <code class="docutils literal">isinstance(obj, basestring)</code> will

1131

return <a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a> for either kind of string. It’s a completely abstract

1132

type, so you can’t create <a class="reference internal" href="../library/functions.html#basestring" title="basestring"><code class="xref py py-class docutils literal">basestring</code></a> instances.

1133

</li>

1134

<li>Interned strings are no longer immortal and will now be garbage-collected in

1135

the usual way when the only reference to them is from the internal dictionary of

1136

interned strings. (Implemented by Oren Tirosh.)

1137

</li>

1138

</ul>

1139

</div>

1140

1141

<h3>Optimizations<a class="headerlink" href="#optimizations" title="Permalink to this headline">¶</a></h3>

1142

1143

<li>The creation of new-style class instances has been made much faster; they’re

1144

now faster than classic classes!</li>

1145

<li>The <code class="xref py py-meth docutils literal">sort()</code> method of list objects has been extensively rewritten by Tim

1146

Peters, and the implementation is significantly faster.</li>

1147

<li>Multiplication of large long integers is now much faster thanks to an

1148

implementation of Karatsuba multiplication, an algorithm that scales better than

1149

the O(n*n) required for the grade-school multiplication algorithm. (Original

1150

patch by Christopher A. Craig, and significantly reworked by Tim Peters.)</li>

1151

<li>The <code class="docutils literal">SET_LINENO</code> opcode is now gone. This may provide a small speed

1152

increase, depending on your compiler’s idiosyncrasies. See section

1153

<a class="reference internal" href="#section-other">Other Changes and Fixes</a> for a longer explanation. (Removed by Michael Hudson.)</li>

1154

<li><a class="reference internal" href="../library/functions.html#xrange" title="xrange"><code class="xref py py-func docutils literal">xrange()</code></a> objects now have their own iterator, making <code class="docutils literal">for i in

1155

xrange(n)</code> slightly faster than <code class="docutils literal">for i in range(n)</code>. (Patch by Raymond

1156

Hettinger.)</li>

1157

<li>A number of small rearrangements have been made in various hotspots to improve

1158

performance, such as inlining a function or removing some code. (Implemented

1159

mostly by GvR, but lots of people have contributed single changes.)</li>

1160

</ul>

1161

The net result of the 2.3 optimizations is that Python 2.3 runs the pystone

1162

benchmark around 25% faster than Python 2.2.

1163

</div>

1164

</div>

1165

1166

<h2>New, Improved, and Deprecated Modules<a class="headerlink" href="#new-improved-and-deprecated-modules" title="Permalink to this headline">¶</a></h2>

1167

As usual, Python’s standard library received a number of enhancements and bug

1168

fixes. Here’s a partial list of the most notable changes, sorted alphabetically

1169

by module name. Consult the <code class="file docutils literal">Misc/NEWS</code> file in the source tree for a more

1170

complete list of changes, or look through the CVS logs for all the details.

1171

<ul>

1172

<li>The <a class="reference internal" href="../library/array.html#module-array" title="array: Space efficient arrays of uniformly typed numeric values."><code class="xref py py-mod docutils literal">array</code></a> module now supports arrays of Unicode characters using the

1173

<code class="docutils literal">'u'</code> format character. Arrays also now support using the <code class="docutils literal">+=</code> assignment

1174

operator to add another array’s contents, and the <code class="docutils literal">*=</code> assignment operator to

1175

repeat an array. (Contributed by Jason Orendorff.)

1176

</li>

1177

<li>The <a class="reference internal" href="../library/bsddb.html#module-bsddb" title="bsddb: Interface to Berkeley DB database library"><code class="xref py py-mod docutils literal">bsddb</code></a> module has been replaced by version 4.1.6 of the <a class="reference external" href="http://pybsddb.sourceforge.net">PyBSDDB</a> package, providing a more complete interface

1178

to the transactional features of the BerkeleyDB library.

1179

The old version of the module has been renamed to <code class="xref py py-mod docutils literal">bsddb185</code> and is no

1180

longer built automatically; you’ll have to edit <code class="file docutils literal">Modules/Setup</code> to enable

1181

it. Note that the new <a class="reference internal" href="../library/bsddb.html#module-bsddb" title="bsddb: Interface to Berkeley DB database library"><code class="xref py py-mod docutils literal">bsddb</code></a> package is intended to be compatible with

1182

the old module, so be sure to file bugs if you discover any incompatibilities.

1183

When upgrading to Python 2.3, if the new interpreter is compiled with a new

1184

version of the underlying BerkeleyDB library, you will almost certainly have to

1185

convert your database files to the new version. You can do this fairly easily

1186

with the new scripts <code class="file docutils literal">db2pickle.py</code> and <code class="file docutils literal">pickle2db.py</code> which you

1187

will find in the distribution’s <code class="file docutils literal">Tools/scripts</code> directory. If you’ve

1188

already been using the PyBSDDB package and importing it as <code class="xref py py-mod docutils literal">bsddb3</code>, you

1189

will have to change your <code class="docutils literal">import</code> statements to import it as <a class="reference internal" href="../library/bsddb.html#module-bsddb" title="bsddb: Interface to Berkeley DB database library"><code class="xref py py-mod docutils literal">bsddb</code></a>.

1190

</li>

1191

<li>The new <a class="reference internal" href="../library/bz2.html#module-bz2" title="bz2: Interface to compression and decompression routines compatible with bzip2."><code class="xref py py-mod docutils literal">bz2</code></a> module is an interface to the bz2 data compression library.

1192

bz2-compressed data is usually smaller than corresponding <a class="reference internal" href="../library/zlib.html#module-zlib" title="zlib: Low-level interface to compression and decompression routines compatible with gzip."><code class="xref py py-mod docutils literal">zlib</code></a>-compressed data. (Contributed by Gustavo Niemeyer.)

1193

</li>

1194

<li>A set of standard date/time types has been added in the new <a class="reference internal" href="../library/datetime.html#module-datetime" title="datetime: Basic date and time types."><code class="xref py py-mod docutils literal">datetime</code></a>

1195

module. See the following section for more details.

1196

</li>

1197

<li>The Distutils <code class="xref py py-class docutils literal">Extension</code> class now supports an extra constructor

1198

argument named depends for listing additional source files that an extension

1199

depends on. This lets Distutils recompile the module if any of the dependency

1200

files are modified. For example, if <code class="file docutils literal">sampmodule.c</code> includes the header

1201

file <code class="file docutils literal">sample.h</code>, you would create the <code class="xref py py-class docutils literal">Extension</code> object like

1202

this:

1203

<div class="highlight-python"><div class="highlight"><pre>ext = Extension("samp",

1204

sources=["sampmodule.c"],

1205

depends=["sample.h"])

1206

</pre></div>

1207

</div>

1208

Modifying <code class="file docutils literal">sample.h</code> would then cause the module to be recompiled.

1209

(Contributed by Jeremy Hylton.)

1210

</li>

1211

<li>Other minor changes to Distutils: it now checks for the <code class="xref std std-envvar docutils literal">CC</code>,

1212

<code class="xref std std-envvar docutils literal">CFLAGS</code>, <code class="xref std std-envvar docutils literal">CPP</code>, <code class="xref std std-envvar docutils literal">LDFLAGS</code>, and <code class="xref std std-envvar docutils literal">CPPFLAGS</code>

1213

environment variables, using them to override the settings in Python’s

1214

configuration (contributed by Robert Weber).

1215

</li>

1216

<li>Previously the <a class="reference internal" href="../library/doctest.html#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> module would only search the docstrings of

1217

public methods and functions for test cases, but it now also examines private

1218

ones as well. The <code class="xref py py-func docutils literal">DocTestSuite(()</code> function creates a

1219

<a class="reference internal" href="../library/unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">unittest.TestSuite</code></a> object from a set of <a class="reference internal" href="../library/doctest.html#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> tests.

1220

</li>

1221

<li>The new <code class="xref py py-func docutils literal">gc.get_referents(object)()</code> function returns a list of all the

1222

objects referenced by object.

1223

</li>

1224

<li>The <a class="reference internal" href="../library/getopt.html#module-getopt" title="getopt: Portable parser for command line options; support both short and long option names."><code class="xref py py-mod docutils literal">getopt</code></a> module gained a new function, <code class="xref py py-func docutils literal">gnu_getopt()</code>, that

1225

supports the same arguments as the existing <a class="reference internal" href="../library/getopt.html#module-getopt" title="getopt: Portable parser for command line options; support both short and long option names."><code class="xref py py-func docutils literal">getopt()</code></a> function but uses

1226

GNU-style scanning mode. The existing <a class="reference internal" href="../library/getopt.html#module-getopt" title="getopt: Portable parser for command line options; support both short and long option names."><code class="xref py py-func docutils literal">getopt()</code></a> stops processing options as

1227

soon as a non-option argument is encountered, but in GNU-style mode processing

1228

continues, meaning that options and arguments can be mixed. For example:

1229

<div class="highlight-python"><div class="highlight"><pre>>>> getopt.getopt(['-f', 'filename', 'output', '-v'], 'f:v')

1230

([('-f', 'filename')], ['output', '-v'])

1231

>>> getopt.gnu_getopt(['-f', 'filename', 'output', '-v'], 'f:v')

1232

([('-f', 'filename'), ('-v', '')], ['output'])

1233

</pre></div>

1234

</div>

1235

(Contributed by Peter Åstrand.)

1236

</li>

1237

<li>The <a class="reference internal" href="../library/grp.html#module-grp" title="grp: The group database (getgrnam() and friends). (Unix)"><code class="xref py py-mod docutils literal">grp</code></a>, <a class="reference internal" href="../library/pwd.html#module-pwd" title="pwd: The password database (getpwnam() and friends). (Unix)"><code class="xref py py-mod docutils literal">pwd</code></a>, and <a class="reference internal" href="../library/resource.html#module-resource" title="resource: An interface to provide resource usage information on the current process. (Unix)"><code class="xref py py-mod docutils literal">resource</code></a> modules now return enhanced

1238

tuples:

1239

<div class="highlight-python"><div class="highlight"><pre>>>> import grp

1240

>>> g = grp.getgrnam('amk')

1241

>>> g.gr_name, g.gr_gid

1242

('amk', 500)

1243

</pre></div>

1244

</div>

1245

</li>

1246

<li>The <a class="reference internal" href="../library/gzip.html#module-gzip" title="gzip: Interfaces for gzip compression and decompression using file objects."><code class="xref py py-mod docutils literal">gzip</code></a> module can now handle files exceeding 2 GiB.

1247

</li>

1248

<li>The new <a class="reference internal" href="../library/heapq.html#module-heapq" title="heapq: Heap queue algorithm (a.k.a. priority queue)."><code class="xref py py-mod docutils literal">heapq</code></a> module contains an implementation of a heap queue

1249

algorithm. A heap is an array-like data structure that keeps items in a

1250

partially sorted order such that, for every index k, <code class="docutils literal">heap[k] <=

1251

heap[2*k+1]</code> and <code class="docutils literal">heap[k] <= heap[2*k+2]</code>. This makes it quick to remove the

1252

smallest item, and inserting a new item while maintaining the heap property is

1253

O(lg n). (See <a class="reference external" href="http://www.nist.gov/dads/HTML/priorityque.html">http://www.nist.gov/dads/HTML/priorityque.html</a> for more

1254

information about the priority queue data structure.)

1255

The <a class="reference internal" href="../library/heapq.html#module-heapq" title="heapq: Heap queue algorithm (a.k.a. priority queue)."><code class="xref py py-mod docutils literal">heapq</code></a> module provides <code class="xref py py-func docutils literal">heappush()</code> and <code class="xref py py-func docutils literal">heappop()</code> functions

1256

for adding and removing items while maintaining the heap property on top of some

1257

other mutable Python sequence type. Here’s an example that uses a Python list:

1258

<div class="highlight-python"><div class="highlight"><pre>>>> import heapq

1259

>>> heap = []

1260

>>> for item in [3, 7, 5, 11, 1]:

1261

... heapq.heappush(heap, item)

1262

...

1263

>>> heap

1264

[1, 3, 5, 11, 7]

1265

>>> heapq.heappop(heap)

1266

1

1267

>>> heapq.heappop(heap)

1268

3

1269

>>> heap

1270

[5, 7, 11]

1271

</pre></div>

1272

</div>

1273

(Contributed by Kevin O’Connor.)

1274

</li>

1275

<li>The IDLE integrated development environment has been updated using the code

1276

from the IDLEfork project (<a class="reference external" href="http://idlefork.sourceforge.net">http://idlefork.sourceforge.net</a>). The most notable feature is

1277

that the code being developed is now executed in a subprocess, meaning that

1278

there’s no longer any need for manual <code class="docutils literal">reload()</code> operations. IDLE’s core code

1279

has been incorporated into the standard library as the <code class="xref py py-mod docutils literal">idlelib</code> package.

1280

</li>

1281

<li>The <a class="reference internal" href="../library/imaplib.html#module-imaplib" title="imaplib: IMAP4 protocol client (requires sockets)."><code class="xref py py-mod docutils literal">imaplib</code></a> module now supports IMAP over SSL. (Contributed by Piers

1282

Lauder and Tino Lange.)

1283

</li>

1284

<li>The <a class="reference internal" href="../library/itertools.html#module-itertools" title="itertools: Functions creating iterators for efficient looping."><code class="xref py py-mod docutils literal">itertools</code></a> contains a number of useful functions for use with

1285

iterators, inspired by various functions provided by the ML and Haskell

1286

languages. For example, <code class="docutils literal">itertools.ifilter(predicate, iterator)</code> returns all

1287

elements in the iterator for which the function <code class="xref py py-func docutils literal">predicate()</code> returns

1288

<a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a>, and <code class="docutils literal">itertools.repeat(obj, N)</code> returns <code class="docutils literal">obj</code> N times.

1289

There are a number of other functions in the module; see the package’s reference

1290

documentation for details.

1291

(Contributed by Raymond Hettinger.)

1292

</li>

1293

<li>Two new functions in the <a class="reference internal" href="../library/math.html#module-math" title="math: Mathematical functions (sin() etc.)."><code class="xref py py-mod docutils literal">math</code></a> module, <code class="xref py py-func docutils literal">degrees(rads)()</code> and

1294

<code class="xref py py-func docutils literal">radians(degs)()</code>, convert between radians and degrees. Other functions in

1295

the <a class="reference internal" href="../library/math.html#module-math" title="math: Mathematical functions (sin() etc.)."><code class="xref py py-mod docutils literal">math</code></a> module such as <a class="reference internal" href="../library/math.html#math.sin" title="math.sin"><code class="xref py py-func docutils literal">math.sin()</code></a> and <a class="reference internal" href="../library/math.html#math.cos" title="math.cos"><code class="xref py py-func docutils literal">math.cos()</code></a> have always

1296

required input values measured in radians. Also, an optional base argument

1297

was added to <a class="reference internal" href="../library/math.html#math.log" title="math.log"><code class="xref py py-func docutils literal">math.log()</code></a> to make it easier to compute logarithms for bases

1298

other than <code class="docutils literal">e</code> and <code class="docutils literal">10</code>. (Contributed by Raymond Hettinger.)

1299

</li>

1300

<li>Several new POSIX functions (<code class="xref py py-func docutils literal">getpgid()</code>, <code class="xref py py-func docutils literal">killpg()</code>, <code class="xref py py-func docutils literal">lchown()</code>,

1301

<code class="xref py py-func docutils literal">loadavg()</code>, <code class="xref py py-func docutils literal">major()</code>, <code class="xref py py-func docutils literal">makedev()</code>, <code class="xref py py-func docutils literal">minor()</code>, and

1302

<code class="xref py py-func docutils literal">mknod()</code>) were added to the <a class="reference internal" href="../library/posix.html#module-posix" title="posix: The most common POSIX system calls (normally used via module os). (Unix)"><code class="xref py py-mod docutils literal">posix</code></a> module that underlies the

1303

<a class="reference internal" href="../library/os.html#module-os" title="os: Miscellaneous operating system interfaces."><code class="xref py py-mod docutils literal">os</code></a> module. (Contributed by Gustavo Niemeyer, Geert Jansen, and Denis S.

1304

Otkidach.)

1305

</li>

1306

<li>In the <a class="reference internal" href="../library/os.html#module-os" title="os: Miscellaneous operating system interfaces."><code class="xref py py-mod docutils literal">os</code></a> module, the <code class="xref py py-func docutils literal">*stat()</code> family of functions can now report

1307

fractions of a second in a timestamp. Such time stamps are represented as

1308

floats, similar to the value returned by <a class="reference internal" href="../library/time.html#time.time" title="time.time"><code class="xref py py-func docutils literal">time.time()</code></a>.

1309

During testing, it was found that some applications will break if time stamps

1310

are floats. For compatibility, when using the tuple interface of the

1311

<code class="xref py py-class docutils literal">stat_result</code> time stamps will be represented as integers. When using

1312

named fields (a feature first introduced in Python 2.2), time stamps are still

1313

represented as integers, unless <a class="reference internal" href="../library/os.html#os.stat_float_times" title="os.stat_float_times"><code class="xref py py-func docutils literal">os.stat_float_times()</code></a> is invoked to enable

1314

float return values:

1315

<div class="highlight-python"><div class="highlight"><pre>>>> os.stat("/tmp").st_mtime

1316

1034791200

1317

>>> os.stat_float_times(True)

1318

>>> os.stat("/tmp").st_mtime

1319

1034791200.6335014

1320

</pre></div>

1321

</div>

1322

In Python 2.4, the default will change to always returning floats.

1323

Application developers should enable this feature only if all their libraries

1324

work properly when confronted with floating point time stamps, or if they use

1325

the tuple API. If used, the feature should be activated on an application level

1326

instead of trying to enable it on a per-use basis.

1327

</li>

1328

<li>The <a class="reference internal" href="../library/optparse.html#module-optparse" title="optparse: Command-line option parsing library. (deprecated)"><code class="xref py py-mod docutils literal">optparse</code></a> module contains a new parser for command-line arguments

1329

that can convert option values to a particular Python type and will

1330

automatically generate a usage message. See the following section for more

1331

details.

1332

</li>

1333

<li>The old and never-documented <code class="xref py py-mod docutils literal">linuxaudiodev</code> module has been deprecated,

1334

and a new version named <a class="reference internal" href="../library/ossaudiodev.html#module-ossaudiodev" title="ossaudiodev: Access to OSS-compatible audio devices. (Linux, FreeBSD)"><code class="xref py py-mod docutils literal">ossaudiodev</code></a> has been added. The module was

1335

renamed because the OSS sound drivers can be used on platforms other than Linux,

1336

and the interface has also been tidied and brought up to date in various ways.

1337

(Contributed by Greg Ward and Nicholas FitzRoy-Dale.)

1338

</li>

1339

<li>The new <a class="reference internal" href="../library/platform.html#module-platform" title="platform: Retrieves as much platform identifying data as possible."><code class="xref py py-mod docutils literal">platform</code></a> module contains a number of functions that try to

1340

determine various properties of the platform you’re running on. There are

1341

functions for getting the architecture, CPU type, the Windows OS version, and

1342

even the Linux distribution version. (Contributed by Marc-André Lemburg.)

1343

</li>

1344

<li>The parser objects provided by the <code class="xref py py-mod docutils literal">pyexpat</code> module can now optionally

1345

buffer character data, resulting in fewer calls to your character data handler

1346

and therefore faster performance. Setting the parser object’s

1347

<code class="xref py py-attr docutils literal">buffer_text</code> attribute to <a class="reference internal" href="../library/constants.html#True" title="True"><code class="xref py py-const docutils literal">True</code></a> will enable buffering.

1348

</li>

1349

<li>The <code class="xref py py-func docutils literal">sample(population, k)()</code> function was added to the <a class="reference internal" href="../library/random.html#module-random" title="random: Generate pseudo-random numbers with various common distributions."><code class="xref py py-mod docutils literal">random</code></a>

1350

module. population is a sequence or <a class="reference internal" href="../library/functions.html#xrange" title="xrange"><code class="xref py py-class docutils literal">xrange</code></a> object containing the

1351

elements of a population, and <code class="xref py py-func docutils literal">sample()</code> chooses k elements from the

1352

population without replacing chosen elements. k can be any value up to

1353

<code class="docutils literal">len(population)</code>. For example:

1354

<div class="highlight-python"><div class="highlight"><pre>>>> days = ['Mo', 'Tu', 'We', 'Th', 'Fr', 'St', 'Sn']

1355

>>> random.sample(days, 3) # Choose 3 elements

1356

['St', 'Sn', 'Th']

1357

>>> random.sample(days, 7) # Choose 7 elements

1358

['Tu', 'Th', 'Mo', 'We', 'St', 'Fr', 'Sn']

1359

>>> random.sample(days, 7) # Choose 7 again

1360

['We', 'Mo', 'Sn', 'Fr', 'Tu', 'St', 'Th']

1361

>>> random.sample(days, 8) # Can't choose eight

1362

Traceback (most recent call last):

1363

File "<stdin>", line 1, in ?

1364

File "random.py", line 414, in sample

1365

raise ValueError, "sample larger than population"

1366

ValueError: sample larger than population

1367

>>> random.sample(xrange(1,10000,2), 10) # Choose ten odd nos. under 10000

1368

[3407, 3805, 1505, 7023, 2401, 2267, 9733, 3151, 8083, 9195]

1369

</pre></div>

1370

</div>

1371

The <a class="reference internal" href="../library/random.html#module-random" title="random: Generate pseudo-random numbers with various common distributions."><code class="xref py py-mod docutils literal">random</code></a> module now uses a new algorithm, the Mersenne Twister,

1372

implemented in C. It’s faster and more extensively studied than the previous

1373

algorithm.

1374

(All changes contributed by Raymond Hettinger.)

1375

</li>

1376

<li>The <a class="reference internal" href="../library/readline.html#module-readline" title="readline: GNU readline support for Python. (Unix)"><code class="xref py py-mod docutils literal">readline</code></a> module also gained a number of new functions:

1377

<code class="xref py py-func docutils literal">get_history_item()</code>, <code class="xref py py-func docutils literal">get_current_history_length()</code>, and

1378

<code class="xref py py-func docutils literal">redisplay()</code>.

1379

</li>

1380

<li>The <a class="reference internal" href="../library/rexec.html#module-rexec" title="rexec: Basic restricted execution framework. (deprecated)"><code class="xref py py-mod docutils literal">rexec</code></a> and <a class="reference internal" href="../library/bastion.html#module-Bastion" title="Bastion: Providing restricted access to objects. (deprecated)"><code class="xref py py-mod docutils literal">Bastion</code></a> modules have been declared dead, and

1381

attempts to import them will fail with a <a class="reference internal" href="../library/exceptions.html#exceptions.RuntimeError" title="exceptions.RuntimeError"><code class="xref py py-exc docutils literal">RuntimeError</code></a>. New-style classes

1382

provide new ways to break out of the restricted execution environment provided

1383

by <a class="reference internal" href="../library/rexec.html#module-rexec" title="rexec: Basic restricted execution framework. (deprecated)"><code class="xref py py-mod docutils literal">rexec</code></a>, and no one has interest in fixing them or time to do so. If

1384

you have applications using <a class="reference internal" href="../library/rexec.html#module-rexec" title="rexec: Basic restricted execution framework. (deprecated)"><code class="xref py py-mod docutils literal">rexec</code></a>, rewrite them to use something else.

1385

(Sticking with Python 2.2 or 2.1 will not make your applications any safer

1386

because there are known bugs in the <a class="reference internal" href="../library/rexec.html#module-rexec" title="rexec: Basic restricted execution framework. (deprecated)"><code class="xref py py-mod docutils literal">rexec</code></a> module in those versions. To

1387

repeat: if you’re using <a class="reference internal" href="../library/rexec.html#module-rexec" title="rexec: Basic restricted execution framework. (deprecated)"><code class="xref py py-mod docutils literal">rexec</code></a>, stop using it immediately.)

1388

</li>

1389

<li>The <code class="xref py py-mod docutils literal">rotor</code> module has been deprecated because the algorithm it uses for

1390

encryption is not believed to be secure. If you need encryption, use one of the

1391

several AES Python modules that are available separately.

1392

</li>

1393

<li>The <a class="reference internal" href="../library/shutil.html#module-shutil" title="shutil: High-level file operations, including copying."><code class="xref py py-mod docutils literal">shutil</code></a> module gained a <code class="xref py py-func docutils literal">move(src, dest)()</code> function that

1394

recursively moves a file or directory to a new location.

1395

</li>

1396

<li>Support for more advanced POSIX signal handling was added to the <a class="reference internal" href="../library/signal.html#module-signal" title="signal: Set handlers for asynchronous events."><code class="xref py py-mod docutils literal">signal</code></a>

1397

but then removed again as it proved impossible to make it work reliably across

1398

platforms.

1399

</li>

1400

<li>The <a class="reference internal" href="../library/socket.html#module-socket" title="socket: Low-level networking interface."><code class="xref py py-mod docutils literal">socket</code></a> module now supports timeouts. You can call the

1401

<code class="xref py py-meth docutils literal">settimeout(t)()</code> method on a socket object to set a timeout of t seconds.

1402

Subsequent socket operations that take longer than t seconds to complete will

1403

abort and raise a <a class="reference internal" href="../library/socket.html#socket.timeout" title="socket.timeout"><code class="xref py py-exc docutils literal">socket.timeout</code></a> exception.

1404

The original timeout implementation was by Tim O’Malley. Michael Gilfix

1405

integrated it into the Python <a class="reference internal" href="../library/socket.html#module-socket" title="socket: Low-level networking interface."><code class="xref py py-mod docutils literal">socket</code></a> module and shepherded it through a

1406

lengthy review. After the code was checked in, Guido van Rossum rewrote parts

1407

of it. (This is a good example of a collaborative development process in

1408

action.)

1409

</li>

1410

<li>On Windows, the <a class="reference internal" href="../library/socket.html#module-socket" title="socket: Low-level networking interface."><code class="xref py py-mod docutils literal">socket</code></a> module now ships with Secure Sockets Layer

1411

(SSL) support.

1412

</li>

1413

<li>The value of the C <code class="xref py py-const docutils literal">PYTHON_API_VERSION</code> macro is now exposed at the

1414

Python level as <code class="docutils literal">sys.api_version</code>. The current exception can be cleared by

1415

calling the new <a class="reference internal" href="../library/sys.html#sys.exc_clear" title="sys.exc_clear"><code class="xref py py-func docutils literal">sys.exc_clear()</code></a> function.

1416

</li>

1417

<li>The new <a class="reference internal" href="../library/tarfile.html#module-tarfile" title="tarfile: Read and write tar-format archive files."><code class="xref py py-mod docutils literal">tarfile</code></a> module allows reading from and writing to

1418

tar-format archive files. (Contributed by Lars Gustäbel.)

1419

</li>

1420

<li>The new <a class="reference internal" href="../library/textwrap.html#module-textwrap" title="textwrap: Text wrapping and filling"><code class="xref py py-mod docutils literal">textwrap</code></a> module contains functions for wrapping strings

1421

containing paragraphs of text. The <code class="xref py py-func docutils literal">wrap(text, width)()</code> function takes a

1422

string and returns a list containing the text split into lines of no more than

1423

the chosen width. The <code class="xref py py-func docutils literal">fill(text, width)()</code> function returns a single

1424

string, reformatted to fit into lines no longer than the chosen width. (As you

1425

can guess, <code class="xref py py-func docutils literal">fill()</code> is built on top of <code class="xref py py-func docutils literal">wrap()</code>. For example:

1426

<div class="highlight-python"><div class="highlight"><pre>>>> import textwrap

1427

>>> paragraph = "Not a whit, we defy augury: ... more text ..."

1428

>>> textwrap.wrap(paragraph, 60)

1429

["Not a whit, we defy augury: there's a special providence in",

1430

"the fall of a sparrow. If it be now, 'tis not to come; if it",

1431

...]

1432

>>> print textwrap.fill(paragraph, 35)

1433

Not a whit, we defy augury: there's

1434

a special providence in the fall of

1435

a sparrow. If it be now, 'tis not

1436

to come; if it be not to come, it

1437

will be now; if it be not now, yet

1438

it will come: the readiness is all.

1439

>>>

1440

</pre></div>

1441

</div>

1442

The module also contains a <code class="xref py py-class docutils literal">TextWrapper</code> class that actually implements

1443

the text wrapping strategy. Both the <code class="xref py py-class docutils literal">TextWrapper</code> class and the

1444

<code class="xref py py-func docutils literal">wrap()</code> and <code class="xref py py-func docutils literal">fill()</code> functions support a number of additional keyword

1445

arguments for fine-tuning the formatting; consult the module’s documentation

1446

for details. (Contributed by Greg Ward.)

1447

</li>

1448

<li>The <a class="reference internal" href="../library/thread.html#module-thread" title="thread: Create multiple threads of control within one interpreter."><code class="xref py py-mod docutils literal">thread</code></a> and <a class="reference internal" href="../library/threading.html#module-threading" title="threading: Higher-level threading interface."><code class="xref py py-mod docutils literal">threading</code></a> modules now have companion modules,

1449

<a class="reference internal" href="../library/dummy_thread.html#module-dummy_thread" title="dummy_thread: Drop-in replacement for the thread module."><code class="xref py py-mod docutils literal">dummy_thread</code></a> and <a class="reference internal" href="../library/dummy_threading.html#module-dummy_threading" title="dummy_threading: Drop-in replacement for the threading module."><code class="xref py py-mod docutils literal">dummy_threading</code></a>, that provide a do-nothing

1450

implementation of the <a class="reference internal" href="../library/thread.html#module-thread" title="thread: Create multiple threads of control within one interpreter."><code class="xref py py-mod docutils literal">thread</code></a> module’s interface for platforms where

1451

threads are not supported. The intention is to simplify thread-aware modules

1452

(ones that don’t rely on threads to run) by putting the following code at the

1453

top:

1454

<div class="highlight-python"><div class="highlight"><pre>try:

1455

import threading as _threading

1456

except ImportError:

1457

import dummy_threading as _threading

1458

</pre></div>

1459

</div>

1460

In this example, <code class="xref py py-mod docutils literal">_threading</code> is used as the module name to make it clear

1461

that the module being used is not necessarily the actual <a class="reference internal" href="../library/threading.html#module-threading" title="threading: Higher-level threading interface."><code class="xref py py-mod docutils literal">threading</code></a>

1462

module. Code can call functions and use classes in <code class="xref py py-mod docutils literal">_threading</code> whether or

1463

not threads are supported, avoiding an <a class="reference internal" href="../reference/compound_stmts.html#if"><code class="xref std std-keyword docutils literal">if</code></a> statement and making the

1464

code slightly clearer. This module will not magically make multithreaded code

1465

run without threads; code that waits for another thread to return or to do

1466

something will simply hang forever.

1467

</li>

1468

<li>The <a class="reference internal" href="../library/time.html#module-time" title="time: Time access and conversions."><code class="xref py py-mod docutils literal">time</code></a> module’s <code class="xref py py-func docutils literal">strptime()</code> function has long been an annoyance

1469

because it uses the platform C library’s <code class="xref py py-func docutils literal">strptime()</code> implementation, and

1470

different platforms sometimes have odd bugs. Brett Cannon contributed a

1471

portable implementation that’s written in pure Python and should behave

1472

identically on all platforms.

1473

</li>

1474

<li>The new <a class="reference internal" href="../library/timeit.html#module-timeit" title="timeit: Measure the execution time of small code snippets."><code class="xref py py-mod docutils literal">timeit</code></a> module helps measure how long snippets of Python code

1475

take to execute. The <code class="file docutils literal">timeit.py</code> file can be run directly from the

1476

command line, or the module’s <code class="xref py py-class docutils literal">Timer</code> class can be imported and used

1477

directly. Here’s a short example that figures out whether it’s faster to

1478

convert an 8-bit string to Unicode by appending an empty Unicode string to it or

1479

by using the <a class="reference internal" href="../library/functions.html#unicode" title="unicode"><code class="xref py py-func docutils literal">unicode()</code></a> function:

1480

<div class="highlight-python"><div class="highlight"><pre>import timeit

1481

1482

timer1 = timeit.Timer('unicode("abc")')

1483

timer2 = timeit.Timer('"abc" + u""')

1484

1485

# Run three trials

1486

print timer1.repeat(repeat=3, number=100000)

1487

print timer2.repeat(repeat=3, number=100000)

1488

1489

# On my laptop this outputs:

1490

# [0.36831796169281006, 0.37441694736480713, 0.35304892063140869]

1491

# [0.17574405670166016, 0.18193507194519043, 0.17565798759460449]

1492

</pre></div>

1493

</div>

1494

</li>

1495

<li>The <a class="reference internal" href="../library/tix.html#module-Tix" title="Tix: Tk Extension Widgets for Tkinter"><code class="xref py py-mod docutils literal">Tix</code></a> module has received various bug fixes and updates for the

1496

current version of the Tix package.

1497

</li>

1498

<li>The <a class="reference internal" href="../library/tkinter.html#module-Tkinter" title="Tkinter: Interface to Tcl/Tk for graphical user interfaces"><code class="xref py py-mod docutils literal">Tkinter</code></a> module now works with a thread-enabled version of Tcl.

1499

Tcl’s threading model requires that widgets only be accessed from the thread in

1500

which they’re created; accesses from another thread can cause Tcl to panic. For

1501

certain Tcl interfaces, <a class="reference internal" href="../library/tkinter.html#module-Tkinter" title="Tkinter: Interface to Tcl/Tk for graphical user interfaces"><code class="xref py py-mod docutils literal">Tkinter</code></a> will now automatically avoid this when a

1502

widget is accessed from a different thread by marshalling a command, passing it

1503

to the correct thread, and waiting for the results. Other interfaces can’t be

1504

handled automatically but <a class="reference internal" href="../library/tkinter.html#module-Tkinter" title="Tkinter: Interface to Tcl/Tk for graphical user interfaces"><code class="xref py py-mod docutils literal">Tkinter</code></a> will now raise an exception on such an

1505

access so that you can at least find out about the problem. See

1506

<a class="reference external" href="https://mail.python.org/pipermail/python-dev/2002-December/031107.html">https://mail.python.org/pipermail/python-dev/2002-December/031107.html</a> for a more

1507

detailed explanation of this change. (Implemented by Martin von Löwis.)

1508

</li>

1509

<li>Calling Tcl methods through <code class="xref py py-mod docutils literal">_tkinter</code> no longer returns only strings.

1510

Instead, if Tcl returns other objects those objects are converted to their

1511

Python equivalent, if one exists, or wrapped with a <code class="xref py py-class docutils literal">_tkinter.Tcl_Obj</code>

1512

object if no Python equivalent exists. This behavior can be controlled through

1513

the <code class="xref py py-meth docutils literal">wantobjects()</code> method of <code class="xref py py-class docutils literal">tkapp</code> objects.

1514

When using <code class="xref py py-mod docutils literal">_tkinter</code> through the <a class="reference internal" href="../library/tkinter.html#module-Tkinter" title="Tkinter: Interface to Tcl/Tk for graphical user interfaces"><code class="xref py py-mod docutils literal">Tkinter</code></a> module (as most Tkinter

1515

applications will), this feature is always activated. It should not cause

1516

compatibility problems, since Tkinter would always convert string results to

1517

Python types where possible.

1518

If any incompatibilities are found, the old behavior can be restored by setting

1519

the <code class="xref py py-attr docutils literal">wantobjects</code> variable in the <a class="reference internal" href="../library/tkinter.html#module-Tkinter" title="Tkinter: Interface to Tcl/Tk for graphical user interfaces"><code class="xref py py-mod docutils literal">Tkinter</code></a> module to false before

1520

creating the first <code class="xref py py-class docutils literal">tkapp</code> object.

1521

<div class="highlight-python"><div class="highlight"><pre>import Tkinter

1522

Tkinter.wantobjects = 0

1523

</pre></div>

1524

</div>

1525

Any breakage caused by this change should be reported as a bug.

1526

</li>

1527

<li>The <a class="reference internal" href="../library/userdict.html#module-UserDict" title="UserDict: Class wrapper for dictionary objects."><code class="xref py py-mod docutils literal">UserDict</code></a> module has a new <code class="xref py py-class docutils literal">DictMixin</code> class which defines

1528

all dictionary methods for classes that already have a minimum mapping

1529

interface. This greatly simplifies writing classes that need to be

1530

substitutable for dictionaries, such as the classes in the <a class="reference internal" href="../library/shelve.html#module-shelve" title="shelve: Python object persistence."><code class="xref py py-mod docutils literal">shelve</code></a>

1531

module.

1532

Adding the mix-in as a superclass provides the full dictionary interface

1533

whenever the class defines <a class="reference internal" href="../reference/datamodel.html#object.__getitem__" title="object.__getitem__"><code class="xref py py-meth docutils literal">__getitem__()</code></a>, <a class="reference internal" href="../reference/datamodel.html#object.__setitem__" title="object.__setitem__"><code class="xref py py-meth docutils literal">__setitem__()</code></a>,

1534

<a class="reference internal" href="../reference/datamodel.html#object.__delitem__" title="object.__delitem__"><code class="xref py py-meth docutils literal">__delitem__()</code></a>, and <code class="xref py py-meth docutils literal">keys()</code>. For example:

1535

<div class="highlight-python"><div class="highlight"><pre>>>> import UserDict

1536

>>> class SeqDict(UserDict.DictMixin):

1537

... """Dictionary lookalike implemented with lists."""

1538

... def __init__(self):

1539

... self.keylist = []

1540

... self.valuelist = []

1541

... def __getitem__(self, key):

1542

... try:

1543

... i = self.keylist.index(key)

1544

... except ValueError:

1545

... raise KeyError

1546

... return self.valuelist[i]

1547

... def __setitem__(self, key, value):

1548

... try:

1549

1550

... self.valuelist[i] = value

1551

... except ValueError:

1552

... self.keylist.append(key)

1553

... self.valuelist.append(value)

1554

... def __delitem__(self, key):

1555

... try:

1556

1557

... except ValueError:

1558

... raise KeyError

1559

... self.keylist.pop(i)

1560

... self.valuelist.pop(i)

1561

... def keys(self):

1562

... return list(self.keylist)

1563

...

1564

>>> s = SeqDict()

1565

>>> dir(s) # See that other dictionary methods are implemented

1566

['__cmp__', '__contains__', '__delitem__', '__doc__', '__getitem__',

1567

'__init__', '__iter__', '__len__', '__module__', '__repr__',

1568

'__setitem__', 'clear', 'get', 'has_key', 'items', 'iteritems',

1569

'iterkeys', 'itervalues', 'keylist', 'keys', 'pop', 'popitem',

1570

'setdefault', 'update', 'valuelist', 'values']

1571

</pre></div>

1572

</div>

1573

(Contributed by Raymond Hettinger.)

1574

</li>

1575

<li>The DOM implementation in <a class="reference internal" href="../library/xml.dom.minidom.html#module-xml.dom.minidom" title="xml.dom.minidom: Minimal Document Object Model (DOM) implementation."><code class="xref py py-mod docutils literal">xml.dom.minidom</code></a> can now generate XML output

1576

in a particular encoding by providing an optional encoding argument to the

1577

<code class="xref py py-meth docutils literal">toxml()</code> and <code class="xref py py-meth docutils literal">toprettyxml()</code> methods of DOM nodes.

1578

</li>

1579

<li>The <a class="reference internal" href="../library/xmlrpclib.html#module-xmlrpclib" title="xmlrpclib: XML-RPC client access."><code class="xref py py-mod docutils literal">xmlrpclib</code></a> module now supports an XML-RPC extension for handling nil

1580

data values such as Python’s <code class="docutils literal">None</code>. Nil values are always supported on

1581

unmarshalling an XML-RPC response. To generate requests containing <code class="docutils literal">None</code>,

1582

you must supply a true value for the allow_none parameter when creating a

1583

<code class="xref py py-class docutils literal">Marshaller</code> instance.

1584

</li>

1585

<li>The new <a class="reference internal" href="../library/docxmlrpcserver.html#module-DocXMLRPCServer" title="DocXMLRPCServer: Self-documenting XML-RPC server implementation."><code class="xref py py-mod docutils literal">DocXMLRPCServer</code></a> module allows writing self-documenting XML-RPC

1586

servers. Run it in demo mode (as a program) to see it in action. Pointing the

1587

Web browser to the RPC server produces pydoc-style documentation; pointing

1588

xmlrpclib to the server allows invoking the actual methods. (Contributed by

1589

Brian Quinlan.)

1590

</li>

1591

<li>Support for internationalized domain names (RFCs 3454, 3490, 3491, and 3492)

1592

has been added. The “idna” encoding can be used to convert between a Unicode

1593

domain name and the ASCII-compatible encoding (ACE) of that name.

1594

<div class="highlight-python"><div class="highlight"><pre>>{}>{}> u"www.Alliancefrançaise.nu".encode("idna")

1595

'www.xn--alliancefranaise-npb.nu'

1596

</pre></div>

1597

</div>

1598

The <a class="reference internal" href="../library/socket.html#module-socket" title="socket: Low-level networking interface."><code class="xref py py-mod docutils literal">socket</code></a> module has also been extended to transparently convert

1599

Unicode hostnames to the ACE version before passing them to the C library.

1600

Modules that deal with hostnames such as <a class="reference internal" href="../library/httplib.html#module-httplib" title="httplib: HTTP and HTTPS protocol client (requires sockets)."><code class="xref py py-mod docutils literal">httplib</code></a> and <a class="reference internal" href="../library/ftplib.html#module-ftplib" title="ftplib: FTP protocol client (requires sockets)."><code class="xref py py-mod docutils literal">ftplib</code></a>)

1601

also support Unicode host names; <a class="reference internal" href="../library/httplib.html#module-httplib" title="httplib: HTTP and HTTPS protocol client (requires sockets)."><code class="xref py py-mod docutils literal">httplib</code></a> also sends HTTP <code class="docutils literal">Host</code>

1602

headers using the ACE version of the domain name. <a class="reference internal" href="../library/urllib.html#module-urllib" title="urllib: Open an arbitrary network resource by URL (requires sockets)."><code class="xref py py-mod docutils literal">urllib</code></a> supports

1603

Unicode URLs with non-ASCII host names as long as the <code class="docutils literal">path</code> part of the URL

1604

is ASCII only.

1605

To implement this change, the <a class="reference internal" href="../library/stringprep.html#module-stringprep" title="stringprep: String preparation, as per RFC 3453"><code class="xref py py-mod docutils literal">stringprep</code></a> module, the <code class="docutils literal">mkstringprep</code>

1606

tool and the <code class="docutils literal">punycode</code> encoding have been added.

1607

</li>

1608

</ul>

1609

1610

1611

Date and time types suitable for expressing timestamps were added as the

1612

<a class="reference internal" href="../library/datetime.html#module-datetime" title="datetime: Basic date and time types."><code class="xref py py-mod docutils literal">datetime</code></a> module. The types don’t support different calendars or many

1613

fancy features, and just stick to the basics of representing time.

1614

The three primary types are: <code class="xref py py-class docutils literal">date</code>, representing a day, month, and year;

1615

<a class="reference internal" href="../library/time.html#module-time" title="time: Time access and conversions."><code class="xref py py-class docutils literal">time</code></a>, consisting of hour, minute, and second; and <a class="reference internal" href="../library/datetime.html#module-datetime" title="datetime: Basic date and time types."><code class="xref py py-class docutils literal">datetime</code></a>,

1616

which contains all the attributes of both <code class="xref py py-class docutils literal">date</code> and <a class="reference internal" href="../library/time.html#module-time" title="time: Time access and conversions."><code class="xref py py-class docutils literal">time</code></a>.

1617

There’s also a <code class="xref py py-class docutils literal">timedelta</code> class representing differences between two

1618

points in time, and time zone logic is implemented by classes inheriting from

1619

the abstract <code class="xref py py-class docutils literal">tzinfo</code> class.

1620

You can create instances of <code class="xref py py-class docutils literal">date</code> and <a class="reference internal" href="../library/time.html#module-time" title="time: Time access and conversions."><code class="xref py py-class docutils literal">time</code></a> by either supplying

1621

keyword arguments to the appropriate constructor, e.g.

1622

<code class="docutils literal">datetime.date(year=1972, month=10, day=15)</code>, or by using one of a number of

1623

class methods. For example, the <code class="xref py py-meth docutils literal">date.today()</code> class method returns the

1624

current local date.

1625

Once created, instances of the date/time classes are all immutable. There are a

1626

number of methods for producing formatted strings from objects:

1627

<div class="highlight-python"><div class="highlight"><pre>>>> import datetime

1628

>>> now = datetime.datetime.now()

1629

>>> now.isoformat()

1630

'2002-12-30T21:27:03.994956'

1631

>>> now.ctime() # Only available on date, datetime

1632

'Mon Dec 30 21:27:03 2002'

1633

>>> now.strftime('%Y %d %b')

1634

'2002 30 Dec'

1635

</pre></div>

1636

</div>

1637

The <code class="xref py py-meth docutils literal">replace()</code> method allows modifying one or more fields of a

1638

<code class="xref py py-class docutils literal">date</code> or <a class="reference internal" href="../library/datetime.html#module-datetime" title="datetime: Basic date and time types."><code class="xref py py-class docutils literal">datetime</code></a> instance, returning a new instance:

1639

<div class="highlight-python"><div class="highlight"><pre>>>> d = datetime.datetime.now()

1640

>>> d

1641

datetime.datetime(2002, 12, 30, 22, 15, 38, 827738)

1642

>>> d.replace(year=2001, hour = 12)

1643

datetime.datetime(2001, 12, 30, 12, 15, 38, 827738)

1644

>>>

1645

</pre></div>

1646

</div>

1647

Instances can be compared, hashed, and converted to strings (the result is the

1648

same as that of <code class="xref py py-meth docutils literal">isoformat()</code>). <code class="xref py py-class docutils literal">date</code> and <a class="reference internal" href="../library/datetime.html#module-datetime" title="datetime: Basic date and time types."><code class="xref py py-class docutils literal">datetime</code></a>

1649

instances can be subtracted from each other, and added to <code class="xref py py-class docutils literal">timedelta</code>

1650

instances. The largest missing feature is that there’s no standard library

1651

support for parsing strings and getting back a <code class="xref py py-class docutils literal">date</code> or

1652

<a class="reference internal" href="../library/datetime.html#module-datetime" title="datetime: Basic date and time types."><code class="xref py py-class docutils literal">datetime</code></a>.

1653

For more information, refer to the module’s reference documentation.

1654

(Contributed by Tim Peters.)

1655

</div>

1656

1657

<h3>The optparse Module<a class="headerlink" href="#the-optparse-module" title="Permalink to this headline">¶</a></h3>

1658

The <a class="reference internal" href="../library/getopt.html#module-getopt" title="getopt: Portable parser for command line options; support both short and long option names."><code class="xref py py-mod docutils literal">getopt</code></a> module provides simple parsing of command-line arguments. The

1659

new <a class="reference internal" href="../library/optparse.html#module-optparse" title="optparse: Command-line option parsing library. (deprecated)"><code class="xref py py-mod docutils literal">optparse</code></a> module (originally named Optik) provides more elaborate

1660

command-line parsing that follows the Unix conventions, automatically creates

1661

the output for <a class="reference internal" href="../using/cmdline.html#cmdoption--help"><code class="xref std std-option docutils literal">--help</code></a>, and can perform different actions for different

1662

options.

1663

You start by creating an instance of <code class="xref py py-class docutils literal">OptionParser</code> and telling it what

1664

your program’s options are.

1665

<div class="highlight-python"><div class="highlight"><pre>import sys

1666

from optparse import OptionParser

1667

1668

op = OptionParser()

1669

op.add_option('-i', '--input',

1670

action='store', type='string', dest='input',

1671

help='set input filename')

1672

op.add_option('-l', '--length',

1673

action='store', type='int', dest='length',

1674

help='set maximum length of output')

1675

</pre></div>

1676

</div>

1677

Parsing a command line is then done by calling the <code class="xref py py-meth docutils literal">parse_args()</code> method.

1678

<div class="highlight-python"><div class="highlight"><pre>options, args = op.parse_args(sys.argv[1:])

1679

print options

1680

print args

1681

</pre></div>

1682

</div>

1683

This returns an object containing all of the option values, and a list of

1684

strings containing the remaining arguments.

1685

Invoking the script with the various arguments now works as you’d expect it to.

1686

Note that the length argument is automatically converted to an integer.

1687

<div class="highlight-python"><div class="highlight"><pre>$ ./python opt.py -i data arg1

1688

1689

['arg1']

1690

$ ./python opt.py --input=data --length=4

1691

1692

[]

1693

1694

</pre></div>

1695

</div>

1696

The help message is automatically generated for you:

1697

<div class="highlight-python"><div class="highlight"><pre>$ ./python opt.py --help

1698

usage: opt.py [options]

1699

1700

options:

1701

-h, --help show this help message and exit

1702

-iINPUT, --input=INPUT

1703

set input filename

1704

-lLENGTH, --length=LENGTH

1705

set maximum length of output

1706

1707

</pre></div>

1708

</div>

1709

See the module’s documentation for more details.

1710

Optik was written by Greg Ward, with suggestions from the readers of the Getopt

1711

SIG.

1712

</div>

1713

</div>

1714

1715

<h2>Pymalloc: A Specialized Object Allocator<a class="headerlink" href="#pymalloc-a-specialized-object-allocator" title="Permalink to this headline">¶</a></h2>

1716

Pymalloc, a specialized object allocator written by Vladimir Marangozov, was a

1717

feature added to Python 2.1. Pymalloc is intended to be faster than the system

1718

<code class="xref c c-func docutils literal">malloc()</code> and to have less memory overhead for allocation patterns typical

1719

of Python programs. The allocator uses C’s <code class="xref c c-func docutils literal">malloc()</code> function to get large

1720

pools of memory and then fulfills smaller memory requests from these pools.

1721

In 2.1 and 2.2, pymalloc was an experimental feature and wasn’t enabled by

1722

default; you had to explicitly enable it when compiling Python by providing the

1723

<code class="xref std std-option docutils literal">--with-pymalloc</code> option to the configure script. In 2.3,

1724

pymalloc has had further enhancements and is now enabled by default; you’ll have

1725

to supply <code class="xref std std-option docutils literal">--without-pymalloc</code> to disable it.

1726

This change is transparent to code written in Python; however, pymalloc may

1727

expose bugs in C extensions. Authors of C extension modules should test their

1728

code with pymalloc enabled, because some incorrect code may cause core dumps at

1729

runtime.

1730

There’s one particularly common error that causes problems. There are a number

1731

of memory allocation functions in Python’s C API that have previously just been

1732

aliases for the C library’s <code class="xref c c-func docutils literal">malloc()</code> and <code class="xref c c-func docutils literal">free()</code>, meaning that if

1733

you accidentally called mismatched functions the error wouldn’t be noticeable.

1734

When the object allocator is enabled, these functions aren’t aliases of

1735

<code class="xref c c-func docutils literal">malloc()</code> and <code class="xref c c-func docutils literal">free()</code> any more, and calling the wrong function to

1736

free memory may get you a core dump. For example, if memory was allocated using

1737

<code class="xref c c-func docutils literal">PyObject_Malloc()</code>, it has to be freed using <code class="xref c c-func docutils literal">PyObject_Free()</code>, not

1738

<code class="xref c c-func docutils literal">free()</code>. A few modules included with Python fell afoul of this and had to

1739

be fixed; doubtless there are more third-party modules that will have the same

1740

problem.

1741

As part of this change, the confusing multiple interfaces for allocating memory

1742

have been consolidated down into two API families. Memory allocated with one

1743

family must not be manipulated with functions from the other family. There is

1744

one family for allocating chunks of memory and another family of functions

1745

specifically for allocating Python objects.

1746

1747

<li>To allocate and free an undistinguished chunk of memory use the “raw memory”

1748

family: <a class="reference internal" href="../c-api/memory.html#c.PyMem_Malloc" title="PyMem_Malloc"><code class="xref c c-func docutils literal">PyMem_Malloc()</code></a>, <a class="reference internal" href="../c-api/memory.html#c.PyMem_Realloc" title="PyMem_Realloc"><code class="xref c c-func docutils literal">PyMem_Realloc()</code></a>, and <a class="reference internal" href="../c-api/memory.html#c.PyMem_Free" title="PyMem_Free"><code class="xref c c-func docutils literal">PyMem_Free()</code></a>.</li>

1749

<li>The “object memory” family is the interface to the pymalloc facility described

1750

above and is biased towards a large number of “small” allocations:

1751

<code class="xref c c-func docutils literal">PyObject_Malloc()</code>, <code class="xref c c-func docutils literal">PyObject_Realloc()</code>, and <code class="xref c c-func docutils literal">PyObject_Free()</code>.</li>

1752

<li>To allocate and free Python objects, use the “object” family

1753

<a class="reference internal" href="../c-api/allocation.html#c.PyObject_New" title="PyObject_New"><code class="xref c c-func docutils literal">PyObject_New()</code></a>, <a class="reference internal" href="../c-api/allocation.html#c.PyObject_NewVar" title="PyObject_NewVar"><code class="xref c c-func docutils literal">PyObject_NewVar()</code></a>, and <a class="reference internal" href="../c-api/allocation.html#c.PyObject_Del" title="PyObject_Del"><code class="xref c c-func docutils literal">PyObject_Del()</code></a>.</li>

1754

</ul>

1755

Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides debugging

1756

features to catch memory overwrites and doubled frees in both extension modules

1757

and in the interpreter itself. To enable this support, compile a debugging

1758

version of the Python interpreter by running configure with

1759

<code class="xref std std-option docutils literal">--with-pydebug</code>.

1760

To aid extension writers, a header file <code class="file docutils literal">Misc/pymemcompat.h</code> is

1761

distributed with the source to Python 2.3 that allows Python extensions to use

1762

the 2.3 interfaces to memory allocation while compiling against any version of

1763

Python since 1.5.2. You would copy the file from Python’s source distribution

1764

and bundle it with the source of your extension.

1765

1766

See also

1767

1768

<dt><a class="reference external" href="https://svn.python.org/view/python/trunk/Objects/obmalloc.c">https://svn.python.org/view/python/trunk/Objects/obmalloc.c</a></dt>

1769

<dd>For the full details of the pymalloc implementation, see the comments at

1770

the top of the file <code class="file docutils literal">Objects/obmalloc.c</code> in the Python source code.

1771

The above link points to the file within the python.org SVN browser.</dd>

1772

</dl>

1773

</div>

1774

</div>

1775

1776

<h2>Build and C API Changes<a class="headerlink" href="#build-and-c-api-changes" title="Permalink to this headline">¶</a></h2>

1777

Changes to Python’s build process and to the C API include:

1778

1779

<li>The cycle detection implementation used by the garbage collection has proven

1780

to be stable, so it’s now been made mandatory. You can no longer compile Python

1781

without it, and the <code class="xref std std-option docutils literal">--with-cycle-gc</code> switch to configure has

1782

been removed.</li>

1783

<li>Python can now optionally be built as a shared library

1784

(<code class="file docutils literal">libpython2.3.so</code>) by supplying <code class="xref std std-option docutils literal">--enable-shared</code> when running

1785

Python’s configure script. (Contributed by Ondrej Palkovsky.)</li>

1786

<li>The <code class="xref c c-macro docutils literal">DL_EXPORT</code> and <code class="xref c c-macro docutils literal">DL_IMPORT</code> macros are now deprecated.

1787

Initialization functions for Python extension modules should now be declared

1788

using the new macro <code class="xref c c-macro docutils literal">PyMODINIT_FUNC</code>, while the Python core will

1789

generally use the <code class="xref c c-macro docutils literal">PyAPI_FUNC</code> and <code class="xref c c-macro docutils literal">PyAPI_DATA</code> macros.</li>

1790

<li>The interpreter can be compiled without any docstrings for the built-in

1791

functions and modules by supplying <code class="xref std std-option docutils literal">--without-doc-strings</code> to the

1792

configure script. This makes the Python executable about 10% smaller,

1793

but will also mean that you can’t get help for Python’s built-ins. (Contributed

1794

by Gustavo Niemeyer.)</li>

1795

<li>The <code class="xref c c-func docutils literal">PyArg_NoArgs()</code> macro is now deprecated, and code that uses it

1796

should be changed. For Python 2.2 and later, the method definition table can

1797

specify the <a class="reference internal" href="../c-api/structures.html#METH_NOARGS" title="METH_NOARGS"><code class="xref py py-const docutils literal">METH_NOARGS</code></a> flag, signalling that there are no arguments,

1798

and the argument checking can then be removed. If compatibility with pre-2.2

1799

versions of Python is important, the code could use <code class="docutils literal">PyArg_ParseTuple(args,

1800

"")</code> instead, but this will be slower than using <a class="reference internal" href="../c-api/structures.html#METH_NOARGS" title="METH_NOARGS"><code class="xref py py-const docutils literal">METH_NOARGS</code></a>.</li>

1801

<li><a class="reference internal" href="../c-api/arg.html#c.PyArg_ParseTuple" title="PyArg_ParseTuple"><code class="xref c c-func docutils literal">PyArg_ParseTuple()</code></a> accepts new format characters for various sizes of

1802

unsigned integers: <code class="docutils literal">B</code> for <code class="xref c c-type docutils literal">unsigned char</code>, <code class="docutils literal">H</code> for <code class="xref c c-type docutils literal">unsigned

1803

short int</code>, <code class="docutils literal">I</code> for <code class="xref c c-type docutils literal">unsigned int</code>, and <code class="docutils literal">K</code> for <code class="xref c c-type docutils literal">unsigned

1804

long long</code>.</li>

1805

<li>A new function, <code class="xref c c-func docutils literal">PyObject_DelItemString(mapping, char *key)()</code> was added

1806

as shorthand for <code class="docutils literal">PyObject_DelItem(mapping, PyString_New(key))</code>.</li>

1807

<li>File objects now manage their internal string buffer differently, increasing

1808

it exponentially when needed. This results in the benchmark tests in

1809

<code class="file docutils literal">Lib/test/test_bufio.py</code> speeding up considerably (from 57 seconds to 1.7

1810

seconds, according to one measurement).</li>

1811

<li>It’s now possible to define class and static methods for a C extension type by

1812

setting either the <a class="reference internal" href="../c-api/structures.html#METH_CLASS" title="METH_CLASS"><code class="xref py py-const docutils literal">METH_CLASS</code></a> or <a class="reference internal" href="../c-api/structures.html#METH_STATIC" title="METH_STATIC"><code class="xref py py-const docutils literal">METH_STATIC</code></a> flags in a

1813

method’s <a class="reference internal" href="../c-api/structures.html#c.PyMethodDef" title="PyMethodDef"><code class="xref c c-type docutils literal">PyMethodDef</code></a> structure.</li>

1814

<li>Python now includes a copy of the Expat XML parser’s source code, removing any

1815

dependence on a system version or local installation of Expat.</li>

1816

<li>If you dynamically allocate type objects in your extension, you should be

1817

aware of a change in the rules relating to the <code class="xref py py-attr docutils literal">__module__</code> and

1818

<code class="xref py py-attr docutils literal">__name__</code> attributes. In summary, you will want to ensure the type’s

1819

dictionary contains a <code class="docutils literal">'__module__'</code> key; making the module name the part of

1820

the type name leading up to the final period will no longer have the desired

1821

effect. For more detail, read the API reference documentation or the source.</li>

1822

</ul>

1823

1824

<h3>Port-Specific Changes<a class="headerlink" href="#port-specific-changes" title="Permalink to this headline">¶</a></h3>

1825

Support for a port to IBM’s OS/2 using the EMX runtime environment was merged

1826

into the main Python source tree. EMX is a POSIX emulation layer over the OS/2

1827

system APIs. The Python port for EMX tries to support all the POSIX-like

1828

capability exposed by the EMX runtime, and mostly succeeds; <code class="xref py py-func docutils literal">fork()</code> and

1829

<a class="reference internal" href="../library/fcntl.html#module-fcntl" title="fcntl: The fcntl() and ioctl() system calls. (Unix)"><code class="xref py py-func docutils literal">fcntl()</code></a> are restricted by the limitations of the underlying emulation

1830

layer. The standard OS/2 port, which uses IBM’s Visual Age compiler, also

1831

gained support for case-sensitive import semantics as part of the integration of

1832

the EMX port into CVS. (Contributed by Andrew MacIntyre.)

1833

On MacOS, most toolbox modules have been weaklinked to improve backward

1834

compatibility. This means that modules will no longer fail to load if a single

1835

routine is missing on the current OS version. Instead calling the missing

1836

routine will raise an exception. (Contributed by Jack Jansen.)

1837

The RPM spec files, found in the <code class="file docutils literal">Misc/RPM/</code> directory in the Python

1838

source distribution, were updated for 2.3. (Contributed by Sean Reifschneider.)

1839

Other new platforms now supported by Python include AtheOS

1840

(<a class="reference external" href="http://www.atheos.cx/">http://www.atheos.cx/</a>), GNU/Hurd, and OpenVMS.

1841

</div>

1842

</div>

1843

1844

<h2>Other Changes and Fixes<a class="headerlink" href="#other-changes-and-fixes" title="Permalink to this headline">¶</a></h2>

1845

As usual, there were a bunch of other improvements and bugfixes scattered

1846

throughout the source tree. A search through the CVS change logs finds there

1847

were 523 patches applied and 514 bugs fixed between Python 2.2 and 2.3. Both

1848

figures are likely to be underestimates.

1849

Some of the more notable changes are:

1850

<ul>

1851

<li>If the <a class="reference internal" href="../using/cmdline.html#envvar-PYTHONINSPECT"><code class="xref std std-envvar docutils literal">PYTHONINSPECT</code></a> environment variable is set, the Python

1852

interpreter will enter the interactive prompt after running a Python program, as

1853

if Python had been invoked with the <a class="reference internal" href="../using/cmdline.html#cmdoption-i"><code class="xref std std-option docutils literal">-i</code></a> option. The environment

1854

variable can be set before running the Python interpreter, or it can be set by

1855

the Python program as part of its execution.

1856

</li>

1857

<li>The <code class="file docutils literal">regrtest.py</code> script now provides a way to allow “all resources

1858

except foo.” A resource name passed to the <a class="reference internal" href="../using/cmdline.html#cmdoption-u"><code class="xref std std-option docutils literal">-u</code></a> option can now be

1859

prefixed with a hyphen (<code class="docutils literal">'-'</code>) to mean “remove this resource.” For example,

1860

the option ‘<code class="docutils literal">-uall,-bsddb</code>‘ could be used to enable the use of all resources

1861

except <code class="docutils literal">bsddb</code>.

1862

</li>

1863

<li>The tools used to build the documentation now work under Cygwin as well as

1864

Unix.

1865

</li>

1866

<li>The <code class="docutils literal">SET_LINENO</code> opcode has been removed. Back in the mists of time, this

1867

opcode was needed to produce line numbers in tracebacks and support trace

1868

functions (for, e.g., <a class="reference internal" href="../library/pdb.html#module-pdb" title="pdb: The Python debugger for interactive interpreters."><code class="xref py py-mod docutils literal">pdb</code></a>). Since Python 1.5, the line numbers in

1869

tracebacks have been computed using a different mechanism that works with

1870

“python -O”. For Python 2.3 Michael Hudson implemented a similar scheme to

1871

determine when to call the trace function, removing the need for <code class="docutils literal">SET_LINENO</code>

1872

entirely.

1873

It would be difficult to detect any resulting difference from Python code, apart

1874

from a slight speed up when Python is run without <a class="reference internal" href="../using/cmdline.html#cmdoption-O"><code class="xref std std-option docutils literal">-O</code></a>.

1875

C extensions that access the <code class="xref py py-attr docutils literal">f_lineno</code> field of frame objects should

1876

instead call <code class="docutils literal">PyCode_Addr2Line(f->f_code, f->f_lasti)</code>. This will have the

1877

added effect of making the code work as desired under “python -O” in earlier

1878

versions of Python.

1879

A nifty new feature is that trace functions can now assign to the

1880

<code class="xref py py-attr docutils literal">f_lineno</code> attribute of frame objects, changing the line that will be

1881

executed next. A <code class="docutils literal">jump</code> command has been added to the <a class="reference internal" href="../library/pdb.html#module-pdb" title="pdb: The Python debugger for interactive interpreters."><code class="xref py py-mod docutils literal">pdb</code></a> debugger

1882

taking advantage of this new feature. (Implemented by Richie Hindle.)

1883

</li>

1884

</ul>

1885

</div>

1886

1887

<h2>Porting to Python 2.3<a class="headerlink" href="#porting-to-python-2-3" title="Permalink to this headline">¶</a></h2>

1888

This section lists previously described changes that may require changes to your

1889

code:

1890

<ul>

1891

<li><a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a> is now always a keyword; if it’s used as a variable name in

1892

your code, a different name must be chosen.

1893

</li>

1894

<li>For strings X and Y, <code class="docutils literal">X in Y</code> now works if X is more than one

1895

character long.

1896

</li>

1897

1898

1899

large to fit into an integer.

1900

</li>

1901

<li>If you have Unicode strings that contain 8-bit characters, you must declare

1902

the file’s encoding (UTF-8, Latin-1, or whatever) by adding a comment to the top

1903

of the file. See section <a class="reference internal" href="#section-encodings">PEP 263: Source Code Encodings</a> for more information.

1904

</li>

1905

<li>Calling Tcl methods through <code class="xref py py-mod docutils literal">_tkinter</code> no longer returns only strings.

1906

Instead, if Tcl returns other objects those objects are converted to their

1907

Python equivalent, if one exists, or wrapped with a <code class="xref py py-class docutils literal">_tkinter.Tcl_Obj</code>

1908

object if no Python equivalent exists.

1909

</li>

1910

<li>Large octal and hex literals such as <code class="docutils literal">0xffffffff</code> now trigger a

1911

<a class="reference internal" href="../library/exceptions.html#exceptions.FutureWarning" title="exceptions.FutureWarning"><code class="xref py py-exc docutils literal">FutureWarning</code></a>. Currently they’re stored as 32-bit numbers and result in a

1912

negative value, but in Python 2.4 they’ll become positive long integers.

1913

There are a few ways to fix this warning. If you really need a positive number,

1914

just add an <code class="docutils literal">L</code> to the end of the literal. If you’re trying to get a 32-bit

1915

integer with low bits set and have previously used an expression such as <code class="docutils literal">~(1

1916

<< 31)</code>, it’s probably clearest to start with all bits set and clear the

1917

desired upper bits. For example, to clear just the top bit (bit 31), you could

1918

write <code class="docutils literal">0xffffffffL &~(1L<<31)</code>.

1919

</li>

1920

<li>You can no longer disable assertions by assigning to <code class="docutils literal">__debug__</code>.

1921

</li>

1922

<li>The Distutils <code class="xref py py-func docutils literal">setup()</code> function has gained various new keyword arguments

1923

such as depends. Old versions of the Distutils will abort if passed unknown

1924

keywords. A solution is to check for the presence of the new

1925

<code class="xref py py-func docutils literal">get_distutil_options()</code> function in your <code class="file docutils literal">setup.py</code> and only uses the

1926

new keywords with a version of the Distutils that supports them:

1927

<div class="highlight-python"><div class="highlight"><pre>from distutils import core

1928

1929

kw = {'sources': 'foo.c', ...}

1930

if hasattr(core, 'get_distutil_options'):

1931

kw['depends'] = ['foo.h']

1932

ext = Extension(**kw)

1933

</pre></div>

1934

</div>

1935

</li>

1936

1937

warning.

1938

</li>

1939

<li>Names of extension types defined by the modules included with Python now

1940

contain the module and a <code class="docutils literal">'.'</code> in front of the type name.

1941

</li>

1942

</ul>

1943

</div>

1944

1945

<h2>Acknowledgements<a class="headerlink" href="#acknowledgements" title="Permalink to this headline">¶</a></h2>

1946

The author would like to thank the following people for offering suggestions,

1947

corrections and assistance with various drafts of this article: Jeff Bauer,

1948

Simon Brunning, Brett Cannon, Michael Chermside, Andrew Dalke, Scott David

1949

Daniels, Fred L. Drake, Jr., David Fraser, Kelly Gerber, Raymond Hettinger,

1950

Michael Hudson, Chris Lambert, Detlef Lannert, Martin von Löwis, Andrew

1951

MacIntyre, Lalo Martins, Chad Netzer, Gustavo Niemeyer, Neal Norwitz, Hans

1952

Nowak, Chris Reedy, Francesco Ricciardi, Vinay Sajip, Neil Schemenauer, Roman

1953

Suzi, Jason Tishler, Just van Rossum.

1954

</div>

1955

</div>

1956

1957

1958

</div>

1959

</div>

1960

</div>

1961

1962

1963

<h3><a href="../contents.html">Table Of Contents</a></h3>

1964

<ul>

1965

<li><a class="reference internal" href="#">What’s New in Python 2.3</a><ul>

1966

<li><a class="reference internal" href="#pep-218-a-standard-set-datatype">PEP 218: A Standard Set Datatype</a></li>

1967

<li><a class="reference internal" href="#pep-255-simple-generators">PEP 255: Simple Generators</a></li>

1968

<li><a class="reference internal" href="#pep-263-source-code-encodings">PEP 263: Source Code Encodings</a></li>

1969

<li><a class="reference internal" href="#pep-273-importing-modules-from-zip-archives">PEP 273: Importing Modules from ZIP Archives</a></li>

1970

<li><a class="reference internal" href="#pep-277-unicode-file-name-support-for-windows-nt">PEP 277: Unicode file name support for Windows NT</a></li>

1971

<li><a class="reference internal" href="#pep-278-universal-newline-support">PEP 278: Universal Newline Support</a></li>

1972

<li><a class="reference internal" href="#pep-279-enumerate">PEP 279: enumerate()</a></li>

1973

<li><a class="reference internal" href="#pep-282-the-logging-package">PEP 282: The logging Package</a></li>

1974

<li><a class="reference internal" href="#pep-285-a-boolean-type">PEP 285: A Boolean Type</a></li>

1975

<li><a class="reference internal" href="#pep-293-codec-error-handling-callbacks">PEP 293: Codec Error Handling Callbacks</a></li>

1976

<li><a class="reference internal" href="#pep-301-package-index-and-metadata-for-distutils">PEP 301: Package Index and Metadata for Distutils</a></li>

1977

<li><a class="reference internal" href="#pep-302-new-import-hooks">PEP 302: New Import Hooks</a></li>

1978

<li><a class="reference internal" href="#pep-305-comma-separated-files">PEP 305: Comma-separated Files</a></li>

1979

<li><a class="reference internal" href="#pep-307-pickle-enhancements">PEP 307: Pickle Enhancements</a></li>

1980

<li><a class="reference internal" href="#extended-slices">Extended Slices</a></li>

1981

<li><a class="reference internal" href="#other-language-changes">Other Language Changes</a><ul>

1982

<li><a class="reference internal" href="#string-changes">String Changes</a></li>

1983

<li><a class="reference internal" href="#optimizations">Optimizations</a></li>

1984

</ul>

1985

</li>

1986

<li><a class="reference internal" href="#new-improved-and-deprecated-modules">New, Improved, and Deprecated Modules</a><ul>

1987

1988

<li><a class="reference internal" href="#the-optparse-module">The optparse Module</a></li>

1989

</ul>

1990

</li>

1991

<li><a class="reference internal" href="#pymalloc-a-specialized-object-allocator">Pymalloc: A Specialized Object Allocator</a></li>

1992

<li><a class="reference internal" href="#build-and-c-api-changes">Build and C API Changes</a><ul>

1993

<li><a class="reference internal" href="#port-specific-changes">Port-Specific Changes</a></li>

1994

</ul>

1995

</li>

1996

<li><a class="reference internal" href="#other-changes-and-fixes">Other Changes and Fixes</a></li>

1997

<li><a class="reference internal" href="#porting-to-python-2-3">Porting to Python 2.3</a></li>

1998

<li><a class="reference internal" href="#acknowledgements">Acknowledgements</a></li>

1999

</ul>

2000

</li>

2001

</ul>

2002

2003

<h4>Previous topic</h4>

2004

<a href="2.4.html"

2005

title="previous chapter">What’s New in Python 2.4</a>

2006

<h4>Next topic</h4>

2007

<a href="2.2.html"

2008

title="next chapter">What’s New in Python 2.2</a>

2009

2010

2011

<li><a href="../bugs.html">Report a Bug</a></li>

2012

<li><a href="../_sources/whatsnew/2.3.txt"

2013

rel="nofollow">Show Source</a></li>

2014

</ul>

2015

2016

2017

<h3>Quick search</h3>

2018

2019

2020

2021

2022

2023

</form>

2024

2025

Enter search terms or a module, class or function name.

2026

2027

</div>

2028

2029

</div>

2030

</div>

2031

2032

</div>

2033

2034

<h3>Navigation</h3>

2035

<ul>

2036

2037

<a href="../genindex.html" title="General Index"

2038

>index</a></li>

2039

2040

<a href="../py-modindex.html" title="Python Module Index"

2041

>modules</a> |</li>

2042

2043

<a href="2.2.html" title="What’s New in Python 2.2"

2044

>next</a> |</li>

2045

2046

<a href="2.4.html" title="What’s New in Python 2.4"

2047

>previous</a> |</li>

2048

<li><img src="../_static/py.png" alt=""

2049

style="vertical-align: middle; margin-top: -1px"/></li>

2050

<li><a href="https://www.python.org/">Python</a> »</li>

2051

<li>

2052

2.7.11

2053

<a href="../index.html">Documentation</a> »

2054

</li>

2055

2056

<li class="nav-item nav-item-1"><a href="index.html" >What’s New in Python</a> »</li>

2057

</ul>

2058

</div>

2059

2060

2061

2062

The Python Software Foundation is a non-profit corporation.

2063

<a href="https://www.python.org/psf/donations/">Please donate.</a>

2064

2065

Last updated on Jan 23, 2016.

2066

<a href="../bugs.html">Found a bug</a>?

2067

2068

Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.3.3.

2069

</div>

2070

2071

</body>

2072

</html>

b'\\ No newline at end of file'

Older »