1
****************************
2
What's New in Python 2.4
3
****************************
7
.. |release| replace:: 1.02
9
.. $Id: whatsnew24.tex 54632 2007-03-31 11:59:54Z georg.brandl $
10
.. Don't write extensive text for new sections; I'll do that.
11
.. Feel free to add commented-out reminders of things that need
12
.. to be covered. --amk
14
This article explains the new features in Python 2.4.1, released on March 30,
17
Python 2.4 is a medium-sized release. It doesn't introduce as many changes as
18
the radical Python 2.2, but introduces more features than the conservative 2.3
19
release. The most significant new language features are function decorators and
20
generator expressions; most other changes are to the standard library.
22
According to the CVS change logs, there were 481 patches applied and 502 bugs
23
fixed between Python 2.3 and 2.4. Both figures are likely to be underestimates.
25
This article doesn't attempt to provide a complete specification of every single
26
new feature, but instead provides a brief introduction to each feature. For
27
full details, you should refer to the documentation for Python 2.4, such as the
28
Python Library Reference and the Python Reference Manual. Often you will be
29
referred to the PEP for a particular new feature for explanations of the
30
implementation and design rationale.
32
.. ======================================================================
35
PEP 218: Built-In Set Objects
36
=============================
38
Python 2.3 introduced the :mod:`sets` module. C implementations of set data
39
types have now been added to the Python core as two new built-in types,
40
:func:`set(iterable)` and :func:`frozenset(iterable)`. They provide high speed
41
operations for membership testing, for eliminating duplicates from sequences,
42
and for mathematical operations like unions, intersections, differences, and
43
symmetric differences. ::
45
>>> a = set('abracadabra') # form a set from a string
46
>>> 'z' in a # fast membership testing
48
>>> a # unique letters in a
49
set(['a', 'r', 'b', 'c', 'd'])
50
>>> ''.join(a) # convert back into a string
53
>>> b = set('alacazam') # form a second set
54
>>> a - b # letters in a but not in b
56
>>> a | b # letters in either a or b
57
set(['a', 'c', 'r', 'd', 'b', 'm', 'z', 'l'])
58
>>> a & b # letters in both a and b
60
>>> a ^ b # letters in a or b but not both
61
set(['r', 'd', 'b', 'm', 'z', 'l'])
63
>>> a.add('z') # add a new element
64
>>> a.update('wxy') # add multiple new elements
66
set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'x', 'z'])
67
>>> a.remove('x') # take one element out
69
set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'z'])
71
The :func:`frozenset` type is an immutable version of :func:`set`. Since it is
72
immutable and hashable, it may be used as a dictionary key or as a member of
75
The :mod:`sets` module remains in the standard library, and may be useful if you
76
wish to subclass the :class:`Set` or :class:`ImmutableSet` classes. There are
77
currently no plans to deprecate the module.
82
:pep:`218` - Adding a Built-In Set Object Type
83
Originally proposed by Greg Wilson and ultimately implemented by Raymond
86
.. ======================================================================
89
PEP 237: Unifying Long Integers and Integers
90
============================================
92
The lengthy transition process for this PEP, begun in Python 2.2, takes another
93
step forward in Python 2.4. In 2.3, certain integer operations that would
94
behave differently after int/long unification triggered :exc:`FutureWarning`
95
warnings and returned values limited to 32 or 64 bits (depending on your
96
platform). In 2.4, these expressions no longer produce a warning and instead
97
produce a different result that's usually a long integer.
99
The problematic expressions are primarily left shifts and lengthy hexadecimal
100
and octal constants. For example, ``2 << 32`` results in a warning in 2.3,
101
evaluating to 0 on 32-bit platforms. In Python 2.4, this expression now returns
102
the correct answer, 8589934592.
107
:pep:`237` - Unifying Long Integers and Integers
108
Original PEP written by Moshe Zadka and GvR. The changes for 2.4 were
109
implemented by Kalle Svensson.
111
.. ======================================================================
114
PEP 289: Generator Expressions
115
==============================
117
The iterator feature introduced in Python 2.2 and the :mod:`itertools` module
118
make it easier to write programs that loop through large data sets without
119
having the entire data set in memory at one time. List comprehensions don't fit
120
into this picture very well because they produce a Python list object containing
121
all of the items. This unavoidably pulls all of the objects into memory, which
122
can be a problem if your data set is very large. When trying to write a
123
functionally-styled program, it would be natural to write something like::
125
links = [link for link in get_all_links() if not link.followed]
131
for link in get_all_links():
136
The first form is more concise and perhaps more readable, but if you're dealing
137
with a large number of link objects you'd have to write the second form to avoid
138
having all link objects in memory at the same time.
140
Generator expressions work similarly to list comprehensions but don't
141
materialize the entire list; instead they create a generator that will return
142
elements one by one. The above example could be written as::
144
links = (link for link in get_all_links() if not link.followed)
148
Generator expressions always have to be written inside parentheses, as in the
149
above example. The parentheses signalling a function call also count, so if you
150
want to create an iterator that will be immediately passed to a function you
153
print sum(obj.count for obj in list_all_objects())
155
Generator expressions differ from list comprehensions in various small ways.
156
Most notably, the loop variable (*obj* in the above example) is not accessible
157
outside of the generator expression. List comprehensions leave the variable
158
assigned to its last value; future versions of Python will change this, making
159
list comprehensions match generator expressions in this respect.
164
:pep:`289` - Generator Expressions
165
Proposed by Raymond Hettinger and implemented by Jiwon Seo with early efforts
166
steered by Hye-Shik Chang.
168
.. ======================================================================
171
PEP 292: Simpler String Substitutions
172
=====================================
174
Some new classes in the standard library provide an alternative mechanism for
175
substituting variables into strings; this style of substitution may be better
176
for applications where untrained users need to edit templates.
178
The usual way of substituting variables by name is the ``%`` operator::
180
>>> '%(page)i: %(title)s' % {'page':2, 'title': 'The Best of Times'}
181
'2: The Best of Times'
183
When writing the template string, it can be easy to forget the ``i`` or ``s``
184
after the closing parenthesis. This isn't a big problem if the template is in a
185
Python module, because you run the code, get an "Unsupported format character"
186
:exc:`ValueError`, and fix the problem. However, consider an application such
187
as Mailman where template strings or translations are being edited by users who
188
aren't aware of the Python language. The format string's syntax is complicated
189
to explain to such users, and if they make a mistake, it's difficult to provide
190
helpful feedback to them.
192
PEP 292 adds a :class:`Template` class to the :mod:`string` module that uses
193
``$`` to indicate a substitution::
196
>>> t = string.Template('$page: $title')
197
>>> t.substitute({'page':2, 'title': 'The Best of Times'})
198
'2: The Best of Times'
200
If a key is missing from the dictionary, the :meth:`substitute` method will
201
raise a :exc:`KeyError`. There's also a :meth:`safe_substitute` method that
202
ignores missing keys::
204
>>> t = string.Template('$page: $title')
205
>>> t.safe_substitute({'page':3})
211
:pep:`292` - Simpler String Substitutions
212
Written and implemented by Barry Warsaw.
214
.. ======================================================================
217
PEP 318: Decorators for Functions and Methods
218
=============================================
220
Python 2.2 extended Python's object model by adding static methods and class
221
methods, but it didn't extend Python's syntax to provide any new way of defining
222
static or class methods. Instead, you had to write a :keyword:`def` statement
223
in the usual way, and pass the resulting method to a :func:`staticmethod` or
224
:func:`classmethod` function that would wrap up the function as a method of the
225
new type. Your code would look like this::
231
meth = classmethod(meth) # Rebind name to wrapped-up class method
233
If the method was very long, it would be easy to miss or forget the
234
:func:`classmethod` invocation after the function body.
236
The intention was always to add some syntax to make such definitions more
237
readable, but at the time of 2.2's release a good syntax was not obvious. Today
238
a good syntax *still* isn't obvious but users are asking for easier access to
239
the feature; a new syntactic feature has been added to meet this need.
241
The new feature is called "function decorators". The name comes from the idea
242
that :func:`classmethod`, :func:`staticmethod`, and friends are storing
243
additional information on a function object; they're *decorating* functions with
246
The notation borrows from Java and uses the ``'@'`` character as an indicator.
247
Using the new syntax, the example above would be written::
256
The ``@classmethod`` is shorthand for the ``meth=classmethod(meth)`` assignment.
257
More generally, if you have the following::
265
It's equivalent to the following pre-decorator code::
270
Decorators must come on the line before a function definition, one decorator per
271
line, and can't be on the same line as the def statement, meaning that ``@A def
272
f(): ...`` is illegal. You can only decorate function definitions, either at
273
the module level or inside a class; you can't decorate class definitions.
275
A decorator is just a function that takes the function to be decorated as an
276
argument and returns either the same function or some new object. The return
277
value of the decorator need not be callable (though it typically is), unless
278
further decorators will be applied to the result. It's easy to write your own
279
decorators. The following simple example just sets an attribute on the function
283
... func.attr = 'decorated'
290
<function f at 0x402ef0d4>
295
As a slightly more realistic example, the following decorator checks that the
296
supplied argument is an integer::
298
def require_int (func):
300
assert isinstance(arg, int)
313
An example in :pep:`318` contains a fancier version of this idea that lets you
314
both specify the required type and check the returned type.
316
Decorator functions can take arguments. If arguments are supplied, your
317
decorator function is called with only those arguments and must return a new
318
decorator function; this function must take a single function and return a
319
function, as previously described. In other words, ``@A @B @C(args)`` becomes::
325
Getting this right can be slightly brain-bending, but it's not too difficult.
327
A small related change makes the :attr:`func_name` attribute of functions
328
writable. This attribute is used to display function names in tracebacks, so
329
decorators should change the name of any new function that's constructed and
335
:pep:`318` - Decorators for Functions, Methods and Classes
336
Written by Kevin D. Smith, Jim Jewett, and Skip Montanaro. Several people
337
wrote patches implementing function decorators, but the one that was actually
338
checked in was patch #979728, written by Mark Russell.
340
http://www.python.org/moin/PythonDecoratorLibrary
341
This Wiki page contains several examples of decorators.
343
.. ======================================================================
346
PEP 322: Reverse Iteration
347
==========================
349
A new built-in function, :func:`reversed(seq)`, takes a sequence and returns an
350
iterator that loops over the elements of the sequence in reverse order. ::
352
>>> for i in reversed(xrange(1,4)):
359
Compared to extended slicing, such as ``range(1,4)[::-1]``, :func:`reversed` is
360
easier to read, runs faster, and uses substantially less memory.
362
Note that :func:`reversed` only accepts sequences, not arbitrary iterators. If
363
you want to reverse an iterator, first convert it to a list with :func:`list`.
366
>>> input = open('/etc/passwd', 'r')
367
>>> for line in reversed(list(input)):
370
root:*:0:0:System Administrator:/var/root:/bin/tcsh
376
:pep:`322` - Reverse Iteration
377
Written and implemented by Raymond Hettinger.
379
.. ======================================================================
382
PEP 324: New subprocess Module
383
==============================
385
The standard library provides a number of ways to execute a subprocess, offering
386
different features and different levels of complexity.
387
:func:`os.system(command)` is easy to use, but slow (it runs a shell process
388
which executes the command) and dangerous (you have to be careful about escaping
389
the shell's metacharacters). The :mod:`popen2` module offers classes that can
390
capture standard output and standard error from the subprocess, but the naming
391
is confusing. The :mod:`subprocess` module cleans this up, providing a unified
392
interface that offers all the features you might need.
394
Instead of :mod:`popen2`'s collection of classes, :mod:`subprocess` contains a
395
single class called :class:`Popen` whose constructor supports a number of
396
different keyword arguments. ::
398
class Popen(args, bufsize=0, executable=None,
399
stdin=None, stdout=None, stderr=None,
400
preexec_fn=None, close_fds=False, shell=False,
401
cwd=None, env=None, universal_newlines=False,
402
startupinfo=None, creationflags=0):
404
*args* is commonly a sequence of strings that will be the arguments to the
405
program executed as the subprocess. (If the *shell* argument is true, *args*
406
can be a string which will then be passed on to the shell for interpretation,
407
just as :func:`os.system` does.)
409
*stdin*, *stdout*, and *stderr* specify what the subprocess's input, output, and
410
error streams will be. You can provide a file object or a file descriptor, or
411
you can use the constant ``subprocess.PIPE`` to create a pipe between the
412
subprocess and the parent.
414
The constructor has a number of handy options:
416
* *close_fds* requests that all file descriptors be closed before running the
419
* *cwd* specifies the working directory in which the subprocess will be executed
420
(defaulting to whatever the parent's working directory is).
422
* *env* is a dictionary specifying environment variables.
424
* *preexec_fn* is a function that gets called before the child is started.
426
* *universal_newlines* opens the child's input and output using Python's
427
universal newline feature.
429
Once you've created the :class:`Popen` instance, you can call its :meth:`wait`
430
method to pause until the subprocess has exited, :meth:`poll` to check if it's
431
exited without pausing, or :meth:`communicate(data)` to send the string *data*
432
to the subprocess's standard input. :meth:`communicate(data)` then reads any
433
data that the subprocess has sent to its standard output or standard error,
434
returning a tuple ``(stdout_data, stderr_data)``.
436
:func:`call` is a shortcut that passes its arguments along to the :class:`Popen`
437
constructor, waits for the command to complete, and returns the status code of
438
the subprocess. It can serve as a safer analog to :func:`os.system`::
440
sts = subprocess.call(['dpkg', '-i', '/tmp/new-package.deb'])
445
# dpkg returned an error
448
The command is invoked without use of the shell. If you really do want to use
449
the shell, you can add ``shell=True`` as a keyword argument and provide a string
450
instead of a sequence::
452
sts = subprocess.call('dpkg -i /tmp/new-package.deb', shell=True)
454
The PEP takes various examples of shell and Python code and shows how they'd be
455
translated into Python code that uses :mod:`subprocess`. Reading this section
456
of the PEP is highly recommended.
461
:pep:`324` - subprocess - New process module
462
Written and implemented by Peter Åstrand, with assistance from Fredrik Lundh and
465
.. ======================================================================
468
PEP 327: Decimal Data Type
469
==========================
471
Python has always supported floating-point (FP) numbers, based on the underlying
472
C :ctype:`double` type, as a data type. However, while most programming
473
languages provide a floating-point type, many people (even programmers) are
474
unaware that floating-point numbers don't represent certain decimal fractions
475
accurately. The new :class:`Decimal` type can represent these fractions
476
accurately, up to a user-specified precision limit.
479
Why is Decimal needed?
480
----------------------
482
The limitations arise from the representation used for floating-point numbers.
483
FP numbers are made up of three components:
485
* The sign, which is positive or negative.
487
* The mantissa, which is a single-digit binary number followed by a fractional
488
part. For example, ``1.01`` in base-2 notation is ``1 + 0/2 + 1/4``, or 1.25 in
491
* The exponent, which tells where the decimal point is located in the number
494
For example, the number 1.25 has positive sign, a mantissa value of 1.01 (in
495
binary), and an exponent of 0 (the decimal point doesn't need to be shifted).
496
The number 5 has the same sign and mantissa, but the exponent is 2 because the
497
mantissa is multiplied by 4 (2 to the power of the exponent 2); 1.25 \* 4 equals
500
Modern systems usually provide floating-point support that conforms to a
501
standard called IEEE 754. C's :ctype:`double` type is usually implemented as a
502
64-bit IEEE 754 number, which uses 52 bits of space for the mantissa. This
503
means that numbers can only be specified to 52 bits of precision. If you're
504
trying to represent numbers whose expansion repeats endlessly, the expansion is
505
cut off after 52 bits. Unfortunately, most software needs to produce output in
506
base 10, and common fractions in base 10 are often repeating decimals in binary.
507
For example, 1.1 decimal is binary ``1.0001100110011 ...``; .1 = 1/16 + 1/32 +
508
1/256 plus an infinite number of additional terms. IEEE 754 has to chop off
509
that infinitely repeated decimal after 52 digits, so the representation is
512
Sometimes you can see this inaccuracy when the number is printed::
517
The inaccuracy isn't always visible when you print the number because the FP-to-
518
decimal-string conversion is provided by the C library, and most C libraries try
519
to produce sensible output. Even if it's not displayed, however, the inaccuracy
520
is still there and subsequent operations can magnify the error.
522
For many applications this doesn't matter. If I'm plotting points and
523
displaying them on my monitor, the difference between 1.1 and 1.1000000000000001
524
is too small to be visible. Reports often limit output to a certain number of
525
decimal places, and if you round the number to two or three or even eight
526
decimal places, the error is never apparent. However, for applications where it
527
does matter, it's a lot of work to implement your own custom arithmetic
530
Hence, the :class:`Decimal` type was created.
533
The :class:`Decimal` type
534
-------------------------
536
A new module, :mod:`decimal`, was added to Python's standard library. It
537
contains two classes, :class:`Decimal` and :class:`Context`. :class:`Decimal`
538
instances represent numbers, and :class:`Context` instances are used to wrap up
539
various settings such as the precision and default rounding mode.
541
:class:`Decimal` instances are immutable, like regular Python integers and FP
542
numbers; once it's been created, you can't change the value an instance
543
represents. :class:`Decimal` instances can be created from integers or
547
>>> decimal.Decimal(1972)
549
>>> decimal.Decimal("1.1")
552
You can also provide tuples containing the sign, the mantissa represented as a
553
tuple of decimal digits, and the exponent::
555
>>> decimal.Decimal((1, (1, 4, 7, 5), -2))
558
Cautionary note: the sign bit is a Boolean value, so 0 is positive and 1 is
561
Converting from floating-point numbers poses a bit of a problem: should the FP
562
number representing 1.1 turn into the decimal number for exactly 1.1, or for 1.1
563
plus whatever inaccuracies are introduced? The decision was to dodge the issue
564
and leave such a conversion out of the API. Instead, you should convert the
565
floating-point number into a string using the desired precision and pass the
566
string to the :class:`Decimal` constructor::
569
>>> decimal.Decimal(str(f))
571
>>> decimal.Decimal('%.12f' % f)
572
Decimal("1.100000000000")
574
Once you have :class:`Decimal` instances, you can perform the usual mathematical
575
operations on them. One limitation: exponentiation requires an integer
578
>>> a = decimal.Decimal('35.72')
579
>>> b = decimal.Decimal('1.73')
587
Decimal("20.64739884393063583815028902")
591
Traceback (most recent call last):
593
decimal.InvalidOperation: x ** (non-integer)
595
You can combine :class:`Decimal` instances with integers, but not with floating-
601
Traceback (most recent call last):
603
TypeError: You can interact Decimal only with int, long or Decimal data types.
606
:class:`Decimal` numbers can be used with the :mod:`math` and :mod:`cmath`
607
modules, but note that they'll be immediately converted to floating-point
608
numbers before the operation is performed, resulting in a possible loss of
609
precision and accuracy. You'll also get back a regular floating-point number
610
and not a :class:`Decimal`. ::
612
>>> import math, cmath
613
>>> d = decimal.Decimal('123456789012.345')
619
:class:`Decimal` instances have a :meth:`sqrt` method that returns a
620
:class:`Decimal`, but if you need other things such as trigonometric functions
621
you'll have to implement them. ::
624
Decimal("351364.1828820134592177245001")
627
The :class:`Context` type
628
-------------------------
630
Instances of the :class:`Context` class encapsulate several settings for
633
* :attr:`prec` is the precision, the number of decimal places.
635
* :attr:`rounding` specifies the rounding mode. The :mod:`decimal` module has
636
constants for the various possibilities: :const:`ROUND_DOWN`,
637
:const:`ROUND_CEILING`, :const:`ROUND_HALF_EVEN`, and various others.
639
* :attr:`traps` is a dictionary specifying what happens on encountering certain
640
error conditions: either an exception is raised or a value is returned. Some
641
examples of error conditions are division by zero, loss of precision, and
644
There's a thread-local default context available by calling :func:`getcontext`;
645
you can change the properties of this context to alter the default precision,
646
rounding, or trap handling. The following example shows the effect of changing
647
the precision of the default context::
649
>>> decimal.getcontext().prec
651
>>> decimal.Decimal(1) / decimal.Decimal(7)
652
Decimal("0.1428571428571428571428571429")
653
>>> decimal.getcontext().prec = 9
654
>>> decimal.Decimal(1) / decimal.Decimal(7)
655
Decimal("0.142857143")
657
The default action for error conditions is selectable; the module can either
658
return a special value such as infinity or not-a-number, or exceptions can be
661
>>> decimal.Decimal(1) / decimal.Decimal(0)
662
Traceback (most recent call last):
664
decimal.DivisionByZero: x / 0
665
>>> decimal.getcontext().traps[decimal.DivisionByZero] = False
666
>>> decimal.Decimal(1) / decimal.Decimal(0)
670
The :class:`Context` instance also has various methods for formatting numbers
671
such as :meth:`to_eng_string` and :meth:`to_sci_string`.
673
For more information, see the documentation for the :mod:`decimal` module, which
674
includes a quick-start tutorial and a reference.
679
:pep:`327` - Decimal Data Type
680
Written by Facundo Batista and implemented by Facundo Batista, Eric Price,
681
Raymond Hettinger, Aahz, and Tim Peters.
683
http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html
684
A more detailed overview of the IEEE-754 representation.
686
http://www.lahey.com/float.htm
687
The article uses Fortran code to illustrate many of the problems that floating-
688
point inaccuracy can cause.
690
http://www2.hursley.ibm.com/decimal/
691
A description of a decimal-based representation. This representation is being
692
proposed as a standard, and underlies the new Python decimal type. Much of this
693
material was written by Mike Cowlishaw, designer of the Rexx language.
695
.. ======================================================================
698
PEP 328: Multi-line Imports
699
===========================
701
One language change is a small syntactic tweak aimed at making it easier to
702
import many names from a module. In a ``from module import names`` statement,
703
*names* is a sequence of names separated by commas. If the sequence is very
704
long, you can either write multiple imports from the same module, or you can use
705
backslashes to escape the line endings like this::
707
from SimpleXMLRPCServer import SimpleXMLRPCServer,\
708
SimpleXMLRPCRequestHandler,\
709
CGIXMLRPCRequestHandler,\
710
resolve_dotted_attribute
712
The syntactic change in Python 2.4 simply allows putting the names within
713
parentheses. Python ignores newlines within a parenthesized expression, so the
714
backslashes are no longer needed::
716
from SimpleXMLRPCServer import (SimpleXMLRPCServer,
717
SimpleXMLRPCRequestHandler,
718
CGIXMLRPCRequestHandler,
719
resolve_dotted_attribute)
721
The PEP also proposes that all :keyword:`import` statements be absolute imports,
722
with a leading ``.`` character to indicate a relative import. This part of the
723
PEP was not implemented for Python 2.4, but was completed for Python 2.5.
728
:pep:`328` - Imports: Multi-Line and Absolute/Relative
729
Written by Aahz. Multi-line imports were implemented by Dima Dorfman.
731
.. ======================================================================
734
PEP 331: Locale-Independent Float/String Conversions
735
====================================================
737
The :mod:`locale` modules lets Python software select various conversions and
738
display conventions that are localized to a particular country or language.
739
However, the module was careful to not change the numeric locale because various
740
functions in Python's implementation required that the numeric locale remain set
741
to the ``'C'`` locale. Often this was because the code was using the C
742
library's :cfunc:`atof` function.
744
Not setting the numeric locale caused trouble for extensions that used third-
745
party C libraries, however, because they wouldn't have the correct locale set.
746
The motivating example was GTK+, whose user interface widgets weren't displaying
747
numbers in the current locale.
749
The solution described in the PEP is to add three new functions to the Python
750
API that perform ASCII-only conversions, ignoring the locale setting:
752
* :cfunc:`PyOS_ascii_strtod(str, ptr)` and :cfunc:`PyOS_ascii_atof(str, ptr)`
753
both convert a string to a C :ctype:`double`.
755
* :cfunc:`PyOS_ascii_formatd(buffer, buf_len, format, d)` converts a
756
:ctype:`double` to an ASCII string.
758
The code for these functions came from the GLib library
759
(http://developer.gnome.org/arch/gtk/glib.html), whose developers kindly
760
relicensed the relevant functions and donated them to the Python Software
761
Foundation. The :mod:`locale` module can now change the numeric locale,
762
letting extensions such as GTK+ produce the correct results.
767
:pep:`331` - Locale-Independent Float/String Conversions
768
Written by Christian R. Reis, and implemented by Gustavo Carneiro.
770
.. ======================================================================
773
Other Language Changes
774
======================
776
Here are all of the changes that Python 2.4 makes to the core Python language.
778
* Decorators for functions and methods were added (:pep:`318`).
780
* Built-in :func:`set` and :func:`frozenset` types were added (:pep:`218`).
781
Other new built-ins include the :func:`reversed(seq)` function (:pep:`322`).
783
* Generator expressions were added (:pep:`289`).
785
* Certain numeric expressions no longer return values restricted to 32 or 64
788
* You can now put parentheses around the list of names in a ``from module import
789
names`` statement (:pep:`328`).
791
* The :meth:`dict.update` method now accepts the same argument forms as the
792
:class:`dict` constructor. This includes any mapping, any iterable of key/value
793
pairs, and keyword arguments. (Contributed by Raymond Hettinger.)
795
* The string methods :meth:`ljust`, :meth:`rjust`, and :meth:`center` now take
796
an optional argument for specifying a fill character other than a space.
797
(Contributed by Raymond Hettinger.)
799
* Strings also gained an :meth:`rsplit` method that works like the :meth:`split`
800
method but splits from the end of the string. (Contributed by Sean
803
>>> 'www.python.org'.split('.', 1)
804
['www', 'python.org']
805
'www.python.org'.rsplit('.', 1)
806
['www.python', 'org']
808
* Three keyword parameters, *cmp*, *key*, and *reverse*, were added to the
809
:meth:`sort` method of lists. These parameters make some common usages of
810
:meth:`sort` simpler. All of these parameters are optional.
812
For the *cmp* parameter, the value should be a comparison function that takes
813
two parameters and returns -1, 0, or +1 depending on how the parameters compare.
814
This function will then be used to sort the list. Previously this was the only
815
parameter that could be provided to :meth:`sort`.
817
*key* should be a single-parameter function that takes a list element and
818
returns a comparison key for the element. The list is then sorted using the
819
comparison keys. The following example sorts a list case-insensitively::
821
>>> L = ['A', 'b', 'c', 'D']
822
>>> L.sort() # Case-sensitive sort
825
>>> # Using 'key' parameter to sort list
826
>>> L.sort(key=lambda x: x.lower())
829
>>> # Old-fashioned way
830
>>> L.sort(cmp=lambda x,y: cmp(x.lower(), y.lower()))
834
The last example, which uses the *cmp* parameter, is the old way to perform a
835
case-insensitive sort. It works but is slower than using a *key* parameter.
836
Using *key* calls :meth:`lower` method once for each element in the list while
837
using *cmp* will call it twice for each comparison, so using *key* saves on
838
invocations of the :meth:`lower` method.
840
For simple key functions and comparison functions, it is often possible to avoid
841
a :keyword:`lambda` expression by using an unbound method instead. For example,
842
the above case-insensitive sort is best written as::
844
>>> L.sort(key=str.lower)
848
Finally, the *reverse* parameter takes a Boolean value. If the value is true,
849
the list will be sorted into reverse order. Instead of ``L.sort() ;
850
L.reverse()``, you can now write ``L.sort(reverse=True)``.
852
The results of sorting are now guaranteed to be stable. This means that two
853
entries with equal keys will be returned in the same order as they were input.
854
For example, you can sort a list of people by name, and then sort the list by
855
age, resulting in a list sorted by age where people with the same age are in
858
(All changes to :meth:`sort` contributed by Raymond Hettinger.)
860
* There is a new built-in function :func:`sorted(iterable)` that works like the
861
in-place :meth:`list.sort` method but can be used in expressions. The
864
* the input may be any iterable;
866
* a newly formed copy is sorted, leaving the original intact; and
868
* the expression returns the new sorted copy
872
>>> L = [9,7,8,3,2,4,1,6,5]
873
>>> [10+i for i in sorted(L)] # usable in a list comprehension
874
[11, 12, 13, 14, 15, 16, 17, 18, 19]
875
>>> L # original is left unchanged
877
>>> sorted('Monty Python') # any iterable may be an input
878
[' ', 'M', 'P', 'h', 'n', 'n', 'o', 'o', 't', 't', 'y', 'y']
880
>>> # List the contents of a dict sorted by key values
881
>>> colormap = dict(red=1, blue=2, green=3, black=4, yellow=5)
882
>>> for k, v in sorted(colormap.iteritems()):
891
(Contributed by Raymond Hettinger.)
893
* Integer operations will no longer trigger an :exc:`OverflowWarning`. The
894
:exc:`OverflowWarning` warning will disappear in Python 2.5.
896
* The interpreter gained a new switch, :option:`-m`, that takes a name, searches
897
for the corresponding module on ``sys.path``, and runs the module as a script.
898
For example, you can now run the Python profiler with ``python -m profile``.
899
(Contributed by Nick Coghlan.)
901
* The :func:`eval(expr, globals, locals)` and :func:`execfile(filename, globals,
902
locals)` functions and the :keyword:`exec` statement now accept any mapping type
903
for the *locals* parameter. Previously this had to be a regular Python
904
dictionary. (Contributed by Raymond Hettinger.)
906
* The :func:`zip` built-in function and :func:`itertools.izip` now return an
907
empty list if called with no arguments. Previously they raised a
908
:exc:`TypeError` exception. This makes them more suitable for use with variable
909
length argument lists::
911
>>> def transpose(array):
912
... return zip(*array)
914
>>> transpose([(1,2,3), (4,5,6)])
915
[(1, 4), (2, 5), (3, 6)]
919
(Contributed by Raymond Hettinger.)
921
* Encountering a failure while importing a module no longer leaves a partially-
922
initialized module object in ``sys.modules``. The incomplete module object left
923
behind would fool further imports of the same module into succeeding, leading to
924
confusing errors. (Fixed by Tim Peters.)
926
* :const:`None` is now a constant; code that binds a new value to the name
927
``None`` is now a syntax error. (Contributed by Raymond Hettinger.)
929
.. ======================================================================
935
* The inner loops for list and tuple slicing were optimized and now run about
936
one-third faster. The inner loops for dictionaries were also optimized,
937
resulting in performance boosts for :meth:`keys`, :meth:`values`, :meth:`items`,
938
:meth:`iterkeys`, :meth:`itervalues`, and :meth:`iteritems`. (Contributed by
941
* The machinery for growing and shrinking lists was optimized for speed and for
942
space efficiency. Appending and popping from lists now runs faster due to more
943
efficient code paths and less frequent use of the underlying system
944
:cfunc:`realloc`. List comprehensions also benefit. :meth:`list.extend` was
945
also optimized and no longer converts its argument into a temporary list before
946
extending the base list. (Contributed by Raymond Hettinger.)
948
* :func:`list`, :func:`tuple`, :func:`map`, :func:`filter`, and :func:`zip` now
949
run several times faster with non-sequence arguments that supply a
950
:meth:`__len__` method. (Contributed by Raymond Hettinger.)
952
* The methods :meth:`list.__getitem__`, :meth:`dict.__getitem__`, and
953
:meth:`dict.__contains__` are are now implemented as :class:`method_descriptor`
954
objects rather than :class:`wrapper_descriptor` objects. This form of access
955
doubles their performance and makes them more suitable for use as arguments to
956
functionals: ``map(mydict.__getitem__, keylist)``. (Contributed by Raymond
959
* Added a new opcode, ``LIST_APPEND``, that simplifies the generated bytecode
960
for list comprehensions and speeds them up by about a third. (Contributed by
963
* The peephole bytecode optimizer has been improved to produce shorter, faster
964
bytecode; remarkably, the resulting bytecode is more readable. (Enhanced by
967
* String concatenations in statements of the form ``s = s + "abc"`` and ``s +=
968
"abc"`` are now performed more efficiently in certain circumstances. This
969
optimization won't be present in other Python implementations such as Jython, so
970
you shouldn't rely on it; using the :meth:`join` method of strings is still
971
recommended when you want to efficiently glue a large number of strings
972
together. (Contributed by Armin Rigo.)
974
The net result of the 2.4 optimizations is that Python 2.4 runs the pystone
975
benchmark around 5% faster than Python 2.3 and 35% faster than Python 2.2.
976
(pystone is not a particularly good benchmark, but it's the most commonly used
977
measurement of Python's performance. Your own applications may show greater or
978
smaller benefits from Python 2.4.)
980
.. pystone is almost useless for comparing different versions of Python;
981
instead, it excels at predicting relative Python performance on different
982
machines. So, this section would be more informative if it used other tools
983
such as pybench and parrotbench. For a more application oriented benchmark,
984
try comparing the timings of test_decimal.py under 2.3 and 2.4.
986
.. ======================================================================
989
New, Improved, and Deprecated Modules
990
=====================================
992
As usual, Python's standard library received a number of enhancements and bug
993
fixes. Here's a partial list of the most notable changes, sorted alphabetically
994
by module name. Consult the :file:`Misc/NEWS` file in the source tree for a more
995
complete list of changes, or look through the CVS logs for all the details.
997
* The :mod:`asyncore` module's :func:`loop` function now has a *count* parameter
998
that lets you perform a limited number of passes through the polling loop. The
999
default is still to loop forever.
1001
* The :mod:`base64` module now has more complete RFC 3548 support for Base64,
1002
Base32, and Base16 encoding and decoding, including optional case folding and
1003
optional alternative alphabets. (Contributed by Barry Warsaw.)
1005
* The :mod:`bisect` module now has an underlying C implementation for improved
1006
performance. (Contributed by Dmitry Vasiliev.)
1008
* The CJKCodecs collections of East Asian codecs, maintained by Hye-Shik Chang,
1009
was integrated into 2.4. The new encodings are:
1011
* Chinese (PRC): gb2312, gbk, gb18030, big5hkscs, hz
1013
* Chinese (ROC): big5, cp950
1015
* Japanese: cp932, euc-jis-2004, euc-jp, euc-jisx0213, iso-2022-jp,
1016
iso-2022-jp-1, iso-2022-jp-2, iso-2022-jp-3, iso-2022-jp-ext, iso-2022-jp-2004,
1017
shift-jis, shift-jisx0213, shift-jis-2004
1019
* Korean: cp949, euc-kr, johab, iso-2022-kr
1021
* Some other new encodings were added: HP Roman8, ISO_8859-11, ISO_8859-16,
1022
PCTP-154, and TIS-620.
1024
* The UTF-8 and UTF-16 codecs now cope better with receiving partial input.
1025
Previously the :class:`StreamReader` class would try to read more data, making
1026
it impossible to resume decoding from the stream. The :meth:`read` method will
1027
now return as much data as it can and future calls will resume decoding where
1028
previous ones left off. (Implemented by Walter Dörwald.)
1030
* There is a new :mod:`collections` module for various specialized collection
1031
datatypes. Currently it contains just one type, :class:`deque`, a double-
1032
ended queue that supports efficiently adding and removing elements from either
1035
>>> from collections import deque
1036
>>> d = deque('ghi') # make a new deque with three items
1037
>>> d.append('j') # add a new entry to the right side
1038
>>> d.appendleft('f') # add a new entry to the left side
1039
>>> d # show the representation of the deque
1040
deque(['f', 'g', 'h', 'i', 'j'])
1041
>>> d.pop() # return and remove the rightmost item
1043
>>> d.popleft() # return and remove the leftmost item
1045
>>> list(d) # list the contents of the deque
1047
>>> 'h' in d # search the deque
1050
Several modules, such as the :mod:`Queue` and :mod:`threading` modules, now take
1051
advantage of :class:`collections.deque` for improved performance. (Contributed
1052
by Raymond Hettinger.)
1054
* The :mod:`ConfigParser` classes have been enhanced slightly. The :meth:`read`
1055
method now returns a list of the files that were successfully parsed, and the
1056
:meth:`set` method raises :exc:`TypeError` if passed a *value* argument that
1057
isn't a string. (Contributed by John Belmonte and David Goodger.)
1059
* The :mod:`curses` module now supports the ncurses extension
1060
:func:`use_default_colors`. On platforms where the terminal supports
1061
transparency, this makes it possible to use a transparent background.
1062
(Contributed by Jörg Lehmann.)
1064
* The :mod:`difflib` module now includes an :class:`HtmlDiff` class that creates
1065
an HTML table showing a side by side comparison of two versions of a text.
1066
(Contributed by Dan Gass.)
1068
* The :mod:`email` package was updated to version 3.0, which dropped various
1069
deprecated APIs and removes support for Python versions earlier than 2.3. The
1070
3.0 version of the package uses a new incremental parser for MIME messages,
1071
available in the :mod:`email.FeedParser` module. The new parser doesn't require
1072
reading the entire message into memory, and doesn't throw exceptions if a
1073
message is malformed; instead it records any problems in the :attr:`defect`
1074
attribute of the message. (Developed by Anthony Baxter, Barry Warsaw, Thomas
1075
Wouters, and others.)
1077
* The :mod:`heapq` module has been converted to C. The resulting tenfold
1078
improvement in speed makes the module suitable for handling high volumes of
1079
data. In addition, the module has two new functions :func:`nlargest` and
1080
:func:`nsmallest` that use heaps to find the N largest or smallest values in a
1081
dataset without the expense of a full sort. (Contributed by Raymond Hettinger.)
1083
* The :mod:`httplib` module now contains constants for HTTP status codes defined
1084
in various HTTP-related RFC documents. Constants have names such as
1085
:const:`OK`, :const:`CREATED`, :const:`CONTINUE`, and
1086
:const:`MOVED_PERMANENTLY`; use pydoc to get a full list. (Contributed by
1089
* The :mod:`imaplib` module now supports IMAP's THREAD command (contributed by
1090
Yves Dionne) and new :meth:`deleteacl` and :meth:`myrights` methods (contributed
1093
* The :mod:`itertools` module gained a :func:`groupby(iterable[, *func*])`
1094
function. *iterable* is something that can be iterated over to return a stream
1095
of elements, and the optional *func* parameter is a function that takes an
1096
element and returns a key value; if omitted, the key is simply the element
1097
itself. :func:`groupby` then groups the elements into subsequences which have
1098
matching values of the key, and returns a series of 2-tuples containing the key
1099
value and an iterator over the subsequence.
1101
Here's an example to make this clearer. The *key* function simply returns
1102
whether a number is even or odd, so the result of :func:`groupby` is to return
1103
consecutive runs of odd or even numbers. ::
1105
>>> import itertools
1106
>>> L = [2, 4, 6, 7, 8, 9, 11, 12, 14]
1107
>>> for key_val, it in itertools.groupby(L, lambda x: x % 2):
1108
... print key_val, list(it)
1117
:func:`groupby` is typically used with sorted input. The logic for
1118
:func:`groupby` is similar to the Unix ``uniq`` filter which makes it handy for
1119
eliminating, counting, or identifying duplicate elements::
1121
>>> word = 'abracadabra'
1122
>>> letters = sorted(word) # Turn string into a sorted list of letters
1124
['a', 'a', 'a', 'a', 'a', 'b', 'b', 'c', 'd', 'r', 'r']
1125
>>> for k, g in itertools.groupby(letters):
1126
... print k, list(g)
1128
a ['a', 'a', 'a', 'a', 'a']
1133
>>> # List unique letters
1134
>>> [k for k, g in groupby(letters)]
1135
['a', 'b', 'c', 'd', 'r']
1136
>>> # Count letter occurrences
1137
>>> [(k, len(list(g))) for k, g in groupby(letters)]
1138
[('a', 5), ('b', 2), ('c', 1), ('d', 1), ('r', 2)]
1140
(Contributed by Hye-Shik Chang.)
1142
* :mod:`itertools` also gained a function named :func:`tee(iterator, N)` that
1143
returns *N* independent iterators that replicate *iterator*. If *N* is omitted,
1144
the default is 2. ::
1147
>>> i1, i2 = itertools.tee(L)
1149
(<itertools.tee object at 0x402c2080>, <itertools.tee object at 0x402c2090>)
1150
>>> list(i1) # Run the first iterator to exhaustion
1152
>>> list(i2) # Run the second iterator to exhaustion
1155
Note that :func:`tee` has to keep copies of the values returned by the
1156
iterator; in the worst case, it may need to keep all of them. This should
1157
therefore be used carefully if the leading iterator can run far ahead of the
1158
trailing iterator in a long stream of inputs. If the separation is large, then
1159
you might as well use :func:`list` instead. When the iterators track closely
1160
with one another, :func:`tee` is ideal. Possible applications include
1161
bookmarking, windowing, or lookahead iterators. (Contributed by Raymond
1164
* A number of functions were added to the :mod:`locale` module, such as
1165
:func:`bind_textdomain_codeset` to specify a particular encoding and a family of
1166
:func:`l\*gettext` functions that return messages in the chosen encoding.
1167
(Contributed by Gustavo Niemeyer.)
1169
* Some keyword arguments were added to the :mod:`logging` package's
1170
:func:`basicConfig` function to simplify log configuration. The default
1171
behavior is to log messages to standard error, but various keyword arguments can
1172
be specified to log to a particular file, change the logging format, or set the
1173
logging level. For example::
1176
logging.basicConfig(filename='/var/log/application.log',
1177
level=0, # Log all messages
1178
format='%(levelname):%(process):%(thread):%(message)')
1180
Other additions to the :mod:`logging` package include a :meth:`log(level, msg)`
1181
convenience method, as well as a :class:`TimedRotatingFileHandler` class that
1182
rotates its log files at a timed interval. The module already had
1183
:class:`RotatingFileHandler`, which rotated logs once the file exceeded a
1184
certain size. Both classes derive from a new :class:`BaseRotatingHandler` class
1185
that can be used to implement other rotating handlers.
1187
(Changes implemented by Vinay Sajip.)
1189
* The :mod:`marshal` module now shares interned strings on unpacking a data
1190
structure. This may shrink the size of certain pickle strings, but the primary
1191
effect is to make :file:`.pyc` files significantly smaller. (Contributed by
1194
* The :mod:`nntplib` module's :class:`NNTP` class gained :meth:`description` and
1195
:meth:`descriptions` methods to retrieve newsgroup descriptions for a single
1196
group or for a range of groups. (Contributed by Jürgen A. Erhard.)
1198
* Two new functions were added to the :mod:`operator` module,
1199
:func:`attrgetter(attr)` and :func:`itemgetter(index)`. Both functions return
1200
callables that take a single argument and return the corresponding attribute or
1201
item; these callables make excellent data extractors when used with :func:`map`
1202
or :func:`sorted`. For example::
1204
>>> L = [('c', 2), ('d', 1), ('a', 4), ('b', 3)]
1205
>>> map(operator.itemgetter(0), L)
1206
['c', 'd', 'a', 'b']
1207
>>> map(operator.itemgetter(1), L)
1209
>>> sorted(L, key=operator.itemgetter(1)) # Sort list by second tuple item
1210
[('d', 1), ('c', 2), ('b', 3), ('a', 4)]
1212
(Contributed by Raymond Hettinger.)
1214
* The :mod:`optparse` module was updated in various ways. The module now passes
1215
its messages through :func:`gettext.gettext`, making it possible to
1216
internationalize Optik's help and error messages. Help messages for options can
1217
now include the string ``'%default'``, which will be replaced by the option's
1218
default value. (Contributed by Greg Ward.)
1220
* The long-term plan is to deprecate the :mod:`rfc822` module in some future
1221
Python release in favor of the :mod:`email` package. To this end, the
1222
:func:`email.Utils.formatdate` function has been changed to make it usable as a
1223
replacement for :func:`rfc822.formatdate`. You may want to write new e-mail
1224
processing code with this in mind. (Change implemented by Anthony Baxter.)
1226
* A new :func:`urandom(n)` function was added to the :mod:`os` module, returning
1227
a string containing *n* bytes of random data. This function provides access to
1228
platform-specific sources of randomness such as :file:`/dev/urandom` on Linux or
1229
the Windows CryptoAPI. (Contributed by Trevor Perrin.)
1231
* Another new function: :func:`os.path.lexists(path)` returns true if the file
1232
specified by *path* exists, whether or not it's a symbolic link. This differs
1233
from the existing :func:`os.path.exists(path)` function, which returns false if
1234
*path* is a symlink that points to a destination that doesn't exist.
1235
(Contributed by Beni Cherniavsky.)
1237
* A new :func:`getsid` function was added to the :mod:`posix` module that
1238
underlies the :mod:`os` module. (Contributed by J. Raynor.)
1240
* The :mod:`poplib` module now supports POP over SSL. (Contributed by Hector
1243
* The :mod:`profile` module can now profile C extension functions. (Contributed
1246
* The :mod:`random` module has a new method called :meth:`getrandbits(N)` that
1247
returns a long integer *N* bits in length. The existing :meth:`randrange`
1248
method now uses :meth:`getrandbits` where appropriate, making generation of
1249
arbitrarily large random numbers more efficient. (Contributed by Raymond
1252
* The regular expression language accepted by the :mod:`re` module was extended
1253
with simple conditional expressions, written as ``(?(group)A|B)``. *group* is
1254
either a numeric group ID or a group name defined with ``(?P<group>...)``
1255
earlier in the expression. If the specified group matched, the regular
1256
expression pattern *A* will be tested against the string; if the group didn't
1257
match, the pattern *B* will be used instead. (Contributed by Gustavo Niemeyer.)
1259
* The :mod:`re` module is also no longer recursive, thanks to a massive amount
1260
of work by Gustavo Niemeyer. In a recursive regular expression engine, certain
1261
patterns result in a large amount of C stack space being consumed, and it was
1262
possible to overflow the stack. For example, if you matched a 30000-byte string
1263
of ``a`` characters against the expression ``(a|b)+``, one stack frame was
1264
consumed per character. Python 2.3 tried to check for stack overflow and raise
1265
a :exc:`RuntimeError` exception, but certain patterns could sidestep the
1266
checking and if you were unlucky Python could segfault. Python 2.4's regular
1267
expression engine can match this pattern without problems.
1269
* The :mod:`signal` module now performs tighter error-checking on the parameters
1270
to the :func:`signal.signal` function. For example, you can't set a handler on
1271
the :const:`SIGKILL` signal; previous versions of Python would quietly accept
1272
this, but 2.4 will raise a :exc:`RuntimeError` exception.
1274
* Two new functions were added to the :mod:`socket` module. :func:`socketpair`
1275
returns a pair of connected sockets and :func:`getservbyport(port)` looks up the
1276
service name for a given port number. (Contributed by Dave Cole and Barry
1279
* The :func:`sys.exitfunc` function has been deprecated. Code should be using
1280
the existing :mod:`atexit` module, which correctly handles calling multiple exit
1281
functions. Eventually :func:`sys.exitfunc` will become a purely internal
1282
interface, accessed only by :mod:`atexit`.
1284
* The :mod:`tarfile` module now generates GNU-format tar files by default.
1285
(Contributed by Lars Gustaebel.)
1287
* The :mod:`threading` module now has an elegantly simple way to support
1288
thread-local data. The module contains a :class:`local` class whose attribute
1289
values are local to different threads. ::
1293
data = threading.local()
1295
data.url = ('www.python.org', 80)
1297
Other threads can assign and retrieve their own values for the :attr:`number`
1298
and :attr:`url` attributes. You can subclass :class:`local` to initialize
1299
attributes or to add methods. (Contributed by Jim Fulton.)
1301
* The :mod:`timeit` module now automatically disables periodic garbage
1302
collection during the timing loop. This change makes consecutive timings more
1303
comparable. (Contributed by Raymond Hettinger.)
1305
* The :mod:`weakref` module now supports a wider variety of objects including
1306
Python functions, class instances, sets, frozensets, deques, arrays, files,
1307
sockets, and regular expression pattern objects. (Contributed by Raymond
1310
* The :mod:`xmlrpclib` module now supports a multi-call extension for
1311
transmitting multiple XML-RPC calls in a single HTTP operation. (Contributed by
1314
* The :mod:`mpz`, :mod:`rotor`, and :mod:`xreadlines` modules have been
1317
.. ======================================================================
1318
.. whole new modules get described in subsections here
1319
.. =====================
1325
The :mod:`cookielib` library supports client-side handling for HTTP cookies,
1326
mirroring the :mod:`Cookie` module's server-side cookie support. Cookies are
1327
stored in cookie jars; the library transparently stores cookies offered by the
1328
web server in the cookie jar, and fetches the cookie from the jar when
1329
connecting to the server. As in web browsers, policy objects control whether
1330
cookies are accepted or not.
1332
In order to store cookies across sessions, two implementations of cookie jars
1333
are provided: one that stores cookies in the Netscape format so applications can
1334
use the Mozilla or Lynx cookie files, and one that stores cookies in the same
1335
format as the Perl libwww library.
1337
:mod:`urllib2` has been changed to interact with :mod:`cookielib`:
1338
:class:`HTTPCookieProcessor` manages a cookie jar that is used when accessing
1341
This module was contributed by John J. Lee.
1343
.. ==================
1349
The :mod:`doctest` module underwent considerable refactoring thanks to Edward
1350
Loper and Tim Peters. Testing can still be as simple as running
1351
:func:`doctest.testmod`, but the refactorings allow customizing the module's
1352
operation in various ways
1354
The new :class:`DocTestFinder` class extracts the tests from a given object's
1365
finder = doctest.DocTestFinder()
1367
# Get list of DocTest instances
1368
tests = finder.find(f)
1370
The new :class:`DocTestRunner` class then runs individual tests and can produce
1371
a summary of the results::
1373
runner = doctest.DocTestRunner()
1375
tried, failed = runner.run(t)
1377
runner.summarize(verbose=1)
1379
The above example produces the following output::
1381
1 items passed all tests:
1384
2 passed and 0 failed.
1387
:class:`DocTestRunner` uses an instance of the :class:`OutputChecker` class to
1388
compare the expected output with the actual output. This class takes a number
1389
of different flags that customize its behaviour; ambitious users can also write
1390
a completely new subclass of :class:`OutputChecker`.
1392
The default output checker provides a number of handy features. For example,
1393
with the :const:`doctest.ELLIPSIS` option flag, an ellipsis (``...``) in the
1394
expected output matches any substring, making it easier to accommodate outputs
1395
that vary in minor ways::
1399
<__main__.C instance at 0x...>
1403
Another special string, ``<BLANKLINE>``, matches a blank line::
1411
Another new capability is producing a diff-style display of the output by
1412
specifying the :const:`doctest.REPORT_UDIFF` (unified diffs),
1413
:const:`doctest.REPORT_CDIFF` (context diffs), or :const:`doctest.REPORT_NDIFF`
1414
(delta-style) option flags. For example::
1423
L = 'here is a rather lengthy list of words'.split()
1427
Running the above function's tests with :const:`doctest.REPORT_UDIFF` specified,
1428
you get the following output::
1430
**********************************************************************
1431
File ``t.py'', line 15, in g
1434
Differences (unified diff with -expected +actual):
1440
**********************************************************************
1442
.. ======================================================================
1445
Build and C API Changes
1446
=======================
1448
Some of the changes to Python's build process and to the C API are:
1450
* Three new convenience macros were added for common return values from
1451
extension functions: :cmacro:`Py_RETURN_NONE`, :cmacro:`Py_RETURN_TRUE`, and
1452
:cmacro:`Py_RETURN_FALSE`. (Contributed by Brett Cannon.)
1454
* Another new macro, :cmacro:`Py_CLEAR(obj)`, decreases the reference count of
1455
*obj* and sets *obj* to the null pointer. (Contributed by Jim Fulton.)
1457
* A new function, :cfunc:`PyTuple_Pack(N, obj1, obj2, ..., objN)`, constructs
1458
tuples from a variable length argument list of Python objects. (Contributed by
1461
* A new function, :cfunc:`PyDict_Contains(d, k)`, implements fast dictionary
1462
lookups without masking exceptions raised during the look-up process.
1463
(Contributed by Raymond Hettinger.)
1465
* The :cmacro:`Py_IS_NAN(X)` macro returns 1 if its float or double argument
1466
*X* is a NaN. (Contributed by Tim Peters.)
1468
* C code can avoid unnecessary locking by using the new
1469
:cfunc:`PyEval_ThreadsInitialized` function to tell if any thread operations
1470
have been performed. If this function returns false, no lock operations are
1471
needed. (Contributed by Nick Coghlan.)
1473
* A new function, :cfunc:`PyArg_VaParseTupleAndKeywords`, is the same as
1474
:cfunc:`PyArg_ParseTupleAndKeywords` but takes a :ctype:`va_list` instead of a
1475
number of arguments. (Contributed by Greg Chapman.)
1477
* A new method flag, :const:`METH_COEXISTS`, allows a function defined in slots
1478
to co-exist with a :ctype:`PyCFunction` having the same name. This can halve
1479
the access time for a method such as :meth:`set.__contains__`. (Contributed by
1482
* Python can now be built with additional profiling for the interpreter itself,
1483
intended as an aid to people developing the Python core. Providing
1484
:option:`----enable-profiling` to the :program:`configure` script will let you
1485
profile the interpreter with :program:`gprof`, and providing the
1486
:option:`----with-tsc` switch enables profiling using the Pentium's Time-Stamp-
1487
Counter register. Note that the :option:`----with-tsc` switch is slightly
1488
misnamed, because the profiling feature also works on the PowerPC platform,
1489
though that processor architecture doesn't call that register "the TSC
1490
register". (Contributed by Jeremy Hylton.)
1492
* The :ctype:`tracebackobject` type has been renamed to
1493
:ctype:`PyTracebackObject`.
1495
.. ======================================================================
1498
Port-Specific Changes
1499
---------------------
1501
* The Windows port now builds under MSVC++ 7.1 as well as version 6.
1502
(Contributed by Martin von Löwis.)
1504
.. ======================================================================
1507
Porting to Python 2.4
1508
=====================
1510
This section lists previously described changes that may require changes to your
1513
* Left shifts and hexadecimal/octal constants that are too large no longer
1514
trigger a :exc:`FutureWarning` and return a value limited to 32 or 64 bits;
1515
instead they return a long integer.
1517
* Integer operations will no longer trigger an :exc:`OverflowWarning`. The
1518
:exc:`OverflowWarning` warning will disappear in Python 2.5.
1520
* The :func:`zip` built-in function and :func:`itertools.izip` now return an
1521
empty list instead of raising a :exc:`TypeError` exception if called with no
1524
* You can no longer compare the :class:`date` and :class:`datetime` instances
1525
provided by the :mod:`datetime` module. Two instances of different classes
1526
will now always be unequal, and relative comparisons (``<``, ``>``) will raise
1529
* :func:`dircache.listdir` now passes exceptions to the caller instead of
1530
returning empty lists.
1532
* :func:`LexicalHandler.startDTD` used to receive the public and system IDs in
1533
the wrong order. This has been corrected; applications relying on the wrong
1534
order need to be fixed.
1536
* :func:`fcntl.ioctl` now warns if the *mutate* argument is omitted and
1539
* The :mod:`tarfile` module now generates GNU-format tar files by default.
1541
* Encountering a failure while importing a module no longer leaves a partially-
1542
initialized module object in ``sys.modules``.
1544
* :const:`None` is now a constant; code that binds a new value to the name
1545
``None`` is now a syntax error.
1547
* The :func:`signals.signal` function now raises a :exc:`RuntimeError` exception
1548
for certain illegal values; previously these errors would pass silently. For
1549
example, you can no longer set a handler on the :const:`SIGKILL` signal.
1551
.. ======================================================================
1559
The author would like to thank the following people for offering suggestions,
1560
corrections and assistance with various drafts of this article: Koray Can, Hye-
1561
Shik Chang, Michael Dyck, Raymond Hettinger, Brian Hurt, Hamish Lawson, Fredrik
1562
Lundh, Sean Reifschneider, Sadruddin Rejeb.
added
removed
Doc/whatsnew/2.4.rst
