3
*********************************************
4
Brief Tour of the Standard Library -- Part II
5
*********************************************
7
This second tour covers more advanced modules that support professional
8
programming needs. These modules rarely occur in small scripts.
11
.. _tut-output-formatting:
16
The :mod:`reprlib` module provides a version of :func:`repr` customized for
17
abbreviated displays of large or deeply nested containers::
20
>>> reprlib.repr(set('supercalifragilisticexpialidocious'))
21
"{'a', 'c', 'd', 'e', 'f', 'g', ...}"
23
The :mod:`pprint` module offers more sophisticated control over printing both
24
built-in and user defined objects in a way that is readable by the interpreter.
25
When the result is longer than one line, the "pretty printer" adds line breaks
26
and indentation to more clearly reveal data structure::
29
>>> t = [[[['black', 'cyan'], 'white', ['green', 'red']], [['magenta',
30
... 'yellow'], 'blue']]]
32
>>> pprint.pprint(t, width=30)
36
[['magenta', 'yellow'],
39
The :mod:`textwrap` module formats paragraphs of text to fit a given screen
43
>>> doc = """The wrap() method is just like fill() except that it returns
44
... a list of strings instead of one big string with newlines to separate
45
... the wrapped lines."""
47
>>> print(textwrap.fill(doc, width=40))
48
The wrap() method is just like fill()
49
except that it returns a list of strings
50
instead of one big string with newlines
51
to separate the wrapped lines.
53
The :mod:`locale` module accesses a database of culture specific data formats.
54
The grouping attribute of locale's format function provides a direct way of
55
formatting numbers with group separators::
58
>>> locale.setlocale(locale.LC_ALL, 'English_United States.1252')
59
'English_United States.1252'
60
>>> conv = locale.localeconv() # get a mapping of conventions
62
>>> locale.format("%d", x, grouping=True)
64
>>> locale.format_string("%s%.*f", (conv['currency_symbol'],
65
... conv['frac_digits'], x), grouping=True)
74
The :mod:`string` module includes a versatile :class:`~string.Template` class
75
with a simplified syntax suitable for editing by end-users. This allows users
76
to customize their applications without having to alter the application.
78
The format uses placeholder names formed by ``$`` with valid Python identifiers
79
(alphanumeric characters and underscores). Surrounding the placeholder with
80
braces allows it to be followed by more alphanumeric letters with no intervening
81
spaces. Writing ``$$`` creates a single escaped ``$``::
83
>>> from string import Template
84
>>> t = Template('${village}folk send $$10 to $cause.')
85
>>> t.substitute(village='Nottingham', cause='the ditch fund')
86
'Nottinghamfolk send $10 to the ditch fund.'
88
The :meth:`~string.Template.substitute` method raises a :exc:`KeyError` when a
89
placeholder is not supplied in a dictionary or a keyword argument. For
90
mail-merge style applications, user supplied data may be incomplete and the
91
:meth:`~string.Template.safe_substitute` method may be more appropriate ---
92
it will leave placeholders unchanged if data is missing::
94
>>> t = Template('Return the $item to $owner.')
95
>>> d = dict(item='unladen swallow')
97
Traceback (most recent call last):
100
>>> t.safe_substitute(d)
101
'Return the unladen swallow to $owner.'
103
Template subclasses can specify a custom delimiter. For example, a batch
104
renaming utility for a photo browser may elect to use percent signs for
105
placeholders such as the current date, image sequence number, or file format::
107
>>> import time, os.path
108
>>> photofiles = ['img_1074.jpg', 'img_1076.jpg', 'img_1077.jpg']
109
>>> class BatchRename(Template):
111
>>> fmt = input('Enter rename style (%d-date %n-seqnum %f-format): ')
112
Enter rename style (%d-date %n-seqnum %f-format): Ashley_%n%f
114
>>> t = BatchRename(fmt)
115
>>> date = time.strftime('%d%b%y')
116
>>> for i, filename in enumerate(photofiles):
117
... base, ext = os.path.splitext(filename)
118
... newname = t.substitute(d=date, n=i, f=ext)
119
... print('{0} --> {1}'.format(filename, newname))
121
img_1074.jpg --> Ashley_0.jpg
122
img_1076.jpg --> Ashley_1.jpg
123
img_1077.jpg --> Ashley_2.jpg
125
Another application for templating is separating program logic from the details
126
of multiple output formats. This makes it possible to substitute custom
127
templates for XML files, plain text reports, and HTML web reports.
130
.. _tut-binary-formats:
132
Working with Binary Data Record Layouts
133
=======================================
135
The :mod:`struct` module provides :func:`~struct.pack` and
136
:func:`~struct.unpack` functions for working with variable length binary
137
record formats. The following example shows
138
how to loop through header information in a ZIP file without using the
139
:mod:`zipfile` module. Pack codes ``"H"`` and ``"I"`` represent two and four
140
byte unsigned numbers respectively. The ``"<"`` indicates that they are
141
standard size and in little-endian byte order::
145
with open('myfile.zip', 'rb') as f:
149
for i in range(3): # show the first 3 file headers
151
fields = struct.unpack('<IIIHH', data[start:start+16])
152
crc32, comp_size, uncomp_size, filenamesize, extra_size = fields
155
filename = data[start:start+filenamesize]
156
start += filenamesize
157
extra = data[start:start+extra_size]
158
print(filename, hex(crc32), comp_size, uncomp_size)
160
start += extra_size + comp_size # skip to the next header
163
.. _tut-multi-threading:
168
Threading is a technique for decoupling tasks which are not sequentially
169
dependent. Threads can be used to improve the responsiveness of applications
170
that accept user input while other tasks run in the background. A related use
171
case is running I/O in parallel with computations in another thread.
173
The following code shows how the high level :mod:`threading` module can run
174
tasks in background while the main program continues to run::
176
import threading, zipfile
178
class AsyncZip(threading.Thread):
179
def __init__(self, infile, outfile):
180
threading.Thread.__init__(self)
182
self.outfile = outfile
184
f = zipfile.ZipFile(self.outfile, 'w', zipfile.ZIP_DEFLATED)
187
print('Finished background zip of:', self.infile)
189
background = AsyncZip('mydata.txt', 'myarchive.zip')
191
print('The main program continues to run in foreground.')
193
background.join() # Wait for the background task to finish
194
print('Main program waited until background was done.')
196
The principal challenge of multi-threaded applications is coordinating threads
197
that share data or other resources. To that end, the threading module provides
198
a number of synchronization primitives including locks, events, condition
199
variables, and semaphores.
201
While those tools are powerful, minor design errors can result in problems that
202
are difficult to reproduce. So, the preferred approach to task coordination is
203
to concentrate all access to a resource in a single thread and then use the
204
:mod:`queue` module to feed that thread with requests from other threads.
205
Applications using :class:`~queue.Queue` objects for inter-thread communication and
206
coordination are easier to design, more readable, and more reliable.
214
The :mod:`logging` module offers a full featured and flexible logging system.
215
At its simplest, log messages are sent to a file or to ``sys.stderr``::
218
logging.debug('Debugging information')
219
logging.info('Informational message')
220
logging.warning('Warning:config file %s not found', 'server.conf')
221
logging.error('Error occurred')
222
logging.critical('Critical error -- shutting down')
224
This produces the following output:
228
WARNING:root:Warning:config file server.conf not found
229
ERROR:root:Error occurred
230
CRITICAL:root:Critical error -- shutting down
232
By default, informational and debugging messages are suppressed and the output
233
is sent to standard error. Other output options include routing messages
234
through email, datagrams, sockets, or to an HTTP Server. New filters can select
235
different routing based on message priority: :const:`~logging.DEBUG`,
236
:const:`~logging.INFO`, :const:`~logging.WARNING`, :const:`~logging.ERROR`,
237
and :const:`~logging.CRITICAL`.
239
The logging system can be configured directly from Python or can be loaded from
240
a user editable configuration file for customized logging without altering the
244
.. _tut-weak-references:
249
Python does automatic memory management (reference counting for most objects and
250
:term:`garbage collection` to eliminate cycles). The memory is freed shortly
251
after the last reference to it has been eliminated.
253
This approach works fine for most applications but occasionally there is a need
254
to track objects only as long as they are being used by something else.
255
Unfortunately, just tracking them creates a reference that makes them permanent.
256
The :mod:`weakref` module provides tools for tracking objects without creating a
257
reference. When the object is no longer needed, it is automatically removed
258
from a weakref table and a callback is triggered for weakref objects. Typical
259
applications include caching objects that are expensive to create::
261
>>> import weakref, gc
263
... def __init__(self, value):
264
... self.value = value
265
... def __repr__(self):
266
... return str(self.value)
268
>>> a = A(10) # create a reference
269
>>> d = weakref.WeakValueDictionary()
270
>>> d['primary'] = a # does not create a reference
271
>>> d['primary'] # fetch the object if it is still alive
273
>>> del a # remove the one reference
274
>>> gc.collect() # run garbage collection right away
276
>>> d['primary'] # entry was automatically removed
277
Traceback (most recent call last):
278
File "<stdin>", line 1, in <module>
279
d['primary'] # entry was automatically removed
280
File "C:/python35/lib/weakref.py", line 46, in __getitem__
287
Tools for Working with Lists
288
============================
290
Many data structure needs can be met with the built-in list type. However,
291
sometimes there is a need for alternative implementations with different
292
performance trade-offs.
294
The :mod:`array` module provides an :class:`~array.array()` object that is like
295
a list that stores only homogeneous data and stores it more compactly. The
296
following example shows an array of numbers stored as two byte unsigned binary
297
numbers (typecode ``"H"``) rather than the usual 16 bytes per entry for regular
298
lists of Python int objects::
300
>>> from array import array
301
>>> a = array('H', [4000, 10, 700, 22222])
305
array('H', [10, 700])
307
The :mod:`collections` module provides a :class:`~collections.deque()` object
308
that is like a list with faster appends and pops from the left side but slower
309
lookups in the middle. These objects are well suited for implementing queues
310
and breadth first tree searches::
312
>>> from collections import deque
313
>>> d = deque(["task1", "task2", "task3"])
314
>>> d.append("task4")
315
>>> print("Handling", d.popleft())
320
unsearched = deque([starting_node])
321
def breadth_first_search(unsearched):
322
node = unsearched.popleft()
323
for m in gen_moves(node):
328
In addition to alternative list implementations, the library also offers other
329
tools such as the :mod:`bisect` module with functions for manipulating sorted
333
>>> scores = [(100, 'perl'), (200, 'tcl'), (400, 'lua'), (500, 'python')]
334
>>> bisect.insort(scores, (300, 'ruby'))
336
[(100, 'perl'), (200, 'tcl'), (300, 'ruby'), (400, 'lua'), (500, 'python')]
338
The :mod:`heapq` module provides functions for implementing heaps based on
339
regular lists. The lowest valued entry is always kept at position zero. This
340
is useful for applications which repeatedly access the smallest element but do
341
not want to run a full list sort::
343
>>> from heapq import heapify, heappop, heappush
344
>>> data = [1, 3, 5, 7, 9, 2, 4, 6, 8, 0]
345
>>> heapify(data) # rearrange the list into heap order
346
>>> heappush(data, -5) # add a new entry
347
>>> [heappop(data) for i in range(3)] # fetch the three smallest entries
353
Decimal Floating Point Arithmetic
354
=================================
356
The :mod:`decimal` module offers a :class:`~decimal.Decimal` datatype for
357
decimal floating point arithmetic. Compared to the built-in :class:`float`
358
implementation of binary floating point, the class is especially helpful for
360
* financial applications and other uses which require exact decimal
362
* control over precision,
363
* control over rounding to meet legal or regulatory requirements,
364
* tracking of significant decimal places, or
365
* applications where the user expects the results to match calculations done by
368
For example, calculating a 5% tax on a 70 cent phone charge gives different
369
results in decimal floating point and binary floating point. The difference
370
becomes significant if the results are rounded to the nearest cent::
372
>>> from decimal import *
373
>>> round(Decimal('0.70') * Decimal('1.05'), 2)
375
>>> round(.70 * 1.05, 2)
378
The :class:`~decimal.Decimal` result keeps a trailing zero, automatically
379
inferring four place significance from multiplicands with two place
380
significance. Decimal reproduces mathematics as done by hand and avoids
381
issues that can arise when binary floating point cannot exactly represent
384
Exact representation enables the :class:`~decimal.Decimal` class to perform
385
modulo calculations and equality tests that are unsuitable for binary floating
388
>>> Decimal('1.00') % Decimal('.10')
393
>>> sum([Decimal('0.1')]*10) == Decimal('1.0')
395
>>> sum([0.1]*10) == 1.0
398
The :mod:`decimal` module provides arithmetic with as much precision as needed::
400
>>> getcontext().prec = 36
401
>>> Decimal(1) / Decimal(7)
402
Decimal('0.142857142857142857142857142857142857')
added
removed
python-3.5.1-docs-html/_sources/tutorial/stdlib2.txt
