~dkuhlman/python-training-materials/Materials

<tr class="field-odd field"><th class="field-name">Author:</th><td class="field-body">R. David Murray <<a class="reference external" href="mailto:rdmurray%40bitdance.com">rdmurray@bitdance.com</a>> (Editor)</td>

100

</tr>

101

</tbody>

102

</table>

103

This article explains the new features in Python 3.4, compared to 3.3.

104

Python 3.4 was released on March 16, 2014. For full details, see the

105

<a class="reference external" href="https://docs.python.org/3.4/whatsnew/changelog.html">changelog</a>.

106

107

See also

108

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0429">PEP 429</a> – Python 3.4 Release Schedule

109

</div>

110

111

<h2>Summary – Release Highlights<a class="headerlink" href="#summary-release-highlights" title="Permalink to this headline">¶</a></h2>

112

New syntax features:

113

114

<li>No new syntax features were added in Python 3.4.</li>

115

</ul>

116

Other new features:

117

118

<li><a class="reference internal" href="#whatsnew-pep-453">pip should always be available</a> (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0453">PEP 453</a>).</li>

119

<li><a class="reference internal" href="#whatsnew-pep-446">Newly created file descriptors are non-inheritable</a>

120

(<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0446">PEP 446</a>).</li>

121

<li>command line option for <a class="reference internal" href="#whatsnew-isolated-mode">isolated mode</a>

122

(<a class="reference external" href="https://bugs.python.org/issue16499">issue 16499</a>).</li>

123

<li><a class="reference internal" href="#codec-handling-improvements">improvements in the handling of codecs</a>

124

that are not text encodings (multiple issues).</li>

125

<li><a class="reference internal" href="#whatsnew-pep-451">A ModuleSpec Type</a> for the Import System

126

(<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0451">PEP 451</a>). (Affects importer authors.)</li>

127

<li>The <a class="reference internal" href="../library/marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal">marshal</code></a> format has been made <a class="reference internal" href="#whatsnew-marshal-3">more compact and efficient</a> (<a class="reference external" href="https://bugs.python.org/issue16475">issue 16475</a>).</li>

128

</ul>

129

New library modules:

130

131

<li><a class="reference internal" href="../library/asyncio.html#module-asyncio" title="asyncio: Asynchronous I/O, event loop, coroutines and tasks."><code class="xref py py-mod docutils literal">asyncio</code></a>: <a class="reference internal" href="#whatsnew-asyncio">New provisional API for asynchronous IO</a> (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-3156">PEP 3156</a>).</li>

132

<li><a class="reference internal" href="../library/ensurepip.html#module-ensurepip" title="ensurepip: Bootstrapping the "pip" installer into an existing Python installation or virtual environment."><code class="xref py py-mod docutils literal">ensurepip</code></a>: <a class="reference internal" href="#whatsnew-ensurepip">Bootstrapping the pip installer</a>

133

(<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0453">PEP 453</a>).</li>

134

<li><a class="reference internal" href="../library/enum.html#module-enum" title="enum: Implementation of an enumeration class."><code class="xref py py-mod docutils literal">enum</code></a>: <a class="reference internal" href="#whatsnew-enum">Support for enumeration types</a>

135

(<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0435">PEP 435</a>).</li>

136

<li><a class="reference internal" href="../library/pathlib.html#module-pathlib" title="pathlib: Object-oriented filesystem paths"><code class="xref py py-mod docutils literal">pathlib</code></a>: <a class="reference internal" href="#whatsnew-pathlib">Object-oriented filesystem paths</a>

137

(<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0428">PEP 428</a>).</li>

138

<li><a class="reference internal" href="../library/selectors.html#module-selectors" title="selectors: High-level I/O multiplexing."><code class="xref py py-mod docutils literal">selectors</code></a>: <a class="reference internal" href="#whatsnew-selectors">High-level and efficient I/O multiplexing</a>, built upon the <a class="reference internal" href="../library/select.html#module-select" title="select: Wait for I/O completion on multiple streams."><code class="xref py py-mod docutils literal">select</code></a> module primitives (part

139

of <a class="pep reference external" href="https://www.python.org/dev/peps/pep-3156">PEP 3156</a>).</li>

140

<li><a class="reference internal" href="../library/statistics.html#module-statistics" title="statistics: mathematical statistics functions"><code class="xref py py-mod docutils literal">statistics</code></a>: A basic <a class="reference internal" href="#whatsnew-statistics">numerically stable statistics library</a> (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0450">PEP 450</a>).</li>

141

<li><a class="reference internal" href="../library/tracemalloc.html#module-tracemalloc" title="tracemalloc: Trace memory allocations."><code class="xref py py-mod docutils literal">tracemalloc</code></a>: <a class="reference internal" href="#whatsnew-tracemalloc">Trace Python memory allocations</a> (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0454">PEP 454</a>).</li>

142

</ul>

143

Significantly improved library modules:

144

145

<li><a class="reference internal" href="#whatsnew-singledispatch">Single-dispatch generic functions</a> in

146

<a class="reference internal" href="../library/functools.html#module-functools" title="functools: Higher-order functions and operations on callable objects."><code class="xref py py-mod docutils literal">functools</code></a> (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0443">PEP 443</a>).</li>

147

<li>New <a class="reference internal" href="../library/pickle.html#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal">pickle</code></a> <a class="reference internal" href="#whatsnew-protocol-4">protocol 4</a> (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-3154">PEP 3154</a>).</li>

148

<li><a class="reference internal" href="../library/multiprocessing.html#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal">multiprocessing</code></a> now has <a class="reference internal" href="#whatsnew-multiprocessing-no-fork">an option to avoid using os.fork

149

on Unix</a> (<a class="reference external" href="https://bugs.python.org/issue8713">issue 8713</a>).</li>

150

<li><a class="reference internal" href="../library/email.html#module-email" title="email: Package supporting the parsing, manipulating, and generating email messages, including MIME documents."><code class="xref py py-mod docutils literal">email</code></a> has a new submodule, <a class="reference internal" href="../library/email.contentmanager.html#module-email.contentmanager" title="email.contentmanager: Storing and Retrieving Content from MIME Parts"><code class="xref py py-mod docutils literal">contentmanager</code></a>, and

151

a new <a class="reference internal" href="../library/email.message.html#email.message.Message" title="email.message.Message"><code class="xref py py-mod docutils literal">Message</code></a> subclass

152

(<code class="xref py py-class docutils literal">EmailMessage</code>) that <a class="reference internal" href="#whatsnew-email-contentmanager">simplify MIME

153

handling</a> (<a class="reference external" href="https://bugs.python.org/issue18891">issue 18891</a>).</li>

154

<li>The <a class="reference internal" href="../library/inspect.html#module-inspect" title="inspect: Extract information and source code from live objects."><code class="xref py py-mod docutils literal">inspect</code></a> and <a class="reference internal" href="../library/pydoc.html#module-pydoc" title="pydoc: Documentation generator and online help system."><code class="xref py py-mod docutils literal">pydoc</code></a> modules are now capable of

155

correct introspection of a much wider variety of callable objects,

156

which improves the output of the Python <a class="reference internal" href="../library/functions.html#help" title="help"><code class="xref py py-func docutils literal">help()</code></a> system.</li>

157

<li>The <a class="reference internal" href="../library/ipaddress.html#module-ipaddress" title="ipaddress: IPv4/IPv6 manipulation library."><code class="xref py py-mod docutils literal">ipaddress</code></a> module API has been declared stable</li>

158

</ul>

159

Security improvements:

160

161

<li><a class="reference internal" href="#whatsnew-pep-456">Secure and interchangeable hash algorithm</a>

162

(<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0456">PEP 456</a>).</li>

163

<li><a class="reference internal" href="#whatsnew-pep-446">Make newly created file descriptors non-inheritable</a>

164

(<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0446">PEP 446</a>) to avoid leaking file descriptors to child processes.</li>

165

<li>New command line option for <a class="reference internal" href="#whatsnew-isolated-mode">isolated mode</a>,

166

(<a class="reference external" href="https://bugs.python.org/issue16499">issue 16499</a>).</li>

167

168

on Unix</a>. spawn and forkserver are

169

more secure because they avoid sharing data with child processes.</li>

170

171

all of the parent’s inheritable handles, only the necessary ones.</li>

172

<li>A new <a class="reference internal" href="../library/hashlib.html#hashlib.pbkdf2_hmac" title="hashlib.pbkdf2_hmac"><code class="xref py py-func docutils literal">hashlib.pbkdf2_hmac()</code></a> function provides

173

the <a class="reference external" href="https://en.wikipedia.org/wiki/PBKDF2">PKCS#5 password-based key derivation function 2</a>.</li>

174

<li><a class="reference internal" href="#whatsnew-tls-11-12">TLSv1.1 and TLSv1.2 support</a> for <a class="reference internal" href="../library/ssl.html#module-ssl" title="ssl: TLS/SSL wrapper for socket objects"><code class="xref py py-mod docutils literal">ssl</code></a>.</li>

175

<li><a class="reference internal" href="#whatsnew34-win-cert-store">Retrieving certificates from the Windows system cert store support</a> for <a class="reference internal" href="../library/ssl.html#module-ssl" title="ssl: TLS/SSL wrapper for socket objects"><code class="xref py py-mod docutils literal">ssl</code></a>.</li>

176

<li><a class="reference internal" href="#whatsnew34-sni">Server-side SNI (Server Name Indication) support</a> for <a class="reference internal" href="../library/ssl.html#module-ssl" title="ssl: TLS/SSL wrapper for socket objects"><code class="xref py py-mod docutils literal">ssl</code></a>.</li>

177

<li>The <a class="reference internal" href="../library/ssl.html#ssl.SSLContext" title="ssl.SSLContext"><code class="xref py py-class docutils literal">ssl.SSLContext</code></a> class has a <a class="reference internal" href="#whatsnew34-sslcontext">lot of improvements</a>.</li>

178

<li>All modules in the standard library that support SSL now support server

179

certificate verification, including hostname matching

180

(<a class="reference internal" href="../library/ssl.html#ssl.match_hostname" title="ssl.match_hostname"><code class="xref py py-func docutils literal">ssl.match_hostname()</code></a>) and CRLs (Certificate Revocation lists, see

181

<a class="reference internal" href="../library/ssl.html#ssl.SSLContext.load_verify_locations" title="ssl.SSLContext.load_verify_locations"><code class="xref py py-func docutils literal">ssl.SSLContext.load_verify_locations()</code></a>).</li>

182

</ul>

183

CPython implementation improvements:

184

185

<li><a class="reference internal" href="#whatsnew-pep-442">Safe object finalization</a> (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0442">PEP 442</a>).</li>

186

<li>Leveraging <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0442">PEP 442</a>, in most cases <a class="reference internal" href="#whatsnew-pep-442">module globals are no longer set

187

to None during finalization</a> (<a class="reference external" href="https://bugs.python.org/issue18214">issue 18214</a>).</li>

188

<li><a class="reference internal" href="#whatsnew-pep-445">Configurable memory allocators</a> (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0445">PEP 445</a>).</li>

189

<li><a class="reference internal" href="#whatsnew-pep-436">Argument Clinic</a> (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0436">PEP 436</a>).</li>

190

</ul>

191

Please read on for a comprehensive list of user-facing changes, including many

192

other smaller improvements, CPython optimizations, deprecations, and potential

193

porting issues.

194

</div>

195

196

<h2>New Features<a class="headerlink" href="#new-features" title="Permalink to this headline">¶</a></h2>

197

198

<h3>PEP 453: Explicit Bootstrapping of PIP in Python Installations<a class="headerlink" href="#pep-453-explicit-bootstrapping-of-pip-in-python-installations" title="Permalink to this headline">¶</a></h3>

199

200

<h4>Bootstrapping pip By Default<a class="headerlink" href="#bootstrapping-pip-by-default" title="Permalink to this headline">¶</a></h4>

201

The new <a class="reference internal" href="../library/ensurepip.html#module-ensurepip" title="ensurepip: Bootstrapping the "pip" installer into an existing Python installation or virtual environment."><code class="xref py py-mod docutils literal">ensurepip</code></a> module (defined in <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0453">PEP 453</a>) provides a standard

202

cross-platform mechanism to bootstrap the pip installer into Python

203

installations and virtual environments. The version of <code class="docutils literal">pip</code> included

204

with Python 3.4.0 is <code class="docutils literal">pip</code> 1.5.4, and future 3.4.x maintenance releases

205

will update the bundled version to the latest version of <code class="docutils literal">pip</code> that is

206

available at the time of creating the release candidate.

207

By default, the commands <code class="docutils literal">pipX</code> and <code class="docutils literal">pipX.Y</code> will be installed on all

208

platforms (where X.Y stands for the version of the Python installation),

209

along with the <code class="docutils literal">pip</code> Python package and its dependencies. On Windows and

210

in virtual environments on all platforms, the unversioned <code class="docutils literal">pip</code> command

211

will also be installed. On other platforms, the system wide unversioned

212

<code class="docutils literal">pip</code> command typically refers to the separately installed Python 2

213

version.

214

The <a class="reference internal" href="../using/scripts.html#scripts-pyvenv">pyvenv</a> command line utility and the <a class="reference internal" href="../library/venv.html#module-venv" title="venv: Creation of virtual environments."><code class="xref py py-mod docutils literal">venv</code></a>

215

module make use of the <a class="reference internal" href="../library/ensurepip.html#module-ensurepip" title="ensurepip: Bootstrapping the "pip" installer into an existing Python installation or virtual environment."><code class="xref py py-mod docutils literal">ensurepip</code></a> module to make <code class="docutils literal">pip</code> readily

216

available in virtual environments. When using the command line utility,

217

<code class="docutils literal">pip</code> is installed by default, while when using the <a class="reference internal" href="../library/venv.html#module-venv" title="venv: Creation of virtual environments."><code class="xref py py-mod docutils literal">venv</code></a> module

218

<a class="reference internal" href="../library/venv.html#venv-api">API</a> installation of <code class="docutils literal">pip</code> must be requested explicitly.

219

For CPython <a class="reference internal" href="../using/unix.html#building-python-on-unix">source builds on POSIX systems</a>,

220

the <code class="docutils literal">make install</code> and <code class="docutils literal">make altinstall</code> commands bootstrap <code class="docutils literal">pip</code> by

221

default. This behaviour can be controlled through configure options, and

222

overridden through Makefile options.

223

On Windows and Mac OS X, the CPython installers now default to installing

224

<code class="docutils literal">pip</code> along with CPython itself (users may opt out of installing it

225

during the installation process). Window users will need to opt in to the

226

automatic <code class="docutils literal">PATH</code> modifications to have <code class="docutils literal">pip</code> available from the command

227

line by default, otherwise it can still be accessed through the Python

228

launcher for Windows as <code class="docutils literal">py -m pip</code>.

229

As <a class="reference external" href="https://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors">discussed in the PEP</a>, platform packagers may choose not to install

230

these commands by default, as long as, when invoked, they provide clear and

231

simple directions on how to install them on that platform (usually using

232

the system package manager).

233

234

Note

235

To avoid conflicts between parallel Python 2 and Python 3 installations,

236

only the versioned <code class="docutils literal">pip3</code> and <code class="docutils literal">pip3.4</code> commands are bootstrapped by

237

default when <code class="docutils literal">ensurepip</code> is invoked directly - the <code class="docutils literal">--default-pip</code>

238

option is needed to also request the unversioned <code class="docutils literal">pip</code> command.

239

<code class="docutils literal">pyvenv</code> and the Windows installer ensure that the unqualified <code class="docutils literal">pip</code>

240

command is made available in those environments, and <code class="docutils literal">pip</code> can always be

241

invoked via the <code class="docutils literal">-m</code> switch rather than directly to avoid ambiguity on

242

systems with multiple Python installations.

243

</div>

244

</div>

245

246

<h4>Documentation Changes<a class="headerlink" href="#documentation-changes" title="Permalink to this headline">¶</a></h4>

247

As part of this change, the <a class="reference internal" href="../installing/index.html#installing-index">Installing Python Modules</a> and

248

<a class="reference internal" href="../distributing/index.html#distributing-index">Distributing Python Modules</a> sections of the documentation have been

249

completely redesigned as short getting started and FAQ documents. Most

250

packaging documentation has now been moved out to the Python Packaging

251

Authority maintained <a class="reference external" href="https://packaging.python.org">Python Packaging User Guide</a> and the documentation of the individual

252

projects.

253

However, as this migration is currently still incomplete, the legacy

254

versions of those guides remaining available as <a class="reference internal" href="../install/index.html#install-index">Installing Python Modules (Legacy version)</a>

255

and <a class="reference internal" href="../distutils/index.html#distutils-index">Distributing Python Modules (Legacy version)</a>.

256

257

See also

258

259

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0453">PEP 453</a> – Explicit bootstrapping of pip in Python installations</dt>

260

<dd>PEP written by Donald Stufft and Nick Coghlan, implemented by

261

Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily.</dd>

262

</dl>

263

</div>

264

</div>

265

</div>

266

267

<h3>PEP 446: Newly Created File Descriptors Are Non-Inheritable<a class="headerlink" href="#pep-446-newly-created-file-descriptors-are-non-inheritable" title="Permalink to this headline">¶</a></h3>

268

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0446">PEP 446</a> makes newly created file descriptors <a class="reference internal" href="../library/os.html#fd-inheritance">non-inheritable</a>. In general, this is the behavior an application will

269

want: when launching a new process, having currently open files also

270

open in the new process can lead to all sorts of hard to find bugs,

271

and potentially to security issues.

272

However, there are occasions when inheritance is desired. To support

273

these cases, the following new functions and methods are available:

274

275

<li><a class="reference internal" href="../library/os.html#os.get_inheritable" title="os.get_inheritable"><code class="xref py py-func docutils literal">os.get_inheritable()</code></a>, <a class="reference internal" href="../library/os.html#os.set_inheritable" title="os.set_inheritable"><code class="xref py py-func docutils literal">os.set_inheritable()</code></a></li>

276

<li><a class="reference internal" href="../library/os.html#os.get_handle_inheritable" title="os.get_handle_inheritable"><code class="xref py py-func docutils literal">os.get_handle_inheritable()</code></a>, <a class="reference internal" href="../library/os.html#os.set_handle_inheritable" title="os.set_handle_inheritable"><code class="xref py py-func docutils literal">os.set_handle_inheritable()</code></a></li>

277

<li><a class="reference internal" href="../library/socket.html#socket.socket.get_inheritable" title="socket.socket.get_inheritable"><code class="xref py py-meth docutils literal">socket.socket.get_inheritable()</code></a>, <a class="reference internal" href="../library/socket.html#socket.socket.set_inheritable" title="socket.socket.set_inheritable"><code class="xref py py-meth docutils literal">socket.socket.set_inheritable()</code></a></li>

278

</ul>

279

280

See also

281

282

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0446">PEP 446</a> – Make newly created file descriptors non-inheritable</dt>

283

<dd>PEP written and implemented by Victor Stinner.</dd>

284

</dl>

285

</div>

286

</div>

287

288

<h3>Improvements to Codec Handling<a class="headerlink" href="#improvements-to-codec-handling" title="Permalink to this headline">¶</a></h3>

289

Since it was first introduced, the <a class="reference internal" href="../library/codecs.html#module-codecs" title="codecs: Encode and decode data and streams."><code class="xref py py-mod docutils literal">codecs</code></a> module has always been

290

intended to operate as a type-neutral dynamic encoding and decoding

291

system. However, its close coupling with the Python text model, especially

292

the type restricted convenience methods on the builtin <a class="reference internal" href="../library/stdtypes.html#str" title="str"><code class="xref py py-class docutils literal">str</code></a>,

293

<a class="reference internal" href="../library/functions.html#bytes" title="bytes"><code class="xref py py-class docutils literal">bytes</code></a> and <a class="reference internal" href="../library/functions.html#bytearray" title="bytearray"><code class="xref py py-class docutils literal">bytearray</code></a> types, has historically obscured that

294

fact.

295

As a key step in clarifying the situation, the <a class="reference internal" href="../library/codecs.html#codecs.encode" title="codecs.encode"><code class="xref py py-meth docutils literal">codecs.encode()</code></a> and

296

<a class="reference internal" href="../library/codecs.html#codecs.decode" title="codecs.decode"><code class="xref py py-meth docutils literal">codecs.decode()</code></a> convenience functions are now properly documented in

297

Python 2.7, 3.3 and 3.4. These functions have existed in the <a class="reference internal" href="../library/codecs.html#module-codecs" title="codecs: Encode and decode data and streams."><code class="xref py py-mod docutils literal">codecs</code></a>

298

module (and have been covered by the regression test suite) since Python 2.4,

299

but were previously only discoverable through runtime introspection.

300

Unlike the convenience methods on <a class="reference internal" href="../library/stdtypes.html#str" title="str"><code class="xref py py-class docutils literal">str</code></a>, <a class="reference internal" href="../library/functions.html#bytes" title="bytes"><code class="xref py py-class docutils literal">bytes</code></a> and

301

<a class="reference internal" href="../library/functions.html#bytearray" title="bytearray"><code class="xref py py-class docutils literal">bytearray</code></a>, the <a class="reference internal" href="../library/codecs.html#module-codecs" title="codecs: Encode and decode data and streams."><code class="xref py py-mod docutils literal">codecs</code></a> convenience functions support arbitrary

302

codecs in both Python 2 and Python 3, rather than being limited to Unicode text

303

encodings (in Python 3) or <code class="docutils literal">basestring</code> <-> <code class="docutils literal">basestring</code> conversions (in

304

Python 2).

305

In Python 3.4, the interpreter is able to identify the known non-text

306

encodings provided in the standard library and direct users towards these

307

general purpose convenience functions when appropriate:

308

<div class="highlight-python3"><div class="highlight"><pre>>>> b"abcdef".decode("hex")

309

Traceback (most recent call last):

310

File "<stdin>", line 1, in <module>

311

LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs

312

313

>>> "hello".encode("rot13")

314

Traceback (most recent call last):

315

File "<stdin>", line 1, in <module>

316

LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs

317

318

>>> open("foo.txt", encoding="hex")

319

Traceback (most recent call last):

320

File "<stdin>", line 1, in <module>

321

LookupError: 'hex' is not a text encoding; use codecs.open() to handle arbitrary codecs

322

</pre></div>

323

</div>

324

In a related change, whenever it is feasible without breaking backwards

325

compatibility, exceptions raised during encoding and decoding operations

326

are wrapped in a chained exception of the same type that mentions the

327

name of the codec responsible for producing the error:

328

<div class="highlight-python3"><div class="highlight"><pre>>>> import codecs

329

330

>>> codecs.decode(b"abcdefgh", "hex")

331

Traceback (most recent call last):

332

File "/usr/lib/python3.4/encodings/hex_codec.py", line 20, in hex_decode

333

return (binascii.a2b_hex(input), len(input))

334

binascii.Error: Non-hexadecimal digit found

335

336

The above exception was the direct cause of the following exception:

337

338

Traceback (most recent call last):

339

File "<stdin>", line 1, in <module>

340

binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found)

341

342

>>> codecs.encode("hello", "bz2")

343

Traceback (most recent call last):

344

File "/usr/lib/python3.4/encodings/bz2_codec.py", line 17, in bz2_encode

345

return (bz2.compress(input), len(input))

346

File "/usr/lib/python3.4/bz2.py", line 498, in compress

347

return comp.compress(data) + comp.flush()

348

TypeError: 'str' does not support the buffer interface

349

350

The above exception was the direct cause of the following exception:

351

352

Traceback (most recent call last):

353

File "<stdin>", line 1, in <module>

354

TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface)

355

</pre></div>

356

</div>

357

Finally, as the examples above show, these improvements have permitted

358

the restoration of the convenience aliases for the non-Unicode codecs that

359

were themselves restored in Python 3.2. This means that encoding binary data

360

to and from its hexadecimal representation (for example) can now be written

361

as:

362

<div class="highlight-python3"><div class="highlight"><pre>>>> from codecs import encode, decode

363

>>> encode(b"hello", "hex")

364

b'68656c6c6f'

365

>>> decode(b"68656c6c6f", "hex")

366

b'hello'

367

</pre></div>

368

</div>

369

The binary and text transforms provided in the standard library are detailed

370

in <a class="reference internal" href="../library/codecs.html#binary-transforms">Binary Transforms</a> and <a class="reference internal" href="../library/codecs.html#text-transforms">Text Transforms</a>.

371

(Contributed by Nick Coghlan in <a class="reference external" href="https://bugs.python.org/issue7475">issue 7475</a>, <a class="reference external" href="https://bugs.python.org/issue17827">issue 17827</a>,

372

<a class="reference external" href="https://bugs.python.org/issue17828">issue 17828</a> and <a class="reference external" href="https://bugs.python.org/issue19619">issue 19619</a>.)

373

</div>

374

375

<h3>PEP 451: A ModuleSpec Type for the Import System<a class="headerlink" href="#pep-451-a-modulespec-type-for-the-import-system" title="Permalink to this headline">¶</a></h3>

376

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0451">PEP 451</a> provides an encapsulation of the information about a module that the

377

import machinery will use to load it (that is, a module specification). This

378

helps simplify both the import implementation and several import-related APIs.

379

The change is also a stepping stone for <a class="reference external" href="https://mail.python.org/pipermail/python-dev/2013-November/130111.html">several future import-related

380

improvements</a>.

381

The public-facing changes from the PEP are entirely backward-compatible.

382

Furthermore, they should be transparent to everyone but importer authors. Key

383

finder and loader methods have been deprecated, but they will continue working.

384

New importers should use the new methods described in the PEP. Existing

385

importers should be updated to implement the new methods. See the

386

<a class="reference internal" href="#deprecated-3-4">Deprecated</a> section for a list of methods that should be replaced and

387

their replacements.

388

</div>

389

390

<h3>Other Language Changes<a class="headerlink" href="#other-language-changes" title="Permalink to this headline">¶</a></h3>

391

Some smaller changes made to the core Python language are:

392

393

<li>Unicode database updated to UCD version 6.3.</li>

394

<li><a class="reference internal" href="../library/functions.html#min" title="min"><code class="xref py py-func docutils literal">min()</code></a> and <a class="reference internal" href="../library/functions.html#max" title="max"><code class="xref py py-func docutils literal">max()</code></a> now accept a default keyword-only argument that

395

can be used to specify the value they return if the iterable they are

396

evaluating has no elements. (Contributed by Julian Berman in

397

<a class="reference external" href="https://bugs.python.org/issue18111">issue 18111</a>.)</li>

398

<li>Module objects are now <a class="reference internal" href="../library/weakref.html#module-weakref" title="weakref: Support for weak references and weak dictionaries."><code class="xref py py-mod docutils literal">weakref</code></a>‘able.</li>

399

<li>Module <code class="docutils literal">__file__</code> attributes (and related values) should now always

400

contain absolute paths by default, with the sole exception of

401

<code class="docutils literal">__main__.__file__</code> when a script has been executed directly using

402

a relative path. (Contributed by Brett Cannon in <a class="reference external" href="https://bugs.python.org/issue18416">issue 18416</a>.)</li>

403

<li>All the UTF-* codecs (except UTF-7) now reject surrogates during both

404

encoding and decoding unless the <code class="docutils literal">surrogatepass</code> error handler is used,

405

with the exception of the UTF-16 decoder (which accepts valid surrogate pairs)

406

and the UTF-16 encoder (which produces them while encoding non-BMP characters).

407

(Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in

408

<a class="reference external" href="https://bugs.python.org/issue12892">issue 12892</a>.)</li>

409

<li>New German EBCDIC <a class="reference internal" href="../library/codecs.html#standard-encodings">codec</a> <code class="docutils literal">cp273</code>. (Contributed

410

by Michael Bierenfeld and Andrew Kuchling in <a class="reference external" href="https://bugs.python.org/issue1097797">issue 1097797</a>.)</li>

411

<li>New Ukrainian <a class="reference internal" href="../library/codecs.html#standard-encodings">codec</a> <code class="docutils literal">cp1125</code>. (Contributed by

412

Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue19668">issue 19668</a>.)</li>

413

<li><a class="reference internal" href="../library/functions.html#bytes" title="bytes"><code class="xref py py-class docutils literal">bytes</code></a>.join() and <a class="reference internal" href="../library/functions.html#bytearray" title="bytearray"><code class="xref py py-class docutils literal">bytearray</code></a>.join() now accept arbitrary

414

buffer objects as arguments. (Contributed by Antoine Pitrou in

415

<a class="reference external" href="https://bugs.python.org/issue15958">issue 15958</a>.)</li>

416

<li>The <a class="reference internal" href="../library/functions.html#int" title="int"><code class="xref py py-class docutils literal">int</code></a> constructor now accepts any object that has an <code class="docutils literal">__index__</code>

417

method for its base argument. (Contributed by Mark Dickinson in

418

<a class="reference external" href="https://bugs.python.org/issue16772">issue 16772</a>.)</li>

419

<li>Frame objects now have a <a class="reference internal" href="../reference/datamodel.html#frame.clear" title="frame.clear"><code class="xref py py-func docutils literal">clear()</code></a> method that clears all

420

references to local variables from the frame. (Contributed by Antoine Pitrou

421

in <a class="reference external" href="https://bugs.python.org/issue17934">issue 17934</a>.)</li>

422

<li><a class="reference internal" href="../library/stdtypes.html#memoryview" title="memoryview"><code class="xref py py-class docutils literal">memoryview</code></a> is now registered as a <a class="reference internal" href="../library/collections.abc.html#module-collections.abc" title="collections.abc: Abstract base classes for containers"><code class="xref py py-class docutils literal">Sequence</code></a>,

423

and supports the <a class="reference internal" href="../library/functions.html#reversed" title="reversed"><code class="xref py py-func docutils literal">reversed()</code></a> builtin. (Contributed by Nick Coghlan

424

and Claudiu Popa in <a class="reference external" href="https://bugs.python.org/issue18690">issue 18690</a> and <a class="reference external" href="https://bugs.python.org/issue19078">issue 19078</a>.)</li>

425

<li>Signatures reported by <a class="reference internal" href="../library/functions.html#help" title="help"><code class="xref py py-func docutils literal">help()</code></a> have been modified and improved in

426

several cases as a result of the introduction of Argument Clinic and other

427

changes to the <a class="reference internal" href="../library/inspect.html#module-inspect" title="inspect: Extract information and source code from live objects."><code class="xref py py-mod docutils literal">inspect</code></a> and <a class="reference internal" href="../library/pydoc.html#module-pydoc" title="pydoc: Documentation generator and online help system."><code class="xref py py-mod docutils literal">pydoc</code></a> modules.</li>

428

<li><a class="reference internal" href="../reference/datamodel.html#object.__length_hint__" title="object.__length_hint__"><code class="xref py py-meth docutils literal">__length_hint__()</code></a> is now part of the formal language

429

specification (see <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0424">PEP 424</a>). (Contributed by Armin Ronacher in

430

<a class="reference external" href="https://bugs.python.org/issue16148">issue 16148</a>.)</li>

431

</ul>

432

</div>

433

</div>

434

435

<h2>New Modules<a class="headerlink" href="#new-modules" title="Permalink to this headline">¶</a></h2>

436

437

<h3>asyncio<a class="headerlink" href="#asyncio" title="Permalink to this headline">¶</a></h3>

438

The new <a class="reference internal" href="../library/asyncio.html#module-asyncio" title="asyncio: Asynchronous I/O, event loop, coroutines and tasks."><code class="xref py py-mod docutils literal">asyncio</code></a> module (defined in <a class="pep reference external" href="https://www.python.org/dev/peps/pep-3156">PEP 3156</a>) provides a standard

439

pluggable event loop model for Python, providing solid asynchronous IO

440

support in the standard library, and making it easier for other event loop

441

implementations to interoperate with the standard library and each other.

442

For Python 3.4, this module is considered a <a class="reference internal" href="../glossary.html#term-provisional-api">provisional API</a>.

443

444

See also

445

446

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-3156">PEP 3156</a> – Asynchronous IO Support Rebooted: the “asyncio” Module</dt>

447

<dd>PEP written and implementation led by Guido van Rossum.</dd>

448

</dl>

449

</div>

450

</div>

451

452

<h3>ensurepip<a class="headerlink" href="#ensurepip" title="Permalink to this headline">¶</a></h3>

453

454

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0453">PEP 453</a> implementation. In the normal course of events end users will not

455

need to interact with this module, but it can be used to manually bootstrap

456

<code class="docutils literal">pip</code> if the automated bootstrapping into an installation or virtual

457

environment was declined.

458

<a class="reference internal" href="../library/ensurepip.html#module-ensurepip" title="ensurepip: Bootstrapping the "pip" installer into an existing Python installation or virtual environment."><code class="xref py py-mod docutils literal">ensurepip</code></a> includes a bundled copy of <code class="docutils literal">pip</code>, up-to-date as of the first

459

release candidate of the release of CPython with which it ships (this applies

460

to both maintenance releases and feature releases). <code class="docutils literal">ensurepip</code> does not

461

access the internet. If the installation has Internet access, after

462

<code class="docutils literal">ensurepip</code> is run the bundled <code class="docutils literal">pip</code> can be used to upgrade <code class="docutils literal">pip</code> to a

463

more recent release than the bundled one. (Note that such an upgraded version

464

of <code class="docutils literal">pip</code> is considered to be a separately installed package and will not be

465

removed if Python is uninstalled.)

466

The module is named ensurepip because if called when <code class="docutils literal">pip</code> is already

467

installed, it does nothing. It also has an <code class="docutils literal">--upgrade</code> option that will

468

cause it to install the bundled copy of <code class="docutils literal">pip</code> if the existing installed

469

version of <code class="docutils literal">pip</code> is older than the bundled copy.

470

</div>

471

472

473

The new <a class="reference internal" href="../library/enum.html#module-enum" title="enum: Implementation of an enumeration class."><code class="xref py py-mod docutils literal">enum</code></a> module (defined in <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0435">PEP 435</a>) provides a standard

474

implementation of enumeration types, allowing other modules (such as

475

<a class="reference internal" href="../library/socket.html#module-socket" title="socket: Low-level networking interface."><code class="xref py py-mod docutils literal">socket</code></a>) to provide more informative error messages and better

476

debugging support by replacing opaque integer constants with backwards

477

compatible enumeration values.

478

479

See also

480

481

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0435">PEP 435</a> – Adding an Enum type to the Python standard library</dt>

482

<dd>PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman,

483

implemented by Ethan Furman.</dd>

484

</dl>

485

</div>

486

</div>

487

488

<h3>pathlib<a class="headerlink" href="#pathlib" title="Permalink to this headline">¶</a></h3>

489

The new <a class="reference internal" href="../library/pathlib.html#module-pathlib" title="pathlib: Object-oriented filesystem paths"><code class="xref py py-mod docutils literal">pathlib</code></a> module offers classes representing filesystem paths

490

with semantics appropriate for different operating systems. Path classes are

491

divided between pure paths, which provide purely computational operations

492

without I/O, and concrete paths, which inherit from pure paths but also

493

provide I/O operations.

494

For Python 3.4, this module is considered a <a class="reference internal" href="../glossary.html#term-provisional-api">provisional API</a>.

495

496

See also

497

498

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0428">PEP 428</a> – The pathlib module – object-oriented filesystem paths</dt>

499

<dd>PEP written and implemented by Antoine Pitrou.</dd>

500

</dl>

501

</div>

502

</div>

503

504

<h3>selectors<a class="headerlink" href="#selectors" title="Permalink to this headline">¶</a></h3>

505

The new <a class="reference internal" href="../library/selectors.html#module-selectors" title="selectors: High-level I/O multiplexing."><code class="xref py py-mod docutils literal">selectors</code></a> module (created as part of implementing <a class="pep reference external" href="https://www.python.org/dev/peps/pep-3156">PEP 3156</a>)

506

allows high-level and efficient I/O multiplexing, built upon the

507

<a class="reference internal" href="../library/select.html#module-select" title="select: Wait for I/O completion on multiple streams."><code class="xref py py-mod docutils literal">select</code></a> module primitives.

508

</div>

509

510

<h3>statistics<a class="headerlink" href="#statistics" title="Permalink to this headline">¶</a></h3>

511

The new <a class="reference internal" href="../library/statistics.html#module-statistics" title="statistics: mathematical statistics functions"><code class="xref py py-mod docutils literal">statistics</code></a> module (defined in <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0450">PEP 450</a>) offers some core

512

statistics functionality directly in the standard library. This module

513

supports calculation of the mean, median, mode, variance and standard

514

deviation of a data series.

515

516

See also

517

518

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0450">PEP 450</a> – Adding A Statistics Module To The Standard Library</dt>

519

<dd>PEP written and implemented by Steven D’Aprano</dd>

520

</dl>

521

</div>

522

</div>

523

524

<h3>tracemalloc<a class="headerlink" href="#tracemalloc" title="Permalink to this headline">¶</a></h3>

525

The new <a class="reference internal" href="../library/tracemalloc.html#module-tracemalloc" title="tracemalloc: Trace memory allocations."><code class="xref py py-mod docutils literal">tracemalloc</code></a> module (defined in <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0454">PEP 454</a>) is a debug tool to

526

trace memory blocks allocated by Python. It provides the following information:

527

528

<li>Trace where an object was allocated</li>

529

<li>Statistics on allocated memory blocks per filename and per line number:

530

total size, number and average size of allocated memory blocks</li>

531

<li>Compute the differences between two snapshots to detect memory leaks</li>

532

</ul>

533

534

See also

535

536

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0454">PEP 454</a> – Add a new tracemalloc module to trace Python memory allocations</dt>

537

<dd>PEP written and implemented by Victor Stinner</dd>

538

</dl>

539

</div>

540

</div>

541

</div>

542

543

<h2>Improved Modules<a class="headerlink" href="#improved-modules" title="Permalink to this headline">¶</a></h2>

544

545

546

New function <a class="reference internal" href="../library/abc.html#abc.get_cache_token" title="abc.get_cache_token"><code class="xref py py-func docutils literal">abc.get_cache_token()</code></a> can be used to know when to invalidate

547

caches that are affected by changes in the object graph. (Contributed

548

by Łukasz Langa in <a class="reference external" href="https://bugs.python.org/issue16832">issue 16832</a>.)

549

New class <a class="reference internal" href="../library/abc.html#abc.ABC" title="abc.ABC"><code class="xref py py-class docutils literal">ABC</code></a> has <a class="reference internal" href="../library/abc.html#abc.ABCMeta" title="abc.ABCMeta"><code class="xref py py-class docutils literal">ABCMeta</code></a> as its meta class.

550

Using <code class="docutils literal">ABC</code> as a base class has essentially the same effect as specifying

551

<code class="docutils literal">metaclass=abc.ABCMeta</code>, but is simpler to type and easier to read.

552

(Contributed by Bruno Dupuis in <a class="reference external" href="https://bugs.python.org/issue16049">issue 16049</a>.)

553

</div>

554

555

556

The <a class="reference internal" href="../library/aifc.html#aifc.aifc.getparams" title="aifc.aifc.getparams"><code class="xref py py-meth docutils literal">getparams()</code></a> method now returns a namedtuple rather than a

557

plain tuple. (Contributed by Claudiu Popa in <a class="reference external" href="https://bugs.python.org/issue17818">issue 17818</a>.)

558

<a class="reference internal" href="../library/aifc.html#aifc.open" title="aifc.open"><code class="xref py py-func docutils literal">aifc.open()</code></a> now supports the context management protocol: when used in a

559

<a class="reference internal" href="../reference/compound_stmts.html#with"><code class="xref std std-keyword docutils literal">with</code></a> block, the <a class="reference internal" href="../library/aifc.html#aifc.aifc.close" title="aifc.aifc.close"><code class="xref py py-meth docutils literal">close()</code></a> method of the returned

560

object will be called automatically at the end of the block. (Contributed by

561

Serhiy Storchacha in <a class="reference external" href="https://bugs.python.org/issue16486">issue 16486</a>.)

562

The <a class="reference internal" href="../library/aifc.html#aifc.aifc.writeframesraw" title="aifc.aifc.writeframesraw"><code class="xref py py-meth docutils literal">writeframesraw()</code></a> and <a class="reference internal" href="../library/aifc.html#aifc.aifc.writeframes" title="aifc.aifc.writeframes"><code class="xref py py-meth docutils literal">writeframes()</code></a>

563

methods now accept any <a class="reference internal" href="../glossary.html#term-bytes-like-object">bytes-like object</a>. (Contributed by Serhiy

564

Storchaka in <a class="reference external" href="https://bugs.python.org/issue8311">issue 8311</a>.)

565

</div>

566

567

<h3>argparse<a class="headerlink" href="#argparse" title="Permalink to this headline">¶</a></h3>

568

The <a class="reference internal" href="../library/argparse.html#argparse.FileType" title="argparse.FileType"><code class="xref py py-class docutils literal">FileType</code></a> class now accepts encoding and

569

errors arguments, which are passed through to <a class="reference internal" href="../library/functions.html#open" title="open"><code class="xref py py-func docutils literal">open()</code></a>. (Contributed

570

by Lucas Maystre in <a class="reference external" href="https://bugs.python.org/issue11175">issue 11175</a>.)

571

</div>

572

573

<h3>audioop<a class="headerlink" href="#audioop" title="Permalink to this headline">¶</a></h3>

574

<a class="reference internal" href="../library/audioop.html#module-audioop" title="audioop: Manipulate raw audio data."><code class="xref py py-mod docutils literal">audioop</code></a> now supports 24-bit samples. (Contributed by Serhiy Storchaka

575

in <a class="reference external" href="https://bugs.python.org/issue12866">issue 12866</a>.)

576

New <a class="reference internal" href="../library/audioop.html#audioop.byteswap" title="audioop.byteswap"><code class="xref py py-func docutils literal">byteswap()</code></a> function converts big-endian samples to

577

little-endian and vice versa. (Contributed by Serhiy Storchaka in

578

<a class="reference external" href="https://bugs.python.org/issue19641">issue 19641</a>.)

579

All <a class="reference internal" href="../library/audioop.html#module-audioop" title="audioop: Manipulate raw audio data."><code class="xref py py-mod docutils literal">audioop</code></a> functions now accept any <a class="reference internal" href="../glossary.html#term-bytes-like-object">bytes-like object</a>. Strings

580

are not accepted: they didn’t work before, now they raise an error right away.

581

(Contributed by Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue16685">issue 16685</a>.)

582

</div>

583

584

585

The encoding and decoding functions in <a class="reference internal" href="../library/base64.html#module-base64" title="base64: RFC 3548: Base16, Base32, Base64 Data Encodings; Base85 and Ascii85"><code class="xref py py-mod docutils literal">base64</code></a> now accept any

586

<a class="reference internal" href="../glossary.html#term-bytes-like-object">bytes-like object</a> in cases where it previously required a

587

<a class="reference internal" href="../library/functions.html#bytes" title="bytes"><code class="xref py py-class docutils literal">bytes</code></a> or <a class="reference internal" href="../library/functions.html#bytearray" title="bytearray"><code class="xref py py-class docutils literal">bytearray</code></a> instance. (Contributed by Nick Coghlan in

588

<a class="reference external" href="https://bugs.python.org/issue17839">issue 17839</a>.)

589

New functions <a class="reference internal" href="../library/base64.html#base64.a85encode" title="base64.a85encode"><code class="xref py py-func docutils literal">a85encode()</code></a>, <a class="reference internal" href="../library/base64.html#base64.a85decode" title="base64.a85decode"><code class="xref py py-func docutils literal">a85decode()</code></a>,

590

<a class="reference internal" href="../library/base64.html#base64.b85encode" title="base64.b85encode"><code class="xref py py-func docutils literal">b85encode()</code></a>, and <a class="reference internal" href="../library/base64.html#base64.b85decode" title="base64.b85decode"><code class="xref py py-func docutils literal">b85decode()</code></a> provide the ability to

591

encode and decode binary data from and to <code class="docutils literal">Ascii85</code> and the git/mercurial

592

<code class="docutils literal">Base85</code> formats, respectively. The <code class="docutils literal">a85</code> functions have options that can

593

be used to make them compatible with the variants of the <code class="docutils literal">Ascii85</code> encoding,

594

including the Adobe variant. (Contributed by Martin Morrison, the Mercurial

595

project, Serhiy Storchaka, and Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue17618">issue 17618</a>.)

596

</div>

597

598

<h3>collections<a class="headerlink" href="#collections" title="Permalink to this headline">¶</a></h3>

599

The <a class="reference internal" href="../library/collections.html#collections.ChainMap.new_child" title="collections.ChainMap.new_child"><code class="xref py py-meth docutils literal">ChainMap.new_child()</code></a> method now accepts an m argument specifying

600

the child map to add to the chain. This allows an existing mapping and/or a

601

custom mapping type to be used for the child. (Contributed by Vinay Sajip in

602

<a class="reference external" href="https://bugs.python.org/issue16613">issue 16613</a>.)

603

</div>

604

605

<h3>colorsys<a class="headerlink" href="#colorsys" title="Permalink to this headline">¶</a></h3>

606

The number of digits in the coefficients for the RGB — YIQ conversions have

607

been expanded so that they match the FCC NTSC versions. The change in

608

results should be less than 1% and may better match results found elsewhere.

609

(Contributed by Brian Landers and Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue14323">issue 14323</a>.)

610

</div>

611

612

<h3>contextlib<a class="headerlink" href="#contextlib" title="Permalink to this headline">¶</a></h3>

613

The new <a class="reference internal" href="../library/contextlib.html#contextlib.suppress" title="contextlib.suppress"><code class="xref py py-class docutils literal">contextlib.suppress</code></a> context manager helps to clarify the

614

intent of code that deliberately suppresses exceptions from a single

615

statement. (Contributed by Raymond Hettinger in <a class="reference external" href="https://bugs.python.org/issue15806">issue 15806</a> and

616

Zero Piraeus in <a class="reference external" href="https://bugs.python.org/issue19266">issue 19266</a>.)

617

The new <a class="reference internal" href="../library/contextlib.html#contextlib.redirect_stdout" title="contextlib.redirect_stdout"><code class="xref py py-func docutils literal">contextlib.redirect_stdout()</code></a> context manager makes it easier

618

for utility scripts to handle inflexible APIs that write their output to

619

<a class="reference internal" href="../library/sys.html#sys.stdout" title="sys.stdout"><code class="xref py py-data docutils literal">sys.stdout</code></a> and don’t provide any options to redirect it. Using the

620

context manager, the <a class="reference internal" href="../library/sys.html#sys.stdout" title="sys.stdout"><code class="xref py py-data docutils literal">sys.stdout</code></a> output can be redirected to any

621

other stream or, in conjunction with <a class="reference internal" href="../library/io.html#io.StringIO" title="io.StringIO"><code class="xref py py-class docutils literal">io.StringIO</code></a>, to a string.

622

The latter can be especially useful, for example, to capture output

623

from a function that was written to implement a command line interface.

624

It is recommended only for utility scripts because it affects the

625

global state of <a class="reference internal" href="../library/sys.html#sys.stdout" title="sys.stdout"><code class="xref py py-data docutils literal">sys.stdout</code></a>. (Contributed by Raymond Hettinger

626

in <a class="reference external" href="https://bugs.python.org/issue15805">issue 15805</a>.)

627

The <a class="reference internal" href="../library/contextlib.html#module-contextlib" title="contextlib: Utilities for with-statement contexts."><code class="xref py py-mod docutils literal">contextlib</code></a> documentation has also been updated to include a

628

<a class="reference internal" href="../library/contextlib.html#single-use-reusable-and-reentrant-cms">discussion</a> of the

629

differences between single use, reusable and reentrant context managers.

630

</div>

631

632

633

<a class="reference internal" href="../library/dbm.html#dbm.open" title="dbm.open"><code class="xref py py-func docutils literal">dbm.open()</code></a> objects now support the context management protocol. When

634

used in a <a class="reference internal" href="../reference/compound_stmts.html#with"><code class="xref std std-keyword docutils literal">with</code></a> statement, the <code class="docutils literal">close</code> method of the database

635

object will be called automatically at the end of the block. (Contributed by

636

Claudiu Popa and Nick Coghlan in <a class="reference external" href="https://bugs.python.org/issue19282">issue 19282</a>.)

637

</div>

638

639

640

Functions <a class="reference internal" href="../library/dis.html#dis.show_code" title="dis.show_code"><code class="xref py py-func docutils literal">show_code()</code></a>, <a class="reference internal" href="../library/dis.html#dis.dis" title="dis.dis"><code class="xref py py-func docutils literal">dis()</code></a>, <a class="reference internal" href="../library/dis.html#dis.distb" title="dis.distb"><code class="xref py py-func docutils literal">distb()</code></a>, and

641

<a class="reference internal" href="../library/dis.html#dis.disassemble" title="dis.disassemble"><code class="xref py py-func docutils literal">disassemble()</code></a> now accept a keyword-only file argument that

642

controls where they write their output.

643

The <a class="reference internal" href="../library/dis.html#module-dis" title="dis: Disassembler for Python bytecode."><code class="xref py py-mod docutils literal">dis</code></a> module is now built around an <a class="reference internal" href="../library/dis.html#dis.Instruction" title="dis.Instruction"><code class="xref py py-class docutils literal">Instruction</code></a> class

644

that provides object oriented access to the details of each individual bytecode

645

operation.

646

A new method, <a class="reference internal" href="../library/dis.html#dis.get_instructions" title="dis.get_instructions"><code class="xref py py-func docutils literal">get_instructions()</code></a>, provides an iterator that emits

647

the Instruction stream for a given piece of Python code. Thus it is now

648

possible to write a program that inspects and manipulates a bytecode

649

object in ways different from those provided by the <a class="reference internal" href="../library/dis.html#module-dis" title="dis: Disassembler for Python bytecode."><code class="xref py py-mod docutils literal">dis</code></a> module

650

itself. For example:

651

<div class="highlight-python3"><div class="highlight"><pre>>>> import dis

652

>>> for instr in dis.get_instructions(lambda x: x + 1):

653

... print(instr.opname)

654

LOAD_FAST

655

LOAD_CONST

656

BINARY_ADD

657

RETURN_VALUE

658

</pre></div>

659

</div>

660

The various display tools in the <a class="reference internal" href="../library/dis.html#module-dis" title="dis: Disassembler for Python bytecode."><code class="xref py py-mod docutils literal">dis</code></a> module have been rewritten to use

661

these new components.

662

In addition, a new application-friendly class <a class="reference internal" href="../library/dis.html#dis.Bytecode" title="dis.Bytecode"><code class="xref py py-class docutils literal">Bytecode</code></a> provides

663

an object-oriented API for inspecting bytecode in both in human-readable form

664

and for iterating over instructions. The <a class="reference internal" href="../library/dis.html#dis.Bytecode" title="dis.Bytecode"><code class="xref py py-class docutils literal">Bytecode</code></a> constructor

665

takes the same arguments that <code class="xref py py-func docutils literal">get_instruction()</code> does (plus an

666

optional current_offset), and the resulting object can be iterated to produce

667

<a class="reference internal" href="../library/dis.html#dis.Instruction" title="dis.Instruction"><code class="xref py py-class docutils literal">Instruction</code></a> objects. But it also has a <a class="reference internal" href="../library/dis.html#dis.Bytecode.dis" title="dis.Bytecode.dis"><code class="xref py py-mod docutils literal">dis</code></a>

668

method, equivalent to calling <a class="reference internal" href="../library/dis.html#dis.dis" title="dis.dis"><code class="xref py py-mod docutils literal">dis</code></a> on the constructor argument, but

669

returned as a multi-line string:

670

<div class="highlight-python3"><div class="highlight"><pre>>>> bytecode = dis.Bytecode(lambda x: x + 1, current_offset=3)

671

>>> for instr in bytecode:

672

... print('{} ({})'.format(instr.opname, instr.opcode))

673

LOAD_FAST (124)

674

LOAD_CONST (100)

675

BINARY_ADD (23)

676

RETURN_VALUE (83)

677

>>> bytecode.dis().splitlines()

678

[' 1 0 LOAD_FAST 0 (x)',

679

' --> 3 LOAD_CONST 1 (1)',

680

' 6 BINARY_ADD',

681

' 7 RETURN_VALUE']

682

</pre></div>

683

</div>

684

<a class="reference internal" href="../library/dis.html#dis.Bytecode" title="dis.Bytecode"><code class="xref py py-class docutils literal">Bytecode</code></a> also has a class method,

685

<a class="reference internal" href="../library/dis.html#dis.Bytecode.from_traceback" title="dis.Bytecode.from_traceback"><code class="xref py py-meth docutils literal">from_traceback()</code></a>, that provides the ability to manipulate a

686

traceback (that is, <code class="docutils literal">print(Bytecode.from_traceback(tb).dis())</code> is equivalent

687

to <code class="docutils literal">distb(tb)</code>).

688

(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in <a class="reference external" href="https://bugs.python.org/issue11816">issue 11816</a>

689

and Claudiu Popa in <a class="reference external" href="https://bugs.python.org/issue17916">issue 17916</a>.)

690

New function <a class="reference internal" href="../library/dis.html#dis.stack_effect" title="dis.stack_effect"><code class="xref py py-func docutils literal">stack_effect()</code></a> computes the effect on the Python stack

691

of a given opcode and argument, information that is not otherwise available.

692

(Contributed by Larry Hastings in <a class="reference external" href="https://bugs.python.org/issue19722">issue 19722</a>.)

693

</div>

694

695

<h3>doctest<a class="headerlink" href="#doctest" title="Permalink to this headline">¶</a></h3>

696

A new <a class="reference internal" href="../library/doctest.html#doctest-options">option flag</a>, <a class="reference internal" href="../library/doctest.html#doctest.FAIL_FAST" title="doctest.FAIL_FAST"><code class="xref py py-data docutils literal">FAIL_FAST</code></a>, halts

697

test running as soon as the first failure is detected. (Contributed by R.

698

David Murray and Daniel Urban in <a class="reference external" href="https://bugs.python.org/issue16522">issue 16522</a>.)

699

The <a class="reference internal" href="../library/doctest.html#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> command line interface now uses <a class="reference internal" href="../library/argparse.html#module-argparse" title="argparse: Command-line option and argument parsing library."><code class="xref py py-mod docutils literal">argparse</code></a>, and has two

700

new options, <code class="docutils literal">-o</code> and <code class="docutils literal">-f</code>. <code class="docutils literal">-o</code> allows <a class="reference internal" href="../library/doctest.html#doctest-options">doctest options</a> to be specified on the command line, and <code class="docutils literal">-f</code> is a

701

shorthand for <code class="docutils literal">-o FAIL_FAST</code> (to parallel the similar option supported by the

702

<a class="reference internal" href="../library/unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> CLI). (Contributed by R. David Murray in <a class="reference external" href="https://bugs.python.org/issue11390">issue 11390</a>.)

703

<a class="reference internal" href="../library/doctest.html#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> will now find doctests in extension module <code class="docutils literal">__doc__</code> strings.

704

(Contributed by Zachary Ware in <a class="reference external" href="https://bugs.python.org/issue3158">issue 3158</a>.)

705

</div>

706

707

<h3>email<a class="headerlink" href="#email" title="Permalink to this headline">¶</a></h3>

708

<a class="reference internal" href="../library/email.message.html#email.message.Message.as_string" title="email.message.Message.as_string"><code class="xref py py-meth docutils literal">as_string()</code></a> now accepts a policy argument to

709

override the default policy of the message when generating a string

710

representation of it. This means that <code class="docutils literal">as_string</code> can now be used in more

711

circumstances, instead of having to create and use a <a class="reference internal" href="../library/email.generator.html#module-email.generator" title="email.generator: Generate flat text email messages from a message structure."><code class="xref py py-mod docutils literal">generator</code></a> in

712

order to pass formatting parameters to its <code class="docutils literal">flatten</code> method. (Contributed by

713

R. David Murray in <a class="reference external" href="https://bugs.python.org/issue18600">issue 18600</a>.)

714

New method <a class="reference internal" href="../library/email.message.html#email.message.Message.as_bytes" title="email.message.Message.as_bytes"><code class="xref py py-meth docutils literal">as_bytes()</code></a> added to produce a bytes

715

representation of the message in a fashion similar to how <code class="docutils literal">as_string</code>

716

produces a string representation. It does not accept the maxheaderlen

717

argument, but does accept the unixfrom and policy arguments. The

718

<a class="reference internal" href="../library/email.message.html#email.message.Message" title="email.message.Message"><code class="xref py py-class docutils literal">Message</code></a> <a class="reference internal" href="../library/email.message.html#email.message.Message.__bytes__" title="email.message.Message.__bytes__"><code class="xref py py-meth docutils literal">__bytes__()</code></a> method

719

calls it, meaning that <code class="docutils literal">bytes(mymsg)</code> will now produce the intuitive

720

result: a bytes object containing the fully formatted message. (Contributed

721

by R. David Murray in <a class="reference external" href="https://bugs.python.org/issue18600">issue 18600</a>.)

722

The <a class="reference internal" href="../library/email.message.html#email.message.Message.set_param" title="email.message.Message.set_param"><code class="xref py py-meth docutils literal">Message.set_param()</code></a> message now accepts a replace keyword argument.

723

When specified, the associated header will be updated without changing

724

its location in the list of headers. For backward compatibility, the default

725

is <code class="docutils literal">False</code>. (Contributed by R. David Murray in <a class="reference external" href="https://bugs.python.org/issue18891">issue 18891</a>.)

726

A pair of new subclasses of <a class="reference internal" href="../library/email.message.html#email.message.Message" title="email.message.Message"><code class="xref py py-class docutils literal">Message</code></a> have been added

727

(<a class="reference internal" href="../library/email.contentmanager.html#email.message.EmailMessage" title="email.message.EmailMessage"><code class="xref py py-class docutils literal">EmailMessage</code></a> and <a class="reference internal" href="../library/email.contentmanager.html#email.message.MIMEPart" title="email.message.MIMEPart"><code class="xref py py-class docutils literal">MIMEPart</code></a>), along with a new sub-module,

728

<a class="reference internal" href="../library/email.contentmanager.html#module-email.contentmanager" title="email.contentmanager: Storing and Retrieving Content from MIME Parts"><code class="xref py py-mod docutils literal">contentmanager</code></a> and a new <a class="reference internal" href="../library/email.policy.html#module-email.policy" title="email.policy: Controlling the parsing and generating of messages"><code class="xref py py-mod docutils literal">policy</code></a> attribute

729

<a class="reference internal" href="../library/email.policy.html#email.policy.EmailPolicy.content_manager" title="email.policy.EmailPolicy.content_manager"><code class="xref py py-attr docutils literal">content_manager</code></a>. All documentation is

730

currently in the new module, which is being added as part of email’s new

731

<a class="reference internal" href="../glossary.html#term-provisional-api">provisional API</a>. These classes provide a number of new methods that

732

make extracting content from and inserting content into email messages much

733

easier. For details, see the <a class="reference internal" href="../library/email.contentmanager.html#module-email.contentmanager" title="email.contentmanager: Storing and Retrieving Content from MIME Parts"><code class="xref py py-mod docutils literal">contentmanager</code></a> documentation and

734

the <a class="reference internal" href="../library/email-examples.html#email-contentmanager-api-examples">Examples using the Provisional API</a>. These API additions complete the

735

bulk of the work that was planned as part of the email6 project. The currently

736

provisional API is scheduled to become final in Python 3.5 (possibly with a few

737

minor additions in the area of error handling). (Contributed by R. David

738

Murray in <a class="reference external" href="https://bugs.python.org/issue18891">issue 18891</a>.)

739

</div>

740

741

<h3>filecmp<a class="headerlink" href="#filecmp" title="Permalink to this headline">¶</a></h3>

742

A new <a class="reference internal" href="../library/filecmp.html#filecmp.clear_cache" title="filecmp.clear_cache"><code class="xref py py-func docutils literal">clear_cache()</code></a> function provides the ability to clear the

743

<a class="reference internal" href="../library/filecmp.html#module-filecmp" title="filecmp: Compare files efficiently."><code class="xref py py-mod docutils literal">filecmp</code></a> comparison cache, which uses <a class="reference internal" href="../library/os.html#os.stat" title="os.stat"><code class="xref py py-func docutils literal">os.stat()</code></a> information to

744

determine if the file has changed since the last compare. This can be used,

745

for example, if the file might have been changed and re-checked in less time

746

than the resolution of a particular filesystem’s file modification time field.

747

(Contributed by Mark Levitt in <a class="reference external" href="https://bugs.python.org/issue18149">issue 18149</a>.)

748

New module attribute <a class="reference internal" href="../library/filecmp.html#filecmp.DEFAULT_IGNORES" title="filecmp.DEFAULT_IGNORES"><code class="xref py py-data docutils literal">DEFAULT_IGNORES</code></a> provides the list of

749

directories that are used as the default value for the ignore parameter of

750

the <a class="reference internal" href="../library/filecmp.html#filecmp.dircmp" title="filecmp.dircmp"><code class="xref py py-func docutils literal">dircmp()</code></a> function. (Contributed by Eli Bendersky in

751

<a class="reference external" href="https://bugs.python.org/issue15442">issue 15442</a>.)

752

</div>

753

754

<h3>functools<a class="headerlink" href="#functools" title="Permalink to this headline">¶</a></h3>

755

The new <a class="reference internal" href="../library/functools.html#functools.partialmethod" title="functools.partialmethod"><code class="xref py py-func docutils literal">partialmethod()</code></a> descriptor brings partial argument

756

application to descriptors, just as <a class="reference internal" href="../library/functools.html#functools.partial" title="functools.partial"><code class="xref py py-func docutils literal">partial()</code></a> provides

757

for normal callables. The new descriptor also makes it easier to get

758

arbitrary callables (including <a class="reference internal" href="../library/functools.html#functools.partial" title="functools.partial"><code class="xref py py-func docutils literal">partial()</code></a> instances)

759

to behave like normal instance methods when included in a class definition.

760

(Contributed by Alon Horev and Nick Coghlan in <a class="reference external" href="https://bugs.python.org/issue4331">issue 4331</a>.)

761

The new <a class="reference internal" href="../library/functools.html#functools.singledispatch" title="functools.singledispatch"><code class="xref py py-func docutils literal">singledispatch()</code></a> decorator brings support for

762

single-dispatch generic functions to the Python standard library. Where

763

object oriented programming focuses on grouping multiple operations on a

764

common set of data into a class, a generic function focuses on grouping

765

multiple implementations of an operation that allows it to work with

766

different kinds of data.

767

768

See also

769

770

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0443">PEP 443</a> – Single-dispatch generic functions</dt>

771

<dd>PEP written and implemented by Łukasz Langa.</dd>

772

</dl>

773

</div>

774

<a class="reference internal" href="../library/functools.html#functools.total_ordering" title="functools.total_ordering"><code class="xref py py-func docutils literal">total_ordering()</code></a> now supports a return value of

775

<a class="reference internal" href="../library/constants.html#NotImplemented" title="NotImplemented"><code class="xref py py-const docutils literal">NotImplemented</code></a> from the underlying comparison function. (Contributed

776

by Katie Miller in <a class="reference external" href="https://bugs.python.org/issue10042">issue 10042</a>.)

777

A pure-python version of the <a class="reference internal" href="../library/functools.html#functools.partial" title="functools.partial"><code class="xref py py-func docutils literal">partial()</code></a> function is now in the

778

stdlib; in CPython it is overridden by the C accelerated version, but it is

779

available for other implementations to use. (Contributed by Brian Thorne in

780

<a class="reference external" href="https://bugs.python.org/issue12428">issue 12428</a>.)

781

</div>

782

783

784

New function <a class="reference internal" href="../library/gc.html#gc.get_stats" title="gc.get_stats"><code class="xref py py-func docutils literal">get_stats()</code></a> returns a list of three per-generation

785

dictionaries containing the collections statistics since interpreter startup.

786

(Contributed by Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue16351">issue 16351</a>.)

787

</div>

788

789

790

A new function <a class="reference internal" href="../library/glob.html#glob.escape" title="glob.escape"><code class="xref py py-func docutils literal">escape()</code></a> provides a way to escape special characters

791

in a filename so that they do not become part of the globbing expansion but are

792

instead matched literally. (Contributed by Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue8402">issue 8402</a>.)

793

</div>

794

795

<h3>hashlib<a class="headerlink" href="#hashlib" title="Permalink to this headline">¶</a></h3>

796

A new <a class="reference internal" href="../library/hashlib.html#hashlib.pbkdf2_hmac" title="hashlib.pbkdf2_hmac"><code class="xref py py-func docutils literal">hashlib.pbkdf2_hmac()</code></a> function provides

797

the <a class="reference external" href="https://en.wikipedia.org/wiki/PBKDF2">PKCS#5 password-based key derivation function 2</a>. (Contributed by Christian

798

Heimes in <a class="reference external" href="https://bugs.python.org/issue18582">issue 18582</a>.)

799

The <a class="reference internal" href="../library/hashlib.html#hashlib.hash.name" title="hashlib.hash.name"><code class="xref py py-attr docutils literal">name</code></a> attribute of <a class="reference internal" href="../library/hashlib.html#module-hashlib" title="hashlib: Secure hash and message digest algorithms."><code class="xref py py-mod docutils literal">hashlib</code></a> hash objects is now

800

a formally supported interface. It has always existed in CPython’s

801

<a class="reference internal" href="../library/hashlib.html#module-hashlib" title="hashlib: Secure hash and message digest algorithms."><code class="xref py py-mod docutils literal">hashlib</code></a> (although it did not return lower case names for all supported

802

hashes), but it was not a public interface and so some other Python

803

implementations have not previously supported it. (Contributed by Jason R.

804

Coombs in <a class="reference external" href="https://bugs.python.org/issue18532">issue 18532</a>.)

805

</div>

806

807

808

<a class="reference internal" href="../library/hmac.html#module-hmac" title="hmac: Keyed-Hashing for Message Authentication (HMAC) implementation"><code class="xref py py-mod docutils literal">hmac</code></a> now accepts <code class="docutils literal">bytearray</code> as well as <code class="docutils literal">bytes</code> for the key

809

argument to the <a class="reference internal" href="../library/hmac.html#hmac.new" title="hmac.new"><code class="xref py py-func docutils literal">new()</code></a> function, and the msg parameter to both the

810

<a class="reference internal" href="../library/hmac.html#hmac.new" title="hmac.new"><code class="xref py py-func docutils literal">new()</code></a> function and the <a class="reference internal" href="../library/hmac.html#hmac.HMAC.update" title="hmac.HMAC.update"><code class="xref py py-meth docutils literal">update()</code></a> method now

811

accepts any type supported by the <a class="reference internal" href="../library/hashlib.html#module-hashlib" title="hashlib: Secure hash and message digest algorithms."><code class="xref py py-mod docutils literal">hashlib</code></a> module. (Contributed

812

by Jonas Borgström in <a class="reference external" href="https://bugs.python.org/issue18240">issue 18240</a>.)

813

The digestmod argument to the <a class="reference internal" href="../library/hmac.html#hmac.new" title="hmac.new"><code class="xref py py-func docutils literal">hmac.new()</code></a> function may now be any hash

814

digest name recognized by <a class="reference internal" href="../library/hashlib.html#module-hashlib" title="hashlib: Secure hash and message digest algorithms."><code class="xref py py-mod docutils literal">hashlib</code></a>. In addition, the current behavior in

815

which the value of digestmod defaults to <code class="docutils literal">MD5</code> is deprecated: in a

816

future version of Python there will be no default value. (Contributed by

817

Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue17276">issue 17276</a>.)

818

With the addition of <a class="reference internal" href="../library/hmac.html#hmac.HMAC.block_size" title="hmac.HMAC.block_size"><code class="xref py py-attr docutils literal">block_size</code></a> and <a class="reference internal" href="../library/hmac.html#hmac.HMAC.name" title="hmac.HMAC.name"><code class="xref py py-attr docutils literal">name</code></a>

819

attributes (and the formal documentation of the <a class="reference internal" href="../library/hmac.html#hmac.HMAC.digest_size" title="hmac.HMAC.digest_size"><code class="xref py py-attr docutils literal">digest_size</code></a>

820

attribute), the <a class="reference internal" href="../library/hmac.html#module-hmac" title="hmac: Keyed-Hashing for Message Authentication (HMAC) implementation"><code class="xref py py-mod docutils literal">hmac</code></a> module now conforms fully to the <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0247">PEP 247</a> API.

821

(Contributed by Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue18775">issue 18775</a>.)

822

</div>

823

824

825

New function <a class="reference internal" href="../library/html.html#html.unescape" title="html.unescape"><code class="xref py py-func docutils literal">unescape()</code></a> function converts HTML5 character references to

826

the corresponding Unicode characters. (Contributed by Ezio Melotti in

827

<a class="reference external" href="https://bugs.python.org/issue2927">issue 2927</a>.)

828

<a class="reference internal" href="../library/html.parser.html#html.parser.HTMLParser" title="html.parser.HTMLParser"><code class="xref py py-class docutils literal">HTMLParser</code></a> accepts a new keyword argument

829

convert_charrefs that, when <code class="docutils literal">True</code>, automatically converts all character

830

references. For backward-compatibility, its value defaults to <code class="docutils literal">False</code>, but

831

it will change to <code class="docutils literal">True</code> in a future version of Python, so you are invited to

832

set it explicitly and update your code to use this new feature. (Contributed

833

by Ezio Melotti in <a class="reference external" href="https://bugs.python.org/issue13633">issue 13633</a>.)

834

The strict argument of <a class="reference internal" href="../library/html.parser.html#html.parser.HTMLParser" title="html.parser.HTMLParser"><code class="xref py py-class docutils literal">HTMLParser</code></a> is now deprecated.

835

(Contributed by Ezio Melotti in <a class="reference external" href="https://bugs.python.org/issue15114">issue 15114</a>.)

836

</div>

837

838

839

<a class="reference internal" href="../library/http.server.html#http.server.BaseHTTPRequestHandler.send_error" title="http.server.BaseHTTPRequestHandler.send_error"><code class="xref py py-meth docutils literal">send_error()</code></a> now accepts an

840

optional additional explain parameter which can be used to provide an

841

extended error description, overriding the hardcoded default if there is one.

842

This extended error description will be formatted using the

843

<code class="xref py py-attr docutils literal">error_message_format</code> attribute and sent as the body

844

of the error response. (Contributed by Karl Cow in <a class="reference external" href="https://bugs.python.org/issue12921">issue 12921</a>.)

845

The <a class="reference internal" href="../library/http.server.html#module-http.server" title="http.server: HTTP server and request handlers."><code class="xref py py-mod docutils literal">http.server</code></a> <a class="reference internal" href="../library/http.server.html#http-server-cli">command line interface</a> now has

846

a <code class="docutils literal">-b/--bind</code> option that causes the server to listen on a specific address.

847

(Contributed by Malte Swart in <a class="reference external" href="https://bugs.python.org/issue17764">issue 17764</a>.)

848

</div>

849

850

<h3>idlelib and IDLE<a class="headerlink" href="#idlelib-and-idle" title="Permalink to this headline">¶</a></h3>

851

Since idlelib implements the IDLE shell and editor and is not intended for

852

import by other programs, it gets improvements with every release. See

853

<code class="file docutils literal">Lib/idlelib/NEWS.txt</code> for a cumulative list of changes since 3.3.0,

854

as well as changes made in future 3.4.x releases. This file is also available

855

from the IDLE Help ‣ About IDLE dialog.

856

</div>

857

858

<h3>importlib<a class="headerlink" href="#importlib" title="Permalink to this headline">¶</a></h3>

859

The <a class="reference internal" href="../library/importlib.html#importlib.abc.InspectLoader" title="importlib.abc.InspectLoader"><code class="xref py py-class docutils literal">InspectLoader</code></a> ABC defines a new method,

860

<a class="reference internal" href="../library/importlib.html#importlib.abc.InspectLoader.source_to_code" title="importlib.abc.InspectLoader.source_to_code"><code class="xref py py-meth docutils literal">source_to_code()</code></a> that accepts source

861

data and a path and returns a code object. The default implementation

862

is equivalent to <code class="docutils literal">compile(data, path, 'exec', dont_inherit=True)</code>.

863

(Contributed by Eric Snow and Brett Cannon in <a class="reference external" href="https://bugs.python.org/issue15627">issue 15627</a>.)

864

<a class="reference internal" href="../library/importlib.html#importlib.abc.InspectLoader" title="importlib.abc.InspectLoader"><code class="xref py py-class docutils literal">InspectLoader</code></a> also now has a default implementation

865

for the <a class="reference internal" href="../library/importlib.html#importlib.abc.InspectLoader.get_code" title="importlib.abc.InspectLoader.get_code"><code class="xref py py-meth docutils literal">get_code()</code></a> method. However,

866

it will normally be desirable to override the default implementation

867

for performance reasons. (Contributed by Brett Cannon in <a class="reference external" href="https://bugs.python.org/issue18072">issue 18072</a>.)

868

The <a class="reference internal" href="../library/importlib.html#importlib.reload" title="importlib.reload"><code class="xref py py-func docutils literal">reload()</code></a> function has been moved from <a class="reference internal" href="../library/imp.html#module-imp" title="imp: Access the implementation of the import statement. (deprecated)"><code class="xref py py-mod docutils literal">imp</code></a> to

869

<a class="reference internal" href="../library/importlib.html#module-importlib" title="importlib: The implementation of the import machinery."><code class="xref py py-mod docutils literal">importlib</code></a> as part of the <a class="reference internal" href="../library/imp.html#module-imp" title="imp: Access the implementation of the import statement. (deprecated)"><code class="xref py py-mod docutils literal">imp</code></a> module deprecation. (Contributed by

870

Berker Peksag in <a class="reference external" href="https://bugs.python.org/issue18193">issue 18193</a>.)

871

<a class="reference internal" href="../library/importlib.html#module-importlib.util" title="importlib.util: Utility code for importers"><code class="xref py py-mod docutils literal">importlib.util</code></a> now has a <a class="reference internal" href="../library/importlib.html#importlib.util.MAGIC_NUMBER" title="importlib.util.MAGIC_NUMBER"><code class="xref py py-data docutils literal">MAGIC_NUMBER</code></a> attribute

872

providing access to the bytecode version number. This replaces the

873

<a class="reference internal" href="../library/imp.html#imp.get_magic" title="imp.get_magic"><code class="xref py py-func docutils literal">get_magic()</code></a> function in the deprecated <a class="reference internal" href="../library/imp.html#module-imp" title="imp: Access the implementation of the import statement. (deprecated)"><code class="xref py py-mod docutils literal">imp</code></a> module.

874

(Contributed by Brett Cannon in <a class="reference external" href="https://bugs.python.org/issue18192">issue 18192</a>.)

875

New <a class="reference internal" href="../library/importlib.html#module-importlib.util" title="importlib.util: Utility code for importers"><code class="xref py py-mod docutils literal">importlib.util</code></a> functions <a class="reference internal" href="../library/importlib.html#importlib.util.cache_from_source" title="importlib.util.cache_from_source"><code class="xref py py-func docutils literal">cache_from_source()</code></a>

876

and <a class="reference internal" href="../library/importlib.html#importlib.util.source_from_cache" title="importlib.util.source_from_cache"><code class="xref py py-func docutils literal">source_from_cache()</code></a> replace the same-named functions

877

in the deprecated <a class="reference internal" href="../library/imp.html#module-imp" title="imp: Access the implementation of the import statement. (deprecated)"><code class="xref py py-mod docutils literal">imp</code></a> module. (Contributed by Brett Cannon in

878

<a class="reference external" href="https://bugs.python.org/issue18194">issue 18194</a>.)

879

The <a class="reference internal" href="../library/importlib.html#module-importlib" title="importlib: The implementation of the import machinery."><code class="xref py py-mod docutils literal">importlib</code></a> bootstrap <code class="xref py py-class docutils literal">NamespaceLoader</code> now conforms to

880

the <a class="reference internal" href="../library/importlib.html#importlib.abc.InspectLoader" title="importlib.abc.InspectLoader"><code class="xref py py-class docutils literal">InspectLoader</code></a> ABC, which means that <code class="docutils literal">runpy</code> and

881

<code class="docutils literal">python -m</code> can now be used with namespace packages. (Contributed

882

by Brett Cannon in <a class="reference external" href="https://bugs.python.org/issue18058">issue 18058</a>.)

883

<a class="reference internal" href="../library/importlib.html#module-importlib.util" title="importlib.util: Utility code for importers"><code class="xref py py-mod docutils literal">importlib.util</code></a> has a new function <a class="reference internal" href="../library/importlib.html#importlib.util.decode_source" title="importlib.util.decode_source"><code class="xref py py-func docutils literal">decode_source()</code></a>

884

that decodes source from bytes using universal newline processing. This is

885

useful for implementing <a class="reference internal" href="../library/importlib.html#importlib.abc.InspectLoader.get_source" title="importlib.abc.InspectLoader.get_source"><code class="xref py py-meth docutils literal">InspectLoader.get_source()</code></a> methods.

886

<a class="reference internal" href="../library/importlib.html#importlib.machinery.ExtensionFileLoader" title="importlib.machinery.ExtensionFileLoader"><code class="xref py py-class docutils literal">importlib.machinery.ExtensionFileLoader</code></a> now has a

887

<a class="reference internal" href="../library/importlib.html#importlib.machinery.ExtensionFileLoader.get_filename" title="importlib.machinery.ExtensionFileLoader.get_filename"><code class="xref py py-meth docutils literal">get_filename()</code></a> method. This was

888

inadvertently omitted in the original implementation. (Contributed by Eric

889

Snow in <a class="reference external" href="https://bugs.python.org/issue19152">issue 19152</a>.)

890

</div>

891

892

<h3>inspect<a class="headerlink" href="#inspect" title="Permalink to this headline">¶</a></h3>

893

The <a class="reference internal" href="../library/inspect.html#module-inspect" title="inspect: Extract information and source code from live objects."><code class="xref py py-mod docutils literal">inspect</code></a> module now offers a basic <a class="reference internal" href="../library/inspect.html#inspect-module-cli">command line interface</a> to quickly display source code and other

894

information for modules, classes and functions. (Contributed by Claudiu Popa

895

and Nick Coghlan in <a class="reference external" href="https://bugs.python.org/issue18626">issue 18626</a>.)

896

<a class="reference internal" href="../library/inspect.html#inspect.unwrap" title="inspect.unwrap"><code class="xref py py-func docutils literal">unwrap()</code></a> makes it easy to unravel wrapper function chains

897

created by <a class="reference internal" href="../library/functools.html#functools.wraps" title="functools.wraps"><code class="xref py py-func docutils literal">functools.wraps()</code></a> (and any other API that sets the

898

<code class="docutils literal">__wrapped__</code> attribute on a wrapper function). (Contributed by

899

Daniel Urban, Aaron Iles and Nick Coghlan in <a class="reference external" href="https://bugs.python.org/issue13266">issue 13266</a>.)

900

As part of the implementation of the new <a class="reference internal" href="../library/enum.html#module-enum" title="enum: Implementation of an enumeration class."><code class="xref py py-mod docutils literal">enum</code></a> module, the

901

<a class="reference internal" href="../library/inspect.html#module-inspect" title="inspect: Extract information and source code from live objects."><code class="xref py py-mod docutils literal">inspect</code></a> module now has substantially better support for custom

902

<code class="docutils literal">__dir__</code> methods and dynamic class attributes provided through

903

metaclasses. (Contributed by Ethan Furman in <a class="reference external" href="https://bugs.python.org/issue18929">issue 18929</a> and

904

<a class="reference external" href="https://bugs.python.org/issue19030">issue 19030</a>.)

905

<a class="reference internal" href="../library/inspect.html#inspect.getfullargspec" title="inspect.getfullargspec"><code class="xref py py-func docutils literal">getfullargspec()</code></a> and <a class="reference internal" href="../library/inspect.html#inspect.getargspec" title="inspect.getargspec"><code class="xref py py-func docutils literal">getargspec()</code></a>

906

now use the <a class="reference internal" href="../library/inspect.html#inspect.signature" title="inspect.signature"><code class="xref py py-func docutils literal">signature()</code></a> API. This allows them to

907

support a much broader range of callables, including those with

908

<code class="docutils literal">__signature__</code> attributes, those with metadata provided by argument

909

clinic, <a class="reference internal" href="../library/functools.html#functools.partial" title="functools.partial"><code class="xref py py-func docutils literal">functools.partial()</code></a> objects and more. Note that, unlike

910

<a class="reference internal" href="../library/inspect.html#inspect.signature" title="inspect.signature"><code class="xref py py-func docutils literal">signature()</code></a>, these functions still ignore <code class="docutils literal">__wrapped__</code>

911

attributes, and report the already bound first argument for bound methods,

912

so it is still necessary to update your code to use

913

914

(Contributed by Yury Selivanov in <a class="reference external" href="https://bugs.python.org/issue17481">issue 17481</a>.)

915

<a class="reference internal" href="../library/inspect.html#inspect.signature" title="inspect.signature"><code class="xref py py-func docutils literal">signature()</code></a> now supports duck types of CPython functions,

916

which adds support for functions compiled with Cython. (Contributed

917

by Stefan Behnel and Yury Selivanov in <a class="reference external" href="https://bugs.python.org/issue17159">issue 17159</a>.)

918

</div>

919

920

<h3>ipaddress<a class="headerlink" href="#ipaddress" title="Permalink to this headline">¶</a></h3>

921

<a class="reference internal" href="../library/ipaddress.html#module-ipaddress" title="ipaddress: IPv4/IPv6 manipulation library."><code class="xref py py-mod docutils literal">ipaddress</code></a> was added to the standard library in Python 3.3 as a

922

<a class="reference internal" href="../glossary.html#term-provisional-api">provisional API</a>. With the release of Python 3.4, this qualification

923

has been removed: <a class="reference internal" href="../library/ipaddress.html#module-ipaddress" title="ipaddress: IPv4/IPv6 manipulation library."><code class="xref py py-mod docutils literal">ipaddress</code></a> is now considered a stable API, covered

924

by the normal standard library requirements to maintain backwards

925

compatibility.

926

A new <a class="reference internal" href="../library/ipaddress.html#ipaddress.IPv4Address.is_global" title="ipaddress.IPv4Address.is_global"><code class="xref py py-attr docutils literal">is_global</code></a> property is <code class="docutils literal">True</code> if

927

an address is globally routeable. (Contributed by Peter Moody in

928

<a class="reference external" href="https://bugs.python.org/issue17400">issue 17400</a>.)

929

</div>

930

931

<h3>logging<a class="headerlink" href="#logging" title="Permalink to this headline">¶</a></h3>

932

The <a class="reference internal" href="../library/logging.handlers.html#logging.handlers.TimedRotatingFileHandler" title="logging.handlers.TimedRotatingFileHandler"><code class="xref py py-class docutils literal">TimedRotatingFileHandler</code></a> has a new atTime

933

parameter that can be used to specify the time of day when rollover should

934

happen. (Contributed by Ronald Oussoren in <a class="reference external" href="https://bugs.python.org/issue9556">issue 9556</a>.)

935

<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.SocketHandler" title="logging.handlers.SocketHandler"><code class="xref py py-class docutils literal">SocketHandler</code></a> and

936

<a class="reference internal" href="../library/logging.handlers.html#logging.handlers.DatagramHandler" title="logging.handlers.DatagramHandler"><code class="xref py py-class docutils literal">DatagramHandler</code></a> now support Unix domain sockets (by

937

setting port to <code class="docutils literal">None</code>). (Contributed by Vinay Sajip in commit

938

ce46195b56a9.)

939

<a class="reference internal" href="../library/logging.config.html#logging.config.fileConfig" title="logging.config.fileConfig"><code class="xref py py-func docutils literal">fileConfig()</code></a> now accepts a

940

<a class="reference internal" href="../library/configparser.html#configparser.RawConfigParser" title="configparser.RawConfigParser"><code class="xref py py-class docutils literal">configparser.RawConfigParser</code></a> subclass instance for the fname

941

parameter. This facilitates using a configuration file when logging

942

configuration is just a part of the overall application configuration, or where

943

the application modifies the configuration before passing it to

944

<a class="reference internal" href="../library/logging.config.html#logging.config.fileConfig" title="logging.config.fileConfig"><code class="xref py py-func docutils literal">fileConfig()</code></a>. (Contributed by Vinay Sajip in

945

<a class="reference external" href="https://bugs.python.org/issue16110">issue 16110</a>.)

946

Logging configuration data received from a socket via the

947

<a class="reference internal" href="../library/logging.config.html#logging.config.listen" title="logging.config.listen"><code class="xref py py-func docutils literal">logging.config.listen()</code></a> function can now be validated before being

948

processed by supplying a verification function as the argument to the new

949

verify keyword argument. (Contributed by Vinay Sajip in <a class="reference external" href="https://bugs.python.org/issue15452">issue 15452</a>.)

950

</div>

951

952

<h3>marshal<a class="headerlink" href="#marshal" title="Permalink to this headline">¶</a></h3>

953

The default <a class="reference internal" href="../library/marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal">marshal</code></a> version has been bumped to 3. The code implementing

954

the new version restores the Python2 behavior of recording only one copy of

955

interned strings and preserving the interning on deserialization, and extends

956

this “one copy” ability to any object type (including handling recursive

957

references). This reduces both the size of <code class="docutils literal">.pyc</code> files and the amount of

958

memory a module occupies in memory when it is loaded from a <code class="docutils literal">.pyc</code> (or

959

<code class="docutils literal">.pyo</code>) file. (Contributed by Kristján Valur Jónsson in <a class="reference external" href="https://bugs.python.org/issue16475">issue 16475</a>,

960

with additional speedups by Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue19219">issue 19219</a>.)

961

</div>

962

963

964

mmap objects can now be <a class="reference internal" href="../library/weakref.html#module-weakref" title="weakref: Support for weak references and weak dictionaries."><code class="xref py py-mod docutils literal">weakref</code></a>ed. (Contributed by Valerie Lambert in

965

<a class="reference external" href="https://bugs.python.org/issue4885">issue 4885</a>.)

966

</div>

967

968

<h3>multiprocessing<a class="headerlink" href="#multiprocessing" title="Permalink to this headline">¶</a></h3>

969

On Unix two new <a class="reference internal" href="../library/multiprocessing.html#multiprocessing-start-methods">start methods</a>,

970

<code class="docutils literal">spawn</code> and <code class="docutils literal">forkserver</code>, have been added for starting processes using

971

<a class="reference internal" href="../library/multiprocessing.html#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal">multiprocessing</code></a>. These make the mixing of processes with threads more

972

robust, and the <code class="docutils literal">spawn</code> method matches the semantics that multiprocessing has

973

always used on Windows. New function

974

<a class="reference internal" href="../library/multiprocessing.html#multiprocessing.get_all_start_methods" title="multiprocessing.get_all_start_methods"><code class="xref py py-func docutils literal">get_all_start_methods()</code></a> reports all start methods

975

available on the platform, <a class="reference internal" href="../library/multiprocessing.html#multiprocessing.get_start_method" title="multiprocessing.get_start_method"><code class="xref py py-func docutils literal">get_start_method()</code></a> reports

976

the current start method, and <a class="reference internal" href="../library/multiprocessing.html#multiprocessing.set_start_method" title="multiprocessing.set_start_method"><code class="xref py py-func docutils literal">set_start_method()</code></a> sets

977

the start method. (Contributed by Richard Oudkerk in <a class="reference external" href="https://bugs.python.org/issue8713">issue 8713</a>.)

978

979

determines how child processes are created. New function

980

<a class="reference internal" href="../library/multiprocessing.html#multiprocessing.get_context" title="multiprocessing.get_context"><code class="xref py py-func docutils literal">get_context()</code></a> returns a context that uses a specified

981

start method. It has the same API as the <a class="reference internal" href="../library/multiprocessing.html#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal">multiprocessing</code></a> module itself,

982

so you can use it to create <a class="reference internal" href="../library/multiprocessing.html#multiprocessing.pool.Pool" title="multiprocessing.pool.Pool"><code class="xref py py-class docutils literal">Pool</code></a>s and other

983

objects that will operate within that context. This allows a framework and an

984

application or different parts of the same application to use multiprocessing

985

without interfering with each other. (Contributed by Richard Oudkerk in

986

<a class="reference external" href="https://bugs.python.org/issue18999">issue 18999</a>.)

987

Except when using the old fork start method, child processes no longer

988

inherit unneeded handles/file descriptors from their parents (part of

989

<a class="reference external" href="https://bugs.python.org/issue8713">issue 8713</a>).

990

<a class="reference internal" href="../library/multiprocessing.html#module-multiprocessing" title="multiprocessing: Process-based parallelism."><code class="xref py py-mod docutils literal">multiprocessing</code></a> now relies on <a class="reference internal" href="../library/runpy.html#module-runpy" title="runpy: Locate and run Python modules without importing them first."><code class="xref py py-mod docutils literal">runpy</code></a> (which implements the

991

<code class="docutils literal">-m</code> switch) to initialise <code class="docutils literal">__main__</code> appropriately in child processes

992

when using the <code class="docutils literal">spawn</code> or <code class="docutils literal">forkserver</code> start methods. This resolves some

993

edge cases where combining multiprocessing, the <code class="docutils literal">-m</code> command line switch,

994

and explicit relative imports could cause obscure failures in child

995

processes. (Contributed by Nick Coghlan in <a class="reference external" href="https://bugs.python.org/issue19946">issue 19946</a>.)

996

</div>

997

998

<h3>operator<a class="headerlink" href="#operator" title="Permalink to this headline">¶</a></h3>

999

New function <a class="reference internal" href="../library/operator.html#operator.length_hint" title="operator.length_hint"><code class="xref py py-func docutils literal">length_hint()</code></a> provides an implementation of the

1000

specification for how the <a class="reference internal" href="../reference/datamodel.html#object.__length_hint__" title="object.__length_hint__"><code class="xref py py-meth docutils literal">__length_hint__()</code></a> special method should

1001

be used, as part of the <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0424">PEP 424</a> formal specification of this language

1002

feature. (Contributed by Armin Ronacher in <a class="reference external" href="https://bugs.python.org/issue16148">issue 16148</a>.)

1003

There is now a pure-python version of the <a class="reference internal" href="../library/operator.html#module-operator" title="operator: Functions corresponding to the standard operators."><code class="xref py py-mod docutils literal">operator</code></a> module available for

1004

reference and for use by alternate implementations of Python. (Contributed by

1005

Zachary Ware in <a class="reference external" href="https://bugs.python.org/issue16694">issue 16694</a>.)

1006

</div>

1007

1008

1009

There are new functions to get and set the <a class="reference internal" href="../library/os.html#fd-inheritance">inheritable flag</a> of a file descriptor (<a class="reference internal" href="../library/os.html#os.get_inheritable" title="os.get_inheritable"><code class="xref py py-func docutils literal">os.get_inheritable()</code></a>,

1010

<a class="reference internal" href="../library/os.html#os.set_inheritable" title="os.set_inheritable"><code class="xref py py-func docutils literal">os.set_inheritable()</code></a>) or a Windows handle

1011

(<a class="reference internal" href="../library/os.html#os.get_handle_inheritable" title="os.get_handle_inheritable"><code class="xref py py-func docutils literal">os.get_handle_inheritable()</code></a>, <a class="reference internal" href="../library/os.html#os.set_handle_inheritable" title="os.set_handle_inheritable"><code class="xref py py-func docutils literal">os.set_handle_inheritable()</code></a>).

1012

New function <a class="reference internal" href="../library/os.html#os.cpu_count" title="os.cpu_count"><code class="xref py py-func docutils literal">cpu_count()</code></a> reports the number of CPUs available on the

1013

platform on which Python is running (or <code class="docutils literal">None</code> if the count can’t be

1014

determined). The <a class="reference internal" href="../library/multiprocessing.html#multiprocessing.cpu_count" title="multiprocessing.cpu_count"><code class="xref py py-func docutils literal">multiprocessing.cpu_count()</code></a> function is now implemented

1015

in terms of this function). (Contributed by Trent Nelson, Yogesh Chaudhari,

1016

Victor Stinner, and Charles-François Natali in <a class="reference external" href="https://bugs.python.org/issue17914">issue 17914</a>.)

1017

<a class="reference internal" href="../library/os.path.html#os.path.samestat" title="os.path.samestat"><code class="xref py py-func docutils literal">os.path.samestat()</code></a> is now available on the Windows platform (and the

1018

<a class="reference internal" href="../library/os.path.html#os.path.samefile" title="os.path.samefile"><code class="xref py py-func docutils literal">os.path.samefile()</code></a> implementation is now shared between Unix and

1019

Windows). (Contributed by Brian Curtin in <a class="reference external" href="https://bugs.python.org/issue11939">issue 11939</a>.)

1020

<a class="reference internal" href="../library/os.path.html#os.path.ismount" title="os.path.ismount"><code class="xref py py-func docutils literal">os.path.ismount()</code></a> now recognizes volumes mounted below a drive

1021

root on Windows. (Contributed by Tim Golden in <a class="reference external" href="https://bugs.python.org/issue9035">issue 9035</a>.)

1022

<a class="reference internal" href="../library/os.html#os.open" title="os.open"><code class="xref py py-func docutils literal">os.open()</code></a> supports two new flags on platforms that provide them,

1023

<a class="reference internal" href="../library/os.html#os.O_PATH" title="os.O_PATH"><code class="xref py py-data docutils literal">O_PATH</code></a> (un-opened file descriptor), and <a class="reference internal" href="../library/os.html#os.O_TMPFILE" title="os.O_TMPFILE"><code class="xref py py-data docutils literal">O_TMPFILE</code></a>

1024

(unnamed temporary file; as of 3.4.0 release available only on Linux systems

1025

with a kernel version of 3.11 or newer that have uapi headers). (Contributed

1026

by Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue18673">issue 18673</a> and Benjamin Peterson, respectively.)

1027

</div>

1028

1029

1030

<a class="reference internal" href="../library/pdb.html#module-pdb" title="pdb: The Python debugger for interactive interpreters."><code class="xref py py-mod docutils literal">pdb</code></a> has been enhanced to handle generators, <a class="reference internal" href="../reference/simple_stmts.html#yield"><code class="xref std std-keyword docutils literal">yield</code></a>, and

1031

<code class="docutils literal">yield from</code> in a more useful fashion. This is especially helpful when

1032

debugging <a class="reference internal" href="../library/asyncio.html#module-asyncio" title="asyncio: Asynchronous I/O, event loop, coroutines and tasks."><code class="xref py py-mod docutils literal">asyncio</code></a> based programs. (Contributed by Andrew Svetlov and

1033

Xavier de Gaye in <a class="reference external" href="https://bugs.python.org/issue16596">issue 16596</a>.)

1034

The <code class="docutils literal">print</code> command has been removed from <a class="reference internal" href="../library/pdb.html#module-pdb" title="pdb: The Python debugger for interactive interpreters."><code class="xref py py-mod docutils literal">pdb</code></a>, restoring access to the

1035

Python <a class="reference internal" href="../library/functions.html#print" title="print"><code class="xref py py-func docutils literal">print()</code></a> function from the pdb command line. Python2’s <code class="docutils literal">pdb</code> did

1036

not have a <code class="docutils literal">print</code> command; instead, entering <code class="docutils literal">print</code> executed the

1037

<code class="docutils literal">print</code> statement. In Python3 <code class="docutils literal">print</code> was mistakenly made an alias for the

1038

pdb <a class="reference internal" href="../library/pdb.html#pdbcommand-p"><code class="xref std std-pdbcmd docutils literal">p</code></a> command. <code class="docutils literal">p</code>, however, prints the <code class="docutils literal">repr</code> of its argument,

1039

not the <code class="docutils literal">str</code> like the Python2 <code class="docutils literal">print</code> command did. Worse, the Python3

1040

<code class="docutils literal">pdb print</code> command shadowed the Python3 <code class="docutils literal">print</code> function, making it

1041

inaccessible at the <code class="docutils literal">pdb</code> prompt. (Contributed by Connor Osborn in

1042

<a class="reference external" href="https://bugs.python.org/issue18764">issue 18764</a>.)

1043

</div>

1044

1045

<h3>pickle<a class="headerlink" href="#pickle" title="Permalink to this headline">¶</a></h3>

1046

<a class="reference internal" href="../library/pickle.html#module-pickle" title="pickle: Convert Python objects to streams of bytes and back."><code class="xref py py-mod docutils literal">pickle</code></a> now supports (but does not use by default) a new pickle protocol,

1047

protocol 4. This new protocol addresses a number of issues that were present

1048

in previous protocols, such as the serialization of nested classes, very large

1049

strings and containers, and classes whose <a class="reference internal" href="../reference/datamodel.html#object.__new__" title="object.__new__"><code class="xref py py-meth docutils literal">__new__()</code></a> method takes

1050

keyword-only arguments. It also provides some efficiency improvements.

1051

1052

See also

1053

1054

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-3154">PEP 3154</a> – Pickle protocol 4</dt>

1055

<dd>PEP written by Antoine Pitrou and implemented by Alexandre Vassalotti.</dd>

1056

</dl>

1057

</div>

1058

</div>

1059

1060

<h3>plistlib<a class="headerlink" href="#plistlib" title="Permalink to this headline">¶</a></h3>

1061

<a class="reference internal" href="../library/plistlib.html#module-plistlib" title="plistlib: Generate and parse Mac OS X plist files."><code class="xref py py-mod docutils literal">plistlib</code></a> now has an API that is similar to the standard pattern for

1062

stdlib serialization protocols, with new <a class="reference internal" href="../library/plistlib.html#plistlib.load" title="plistlib.load"><code class="xref py py-func docutils literal">load()</code></a>,

1063

<a class="reference internal" href="../library/plistlib.html#plistlib.dump" title="plistlib.dump"><code class="xref py py-func docutils literal">dump()</code></a>, <a class="reference internal" href="../library/plistlib.html#plistlib.loads" title="plistlib.loads"><code class="xref py py-func docutils literal">loads()</code></a>, and <a class="reference internal" href="../library/plistlib.html#plistlib.dumps" title="plistlib.dumps"><code class="xref py py-func docutils literal">dumps()</code></a>

1064

functions. (The older API is now deprecated.) In addition to the already

1065

supported XML plist format (<a class="reference internal" href="../library/plistlib.html#plistlib.FMT_XML" title="plistlib.FMT_XML"><code class="xref py py-data docutils literal">FMT_XML</code></a>), it also now supports

1066

the binary plist format (<a class="reference internal" href="../library/plistlib.html#plistlib.FMT_BINARY" title="plistlib.FMT_BINARY"><code class="xref py py-data docutils literal">FMT_BINARY</code></a>). (Contributed by Ronald

1067

Oussoren and others in <a class="reference external" href="https://bugs.python.org/issue14455">issue 14455</a>.)

1068

</div>

1069

1070

<h3>poplib<a class="headerlink" href="#poplib" title="Permalink to this headline">¶</a></h3>

1071

Two new methods have been added to <a class="reference internal" href="../library/poplib.html#module-poplib" title="poplib: POP3 protocol client (requires sockets)."><code class="xref py py-mod docutils literal">poplib</code></a>: <a class="reference internal" href="../library/poplib.html#poplib.POP3.capa" title="poplib.POP3.capa"><code class="xref py py-meth docutils literal">capa()</code></a>,

1072

which returns the list of capabilities advertised by the POP server, and

1073

<a class="reference internal" href="../library/poplib.html#poplib.POP3.stls" title="poplib.POP3.stls"><code class="xref py py-meth docutils literal">stls()</code></a>, which switches a clear-text POP3 session into an

1074

encrypted POP3 session if the POP server supports it. (Contributed by Lorenzo

1075

Catucci in <a class="reference external" href="https://bugs.python.org/issue4473">issue 4473</a>.)

1076

</div>

1077

1078

<h3>pprint<a class="headerlink" href="#pprint" title="Permalink to this headline">¶</a></h3>

1079

The <a class="reference internal" href="../library/pprint.html#module-pprint" title="pprint: Data pretty printer."><code class="xref py py-mod docutils literal">pprint</code></a> module’s <a class="reference internal" href="../library/pprint.html#pprint.PrettyPrinter" title="pprint.PrettyPrinter"><code class="xref py py-class docutils literal">PrettyPrinter</code></a> class and its

1080

<a class="reference internal" href="../library/pprint.html#pprint.pformat" title="pprint.pformat"><code class="xref py py-func docutils literal">pformat()</code></a>, and <a class="reference internal" href="../library/pprint.html#pprint.pprint" title="pprint.pprint"><code class="xref py py-func docutils literal">pprint()</code></a> functions have a new

1081

option, compact, that controls how the output is formatted. Currently

1082

setting compact to <code class="docutils literal">True</code> means that sequences will be printed with as many

1083

sequence elements as will fit within width on each (indented) line.

1084

(Contributed by Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue19132">issue 19132</a>.)

1085

Long strings are now wrapped using Python’s normal line continuation

1086

syntax. (Contributed by Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue17150">issue 17150</a>.)

1087

</div>

1088

1089

1090

<a class="reference internal" href="../library/pty.html#pty.spawn" title="pty.spawn"><code class="xref py py-func docutils literal">pty.spawn()</code></a> now returns the status value from <a class="reference internal" href="../library/os.html#os.waitpid" title="os.waitpid"><code class="xref py py-func docutils literal">os.waitpid()</code></a> on

1091

the child process, instead of <code class="docutils literal">None</code>. (Contributed by Gregory P. Smith.)

1092

</div>

1093

1094

<h3>pydoc<a class="headerlink" href="#pydoc" title="Permalink to this headline">¶</a></h3>

1095

The <a class="reference internal" href="../library/pydoc.html#module-pydoc" title="pydoc: Documentation generator and online help system."><code class="xref py py-mod docutils literal">pydoc</code></a> module is now based directly on the <a class="reference internal" href="../library/inspect.html#inspect.signature" title="inspect.signature"><code class="xref py py-func docutils literal">inspect.signature()</code></a>

1096

introspection API, allowing it to provide signature information for a wider

1097

variety of callable objects. This change also means that <code class="docutils literal">__wrapped__</code>

1098

attributes are now taken into account when displaying help information.

1099

(Contributed by Larry Hastings in <a class="reference external" href="https://bugs.python.org/issue19674">issue 19674</a>.)

1100

The <a class="reference internal" href="../library/pydoc.html#module-pydoc" title="pydoc: Documentation generator and online help system."><code class="xref py py-mod docutils literal">pydoc</code></a> module no longer displays the <code class="docutils literal">self</code> parameter for

1101

already bound methods. Instead, it aims to always display the exact current

1102

signature of the supplied callable. (Contributed by Larry Hastings in

1103

<a class="reference external" href="https://bugs.python.org/issue20710">issue 20710</a>.)

1104

In addition to the changes that have been made to <a class="reference internal" href="../library/pydoc.html#module-pydoc" title="pydoc: Documentation generator and online help system."><code class="xref py py-mod docutils literal">pydoc</code></a> directly,

1105

its handling of custom <code class="docutils literal">__dir__</code> methods and various descriptor

1106

behaviours has also been improved substantially by the underlying changes in

1107

the <a class="reference internal" href="../library/inspect.html#module-inspect" title="inspect: Extract information and source code from live objects."><code class="xref py py-mod docutils literal">inspect</code></a> module.

1108

As the <a class="reference internal" href="../library/functions.html#help" title="help"><code class="xref py py-func docutils literal">help()</code></a> builtin is based on <a class="reference internal" href="../library/pydoc.html#module-pydoc" title="pydoc: Documentation generator and online help system."><code class="xref py py-mod docutils literal">pydoc</code></a>, the above changes also

1109

affect the behaviour of <a class="reference internal" href="../library/functions.html#help" title="help"><code class="xref py py-func docutils literal">help()</code></a>.

1110

</div>

1111

1112

1113

New <a class="reference internal" href="../library/re.html#re.fullmatch" title="re.fullmatch"><code class="xref py py-func docutils literal">fullmatch()</code></a> function and <a class="reference internal" href="../library/re.html#re.regex.fullmatch" title="re.regex.fullmatch"><code class="xref py py-meth docutils literal">regex.fullmatch()</code></a> method anchor

1114

the pattern at both ends of the string to match. This provides a way to be

1115

explicit about the goal of the match, which avoids a class of subtle bugs where

1116

<code class="docutils literal">$</code> characters get lost during code changes or the addition of alternatives

1117

to an existing regular expression. (Contributed by Matthew Barnett in

1118

<a class="reference external" href="https://bugs.python.org/issue16203">issue 16203</a>.)

1119

The repr of <a class="reference internal" href="../library/re.html#re-objects">regex objects</a> now includes the pattern

1120

and the flags; the repr of <a class="reference internal" href="../library/re.html#match-objects">match objects</a> now

1121

includes the start, end, and the part of the string that matched. (Contributed

1122

by Hugo Lopes Tavares and Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue13592">issue 13592</a> and

1123

<a class="reference external" href="https://bugs.python.org/issue17087">issue 17087</a>.)

1124

</div>

1125

1126

<h3>resource<a class="headerlink" href="#resource" title="Permalink to this headline">¶</a></h3>

1127

New <a class="reference internal" href="../library/resource.html#resource.prlimit" title="resource.prlimit"><code class="xref py py-func docutils literal">prlimit()</code></a> function, available on Linux platforms with a

1128

kernel version of 2.6.36 or later and glibc of 2.13 or later, provides the

1129

ability to query or set the resource limits for processes other than the one

1130

making the call. (Contributed by Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue16595">issue 16595</a>.)

1131

On Linux kernel version 2.6.36 or later, there are also some new

1132

Linux specific constants: <a class="reference internal" href="../library/resource.html#resource.RLIMIT_MSGQUEUE" title="resource.RLIMIT_MSGQUEUE"><code class="xref py py-attr docutils literal">RLIMIT_MSGQUEUE</code></a>,

1133

<a class="reference internal" href="../library/resource.html#resource.RLIMIT_NICE" title="resource.RLIMIT_NICE"><code class="xref py py-attr docutils literal">RLIMIT_NICE</code></a>, <a class="reference internal" href="../library/resource.html#resource.RLIMIT_RTPRIO" title="resource.RLIMIT_RTPRIO"><code class="xref py py-attr docutils literal">RLIMIT_RTPRIO</code></a>,

1134

<a class="reference internal" href="../library/resource.html#resource.RLIMIT_RTTIME" title="resource.RLIMIT_RTTIME"><code class="xref py py-attr docutils literal">RLIMIT_RTTIME</code></a>, and <a class="reference internal" href="../library/resource.html#resource.RLIMIT_SIGPENDING" title="resource.RLIMIT_SIGPENDING"><code class="xref py py-attr docutils literal">RLIMIT_SIGPENDING</code></a>.

1135

(Contributed by Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue19324">issue 19324</a>.)

1136

On FreeBSD version 9 and later, there some new FreeBSD specific constants:

1137

<a class="reference internal" href="../library/resource.html#resource.RLIMIT_SBSIZE" title="resource.RLIMIT_SBSIZE"><code class="xref py py-attr docutils literal">RLIMIT_SBSIZE</code></a>, <a class="reference internal" href="../library/resource.html#resource.RLIMIT_SWAP" title="resource.RLIMIT_SWAP"><code class="xref py py-attr docutils literal">RLIMIT_SWAP</code></a>, and

1138

<a class="reference internal" href="../library/resource.html#resource.RLIMIT_NPTS" title="resource.RLIMIT_NPTS"><code class="xref py py-attr docutils literal">RLIMIT_NPTS</code></a>. (Contributed by Claudiu Popa in

1139

<a class="reference external" href="https://bugs.python.org/issue19343">issue 19343</a>.)

1140

</div>

1141

1142

<h3>select<a class="headerlink" href="#select" title="Permalink to this headline">¶</a></h3>

1143

<a class="reference internal" href="../library/select.html#select.epoll" title="select.epoll"><code class="xref py py-class docutils literal">epoll</code></a> objects now support the context management protocol.

1144

When used in a <a class="reference internal" href="../reference/compound_stmts.html#with"><code class="xref std std-keyword docutils literal">with</code></a> statement, the <a class="reference internal" href="../library/select.html#select.epoll.close" title="select.epoll.close"><code class="xref py py-meth docutils literal">close()</code></a>

1145

method will be called automatically at the end of the block. (Contributed

1146

by Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue16488">issue 16488</a>.)

1147

<a class="reference internal" href="../library/select.html#select.devpoll" title="select.devpoll"><code class="xref py py-class docutils literal">devpoll</code></a> objects now have <a class="reference internal" href="../library/select.html#select.devpoll.fileno" title="select.devpoll.fileno"><code class="xref py py-meth docutils literal">fileno()</code></a> and

1148

<a class="reference internal" href="../library/select.html#select.devpoll.close" title="select.devpoll.close"><code class="xref py py-meth docutils literal">close()</code></a> methods, as well as a new attribute

1149

<a class="reference internal" href="../library/select.html#select.devpoll.closed" title="select.devpoll.closed"><code class="xref py py-attr docutils literal">closed</code></a>. (Contributed by Victor Stinner in

1150

<a class="reference external" href="https://bugs.python.org/issue18794">issue 18794</a>.)

1151

</div>

1152

1153

<h3>shelve<a class="headerlink" href="#shelve" title="Permalink to this headline">¶</a></h3>

1154

<a class="reference internal" href="../library/shelve.html#shelve.Shelf" title="shelve.Shelf"><code class="xref py py-class docutils literal">Shelf</code></a> instances may now be used in <a class="reference internal" href="../reference/compound_stmts.html#with"><code class="xref std std-keyword docutils literal">with</code></a> statements,

1155

and will be automatically closed at the end of the <a class="reference internal" href="../reference/compound_stmts.html#with"><code class="xref std std-keyword docutils literal">with</code></a> block.

1156

(Contributed by Filip Gruszczyński in <a class="reference external" href="https://bugs.python.org/issue13896">issue 13896</a>.)

1157

</div>

1158

1159

<h3>shutil<a class="headerlink" href="#shutil" title="Permalink to this headline">¶</a></h3>

1160

<a class="reference internal" href="../library/shutil.html#shutil.copyfile" title="shutil.copyfile"><code class="xref py py-func docutils literal">copyfile()</code></a> now raises a specific <a class="reference internal" href="../library/shutil.html#shutil.Error" title="shutil.Error"><code class="xref py py-exc docutils literal">Error</code></a> subclass,

1161

<a class="reference internal" href="../library/shutil.html#shutil.SameFileError" title="shutil.SameFileError"><code class="xref py py-exc docutils literal">SameFileError</code></a>, when the source and destination are the same

1162

file, which allows an application to take appropriate action on this specific

1163

error. (Contributed by Atsuo Ishimoto and Hynek Schlawack in

1164

<a class="reference external" href="https://bugs.python.org/issue1492704">issue 1492704</a>.)

1165

</div>

1166

1167

<h3>smtpd<a class="headerlink" href="#smtpd" title="Permalink to this headline">¶</a></h3>

1168

The <a class="reference internal" href="../library/smtpd.html#smtpd.SMTPServer" title="smtpd.SMTPServer"><code class="xref py py-class docutils literal">SMTPServer</code></a> and <a class="reference internal" href="../library/smtpd.html#smtpd.SMTPChannel" title="smtpd.SMTPChannel"><code class="xref py py-class docutils literal">SMTPChannel</code></a> classes now

1169

accept a map keyword argument which, if specified, is passed in to

1170

<a class="reference internal" href="../library/asynchat.html#asynchat.async_chat" title="asynchat.async_chat"><code class="xref py py-class docutils literal">asynchat.async_chat</code></a> as its map argument. This allows an application

1171

to avoid affecting the global socket map. (Contributed by Vinay Sajip in

1172

<a class="reference external" href="https://bugs.python.org/issue11959">issue 11959</a>.)

1173

</div>

1174

1175

<h3>smtplib<a class="headerlink" href="#smtplib" title="Permalink to this headline">¶</a></h3>

1176

<a class="reference internal" href="../library/smtplib.html#smtplib.SMTPException" title="smtplib.SMTPException"><code class="xref py py-exc docutils literal">SMTPException</code></a> is now a subclass of <a class="reference internal" href="../library/exceptions.html#OSError" title="OSError"><code class="xref py py-exc docutils literal">OSError</code></a>, which allows

1177

both socket level errors and SMTP protocol level errors to be caught in one

1178

try/except statement by code that only cares whether or not an error occurred.

1179

(Contributed by Ned Jackson Lovely in <a class="reference external" href="https://bugs.python.org/issue2118">issue 2118</a>.)

1180

</div>

1181

1182

<h3>socket<a class="headerlink" href="#socket" title="Permalink to this headline">¶</a></h3>

1183

The socket module now supports the <a class="reference internal" href="../library/socket.html#socket.CAN_BCM" title="socket.CAN_BCM"><code class="xref py py-data docutils literal">CAN_BCM</code></a> protocol on

1184

platforms that support it. (Contributed by Brian Thorne in <a class="reference external" href="https://bugs.python.org/issue15359">issue 15359</a>.)

1185

Socket objects have new methods to get or set their <a class="reference internal" href="../library/os.html#fd-inheritance">inheritable flag</a>, <a class="reference internal" href="../library/socket.html#socket.socket.get_inheritable" title="socket.socket.get_inheritable"><code class="xref py py-meth docutils literal">get_inheritable()</code></a> and

1186

<a class="reference internal" href="../library/socket.html#socket.socket.set_inheritable" title="socket.socket.set_inheritable"><code class="xref py py-meth docutils literal">set_inheritable()</code></a>.

1187

The <code class="docutils literal">socket.AF_*</code> and <code class="docutils literal">socket.SOCK_*</code> constants are now enumeration values

1188

using the new <a class="reference internal" href="../library/enum.html#module-enum" title="enum: Implementation of an enumeration class."><code class="xref py py-mod docutils literal">enum</code></a> module. This allows meaningful names to be printed

1189

during debugging, instead of integer “magic numbers”.

1190

The <a class="reference internal" href="../library/socket.html#socket.AF_LINK" title="socket.AF_LINK"><code class="xref py py-data docutils literal">AF_LINK</code></a> constant is now available on BSD and OSX.

1191

<a class="reference internal" href="../library/socket.html#socket.inet_pton" title="socket.inet_pton"><code class="xref py py-func docutils literal">inet_pton()</code></a> and <a class="reference internal" href="../library/socket.html#socket.inet_ntop" title="socket.inet_ntop"><code class="xref py py-func docutils literal">inet_ntop()</code></a> are now supported

1192

on Windows. (Contributed by Atsuo Ishimoto in <a class="reference external" href="https://bugs.python.org/issue7171">issue 7171</a>.)

1193

</div>

1194

1195

<h3>sqlite3<a class="headerlink" href="#sqlite3" title="Permalink to this headline">¶</a></h3>

1196

A new boolean parameter to the <a class="reference internal" href="../library/sqlite3.html#sqlite3.connect" title="sqlite3.connect"><code class="xref py py-func docutils literal">connect()</code></a> function, uri, can be

1197

used to indicate that the database parameter is a <code class="docutils literal">uri</code> (see the <a class="reference external" href="https://www.sqlite.org/uri.html">SQLite

1198

URI documentation</a>). (Contributed by poq in

1199

<a class="reference external" href="https://bugs.python.org/issue13773">issue 13773</a>.)

1200

</div>

1201

1202

1203

<a class="reference internal" href="../library/ssl.html#ssl.PROTOCOL_TLSv1_1" title="ssl.PROTOCOL_TLSv1_1"><code class="xref py py-data docutils literal">PROTOCOL_TLSv1_1</code></a> and <a class="reference internal" href="../library/ssl.html#ssl.PROTOCOL_TLSv1_2" title="ssl.PROTOCOL_TLSv1_2"><code class="xref py py-data docutils literal">PROTOCOL_TLSv1_2</code></a> (TLSv1.1 and

1204

TLSv1.2 support) have been added; support for these protocols is only available if

1205

Python is linked with OpenSSL 1.0.1 or later. (Contributed by Michele Orrù and

1206

Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue16692">issue 16692</a>.)

1207

New function <a class="reference internal" href="../library/ssl.html#ssl.create_default_context" title="ssl.create_default_context"><code class="xref py py-func docutils literal">create_default_context()</code></a> provides a standard way to

1208

obtain an <a class="reference internal" href="../library/ssl.html#ssl.SSLContext" title="ssl.SSLContext"><code class="xref py py-class docutils literal">SSLContext</code></a> whose settings are intended to be a

1209

reasonable balance between compatibility and security. These settings are

1210

more stringent than the defaults provided by the <a class="reference internal" href="../library/ssl.html#ssl.SSLContext" title="ssl.SSLContext"><code class="xref py py-class docutils literal">SSLContext</code></a>

1211

constructor, and may be adjusted in the future, without prior deprecation, if

1212

best-practice security requirements change. The new recommended best

1213

practice for using stdlib libraries that support SSL is to use

1214

<a class="reference internal" href="../library/ssl.html#ssl.create_default_context" title="ssl.create_default_context"><code class="xref py py-func docutils literal">create_default_context()</code></a> to obtain an <a class="reference internal" href="../library/ssl.html#ssl.SSLContext" title="ssl.SSLContext"><code class="xref py py-class docutils literal">SSLContext</code></a>

1215

object, modify it if needed, and then pass it as the context argument

1216

of the appropriate stdlib API. (Contributed by Christian Heimes

1217

in <a class="reference external" href="https://bugs.python.org/issue19689">issue 19689</a>.)

1218

<a class="reference internal" href="../library/ssl.html#ssl.SSLContext" title="ssl.SSLContext"><code class="xref py py-class docutils literal">SSLContext</code></a> method <a class="reference internal" href="../library/ssl.html#ssl.SSLContext.load_verify_locations" title="ssl.SSLContext.load_verify_locations"><code class="xref py py-meth docutils literal">load_verify_locations()</code></a>

1219

accepts a new optional argument cadata, which can be used to provide PEM or

1220

DER encoded certificates directly via strings or bytes, respectively.

1221

(Contributed by Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue18138">issue 18138</a>.)

1222

New function <a class="reference internal" href="../library/ssl.html#ssl.get_default_verify_paths" title="ssl.get_default_verify_paths"><code class="xref py py-func docutils literal">get_default_verify_paths()</code></a> returns

1223

a named tuple of the paths and environment variables that the

1224

<a class="reference internal" href="../library/ssl.html#ssl.SSLContext.set_default_verify_paths" title="ssl.SSLContext.set_default_verify_paths"><code class="xref py py-meth docutils literal">set_default_verify_paths()</code></a> method uses to set

1225

OpenSSL’s default <code class="docutils literal">cafile</code> and <code class="docutils literal">capath</code>. This can be an aid in

1226

debugging default verification issues. (Contributed by Christian Heimes

1227

in <a class="reference external" href="https://bugs.python.org/issue18143">issue 18143</a>.)

1228

1229

<a class="reference internal" href="../library/ssl.html#ssl.SSLContext.cert_store_stats" title="ssl.SSLContext.cert_store_stats"><code class="xref py py-meth docutils literal">cert_store_stats()</code></a>, that reports the number of loaded

1230

<code class="docutils literal">X.509</code> certs, <code class="docutils literal">X.509 CA</code> certs, and certificate revocation lists (<code class="docutils literal">crl</code>s), as well as a <a class="reference internal" href="../library/ssl.html#ssl.SSLContext.get_ca_certs" title="ssl.SSLContext.get_ca_certs"><code class="xref py py-meth docutils literal">get_ca_certs()</code></a> method that returns a

1231

list of the loaded <code class="docutils literal">CA</code> certificates. (Contributed by Christian Heimes in

1232

<a class="reference external" href="https://bugs.python.org/issue18147">issue 18147</a>.)

1233

If OpenSSL 0.9.8 or later is available, <a class="reference internal" href="../library/ssl.html#ssl.SSLContext" title="ssl.SSLContext"><code class="xref py py-class docutils literal">SSLContext</code></a> has a new

1234

attribute <a class="reference internal" href="../library/ssl.html#ssl.SSLContext.verify_flags" title="ssl.SSLContext.verify_flags"><code class="xref py py-attr docutils literal">verify_flags</code></a> that can be used to control the

1235

certificate verification process by setting it to some combination of the new

1236

constants <a class="reference internal" href="../library/ssl.html#ssl.VERIFY_DEFAULT" title="ssl.VERIFY_DEFAULT"><code class="xref py py-data docutils literal">VERIFY_DEFAULT</code></a>, <a class="reference internal" href="../library/ssl.html#ssl.VERIFY_CRL_CHECK_LEAF" title="ssl.VERIFY_CRL_CHECK_LEAF"><code class="xref py py-data docutils literal">VERIFY_CRL_CHECK_LEAF</code></a>,

1237

<a class="reference internal" href="../library/ssl.html#ssl.VERIFY_CRL_CHECK_CHAIN" title="ssl.VERIFY_CRL_CHECK_CHAIN"><code class="xref py py-data docutils literal">VERIFY_CRL_CHECK_CHAIN</code></a>, or <a class="reference internal" href="../library/ssl.html#ssl.VERIFY_X509_STRICT" title="ssl.VERIFY_X509_STRICT"><code class="xref py py-data docutils literal">VERIFY_X509_STRICT</code></a>.

1238

OpenSSL does not do any CRL verification by default. (Contributed by

1239

Christien Heimes in <a class="reference external" href="https://bugs.python.org/issue8813">issue 8813</a>.)

1240

New <a class="reference internal" href="../library/ssl.html#ssl.SSLContext" title="ssl.SSLContext"><code class="xref py py-class docutils literal">SSLContext</code></a> method <a class="reference internal" href="../library/ssl.html#ssl.SSLContext.load_default_certs" title="ssl.SSLContext.load_default_certs"><code class="xref py py-meth docutils literal">load_default_certs()</code></a>

1241

loads a set of default “certificate authority” (CA) certificates from default

1242

locations, which vary according to the platform. It can be used to load both

1243

TLS web server authentication certificates

1244

(<code class="docutils literal">purpose=</code><a class="reference internal" href="../library/ssl.html#ssl.Purpose.SERVER_AUTH" title="ssl.Purpose.SERVER_AUTH"><code class="xref py py-data docutils literal">SERVER_AUTH</code></a>) for a client to use to verify a

1245

server, and certificates for a server to use in verifying client certificates

1246

(<code class="docutils literal">purpose=</code><a class="reference internal" href="../library/ssl.html#ssl.Purpose.CLIENT_AUTH" title="ssl.Purpose.CLIENT_AUTH"><code class="xref py py-data docutils literal">CLIENT_AUTH</code></a>). (Contributed by Christian

1247

Heimes in <a class="reference external" href="https://bugs.python.org/issue19292">issue 19292</a>.)

1248

Two new windows-only functions, <a class="reference internal" href="../library/ssl.html#ssl.enum_certificates" title="ssl.enum_certificates"><code class="xref py py-func docutils literal">enum_certificates()</code></a> and

1249

<a class="reference internal" href="../library/ssl.html#ssl.enum_crls" title="ssl.enum_crls"><code class="xref py py-func docutils literal">enum_crls()</code></a> provide the ability to retrieve certificates,

1250

certificate information, and CRLs from the Windows cert store. (Contributed

1251

by Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue17134">issue 17134</a>.)

1252

Support for server-side SNI (Server Name Indication) using the new

1253

<a class="reference internal" href="../library/ssl.html#ssl.SSLContext.set_servername_callback" title="ssl.SSLContext.set_servername_callback"><code class="xref py py-meth docutils literal">ssl.SSLContext.set_servername_callback()</code></a> method.

1254

(Contributed by Daniel Black in <a class="reference external" href="https://bugs.python.org/issue8109">issue 8109</a>.)

1255

The dictionary returned by <a class="reference internal" href="../library/ssl.html#ssl.SSLSocket.getpeercert" title="ssl.SSLSocket.getpeercert"><code class="xref py py-meth docutils literal">SSLSocket.getpeercert()</code></a> contains additional

1256

<code class="docutils literal">X509v3</code> extension items: <code class="docutils literal">crlDistributionPoints</code>, <code class="docutils literal">calIssuers</code>, and

1257

<code class="docutils literal">OCSP</code> URIs. (Contributed by Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue18379">issue 18379</a>.)

1258

</div>

1259

1260

1261

The <a class="reference internal" href="../library/stat.html#module-stat" title="stat: Utilities for interpreting the results of os.stat(), os.lstat() and os.fstat()."><code class="xref py py-mod docutils literal">stat</code></a> module is now backed by a C implementation in <code class="xref py py-mod docutils literal">_stat</code>. A C

1262

implementation is required as most of the values aren’t standardized and

1263

are platform-dependent. (Contributed by Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue11016">issue 11016</a>.)

1264

The module supports new <a class="reference internal" href="../library/stat.html#stat.ST_MODE" title="stat.ST_MODE"><code class="xref py py-mod docutils literal">ST_MODE</code></a> flags, <a class="reference internal" href="../library/stat.html#stat.S_IFDOOR" title="stat.S_IFDOOR"><code class="xref py py-mod docutils literal">S_IFDOOR</code></a>,

1265

<a class="reference internal" href="../library/stat.html#stat.S_IFPORT" title="stat.S_IFPORT"><code class="xref py py-attr docutils literal">S_IFPORT</code></a>, and <a class="reference internal" href="../library/stat.html#stat.S_IFWHT" title="stat.S_IFWHT"><code class="xref py py-attr docutils literal">S_IFWHT</code></a>. (Contributed by

1266

Christian Hiemes in <a class="reference external" href="https://bugs.python.org/issue11016">issue 11016</a>.)

1267

</div>

1268

1269

<h3>struct<a class="headerlink" href="#struct" title="Permalink to this headline">¶</a></h3>

1270

New function <a class="reference internal" href="../library/struct.html#struct.iter_unpack" title="struct.iter_unpack"><code class="xref py py-mod docutils literal">iter_unpack</code></a> and a new

1271

<a class="reference internal" href="../library/struct.html#struct.Struct.iter_unpack" title="struct.Struct.iter_unpack"><code class="xref py py-meth docutils literal">struct.Struct.iter_unpack()</code></a> method on compiled formats provide streamed

1272

unpacking of a buffer containing repeated instances of a given format of data.

1273

(Contributed by Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue17804">issue 17804</a>.)

1274

</div>

1275

1276

<h3>subprocess<a class="headerlink" href="#subprocess" title="Permalink to this headline">¶</a></h3>

1277

<a class="reference internal" href="../library/subprocess.html#subprocess.check_output" title="subprocess.check_output"><code class="xref py py-func docutils literal">check_output()</code></a> now accepts an input argument that can

1278

be used to provide the contents of <code class="docutils literal">stdin</code> for the command that is run.

1279

(Contributed by Zack Weinberg in <a class="reference external" href="https://bugs.python.org/issue16624">issue 16624</a>.)

1280

<code class="xref py py-func docutils literal">getstatus()</code> and <a class="reference internal" href="../library/subprocess.html#subprocess.getstatusoutput" title="subprocess.getstatusoutput"><code class="xref py py-func docutils literal">getstatusoutput()</code></a> now

1281

work on Windows. This change was actually inadvertently made in 3.3.4.

1282

(Contributed by Tim Golden in <a class="reference external" href="https://bugs.python.org/issue10197">issue 10197</a>.)

1283

</div>

1284

1285

<h3>sunau<a class="headerlink" href="#sunau" title="Permalink to this headline">¶</a></h3>

1286

The <code class="xref py py-meth docutils literal">getparams()</code> method now returns a namedtuple rather than a

1287

plain tuple. (Contributed by Claudiu Popa in <a class="reference external" href="https://bugs.python.org/issue18901">issue 18901</a>.)

1288

<a class="reference internal" href="../library/sunau.html#sunau.open" title="sunau.open"><code class="xref py py-meth docutils literal">sunau.open()</code></a> now supports the context management protocol: when used in a

1289

<a class="reference internal" href="../reference/compound_stmts.html#with"><code class="xref std std-keyword docutils literal">with</code></a> block, the <code class="docutils literal">close</code> method of the returned object will be

1290

called automatically at the end of the block. (Contributed by Serhiy Storchaka

1291

in <a class="reference external" href="https://bugs.python.org/issue18878">issue 18878</a>.)

1292

<a class="reference internal" href="../library/sunau.html#sunau.AU_write.setsampwidth" title="sunau.AU_write.setsampwidth"><code class="xref py py-meth docutils literal">AU_write.setsampwidth()</code></a> now supports 24 bit samples, thus adding

1293

support for writing 24 sample using the module. (Contributed by

1294

Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue19261">issue 19261</a>.)

1295

The <a class="reference internal" href="../library/sunau.html#sunau.AU_write.writeframesraw" title="sunau.AU_write.writeframesraw"><code class="xref py py-meth docutils literal">writeframesraw()</code></a> and

1296

<a class="reference internal" href="../library/sunau.html#sunau.AU_write.writeframes" title="sunau.AU_write.writeframes"><code class="xref py py-meth docutils literal">writeframes()</code></a> methods now accept any <a class="reference internal" href="../glossary.html#term-bytes-like-object">bytes-like

1297

object</a>. (Contributed by Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue8311">issue 8311</a>.)

1298

</div>

1299

1300

1301

New function <a class="reference internal" href="../library/sys.html#sys.getallocatedblocks" title="sys.getallocatedblocks"><code class="xref py py-func docutils literal">sys.getallocatedblocks()</code></a> returns the current number of

1302

blocks allocated by the interpreter. (In CPython with the default

1303

<code class="docutils literal">--with-pymalloc</code> setting, this is allocations made through the

1304

<code class="xref c c-func docutils literal">PyObject_Malloc()</code> API.) This can be useful for tracking memory leaks,

1305

especially if automated via a test suite. (Contributed by Antoine Pitrou

1306

in <a class="reference external" href="https://bugs.python.org/issue13390">issue 13390</a>.)

1307

When the Python interpreter starts in <a class="reference internal" href="../tutorial/interpreter.html#tut-interactive">interactive mode</a>, it checks for an <a class="reference internal" href="../library/sys.html#sys.__interactivehook__" title="sys.__interactivehook__"><code class="xref py py-data docutils literal">__interactivehook__</code></a> attribute

1308

on the <a class="reference internal" href="../library/sys.html#module-sys" title="sys: Access system-specific parameters and functions."><code class="xref py py-mod docutils literal">sys</code></a> module. If the attribute exists, its value is called with no

1309

arguments just before interactive mode is started. The check is made after the

1310

<a class="reference internal" href="../using/cmdline.html#envvar-PYTHONSTARTUP"><code class="xref std std-envvar docutils literal">PYTHONSTARTUP</code></a> file is read, so it can be set there. The <a class="reference internal" href="../library/site.html#module-site" title="site: Module responsible for site-specific configuration."><code class="xref py py-mod docutils literal">site</code></a>

1311

module <a class="reference internal" href="../library/site.html#rlcompleter-config">sets it</a> to a function that enables tab

1312

completion and history saving (in <code class="file docutils literal">~/.python-history</code>) if the platform

1313

supports <a class="reference internal" href="../library/readline.html#module-readline" title="readline: GNU readline support for Python. (Unix)"><code class="xref py py-mod docutils literal">readline</code></a>. If you do not want this (new) behavior, you can

1314

override it in <a class="reference internal" href="../using/cmdline.html#envvar-PYTHONSTARTUP"><code class="xref std std-envvar docutils literal">PYTHONSTARTUP</code></a>, <code class="xref py py-mod docutils literal">sitecustomize</code>, or

1315

<code class="xref py py-mod docutils literal">usercustomize</code> by deleting this attribute from <a class="reference internal" href="../library/sys.html#module-sys" title="sys: Access system-specific parameters and functions."><code class="xref py py-mod docutils literal">sys</code></a> (or setting it

1316

to some other callable). (Contributed by Éric Araujo and Antoine Pitrou in

1317

<a class="reference external" href="https://bugs.python.org/issue5845">issue 5845</a>.)

1318

</div>

1319

1320

<h3>tarfile<a class="headerlink" href="#tarfile" title="Permalink to this headline">¶</a></h3>

1321

The <a class="reference internal" href="../library/tarfile.html#module-tarfile" title="tarfile: Read and write tar-format archive files."><code class="xref py py-mod docutils literal">tarfile</code></a> module now supports a simple <a class="reference internal" href="../library/tarfile.html#tarfile-commandline">Command Line Interface</a> when

1322

called as a script directly or via <code class="docutils literal">-m</code>. This can be used to create and

1323

extract tarfile archives. (Contributed by Berker Peksag in <a class="reference external" href="https://bugs.python.org/issue13477">issue 13477</a>.)

1324

</div>

1325

1326

<h3>textwrap<a class="headerlink" href="#textwrap" title="Permalink to this headline">¶</a></h3>

1327

The <a class="reference internal" href="../library/textwrap.html#textwrap.TextWrapper" title="textwrap.TextWrapper"><code class="xref py py-class docutils literal">TextWrapper</code></a> class has two new attributes/constructor

1328

arguments: <a class="reference internal" href="../library/textwrap.html#textwrap.TextWrapper.max_lines" title="textwrap.TextWrapper.max_lines"><code class="xref py py-attr docutils literal">max_lines</code></a>, which limits the number of

1329

lines in the output, and <a class="reference internal" href="../library/textwrap.html#textwrap.TextWrapper.placeholder" title="textwrap.TextWrapper.placeholder"><code class="xref py py-attr docutils literal">placeholder</code></a>, which is a

1330

string that will appear at the end of the output if it has been truncated

1331

because of max_lines. Building on these capabilities, a new convenience

1332

function <a class="reference internal" href="../library/textwrap.html#textwrap.shorten" title="textwrap.shorten"><code class="xref py py-func docutils literal">shorten()</code></a> collapses all of the whitespace in the input

1333

to single spaces and produces a single line of a given width that ends with

1334

the placeholder (by default, <code class="docutils literal">[...]</code>). (Contributed by Antoine Pitrou and

1335

Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue18585">issue 18585</a> and <a class="reference external" href="https://bugs.python.org/issue18725">issue 18725</a>.)

1336

</div>

1337

1338

<h3>threading<a class="headerlink" href="#threading" title="Permalink to this headline">¶</a></h3>

1339

The <a class="reference internal" href="../library/threading.html#threading.Thread" title="threading.Thread"><code class="xref py py-class docutils literal">Thread</code></a> object representing the main thread can be

1340

obtained from the new <a class="reference internal" href="../library/threading.html#threading.main_thread" title="threading.main_thread"><code class="xref py py-func docutils literal">main_thread()</code></a> function. In normal

1341

conditions this will be the thread from which the Python interpreter was

1342

started. (Contributed by Andrew Svetlov in <a class="reference external" href="https://bugs.python.org/issue18882">issue 18882</a>.)

1343

</div>

1344

1345

<h3>traceback<a class="headerlink" href="#traceback" title="Permalink to this headline">¶</a></h3>

1346

A new <a class="reference internal" href="../library/traceback.html#traceback.clear_frames" title="traceback.clear_frames"><code class="xref py py-func docutils literal">traceback.clear_frames()</code></a> function takes a traceback object

1347

and clears the local variables in all of the frames it references,

1348

reducing the amount of memory consumed. (Contributed by Andrew Kuchling in

1349

<a class="reference external" href="https://bugs.python.org/issue1565525">issue 1565525</a>.)

1350

</div>

1351

1352

<h3>types<a class="headerlink" href="#types" title="Permalink to this headline">¶</a></h3>

1353

A new <a class="reference internal" href="../library/types.html#types.DynamicClassAttribute" title="types.DynamicClassAttribute"><code class="xref py py-func docutils literal">DynamicClassAttribute()</code></a> descriptor provides a way to define

1354

an attribute that acts normally when looked up through an instance object, but

1355

which is routed to the class <code class="docutils literal">__getattr__</code> when looked up through the

1356

class. This allows one to have properties active on a class, and have virtual

1357

attributes on the class with the same name (see <code class="xref py py-mod docutils literal">Enum</code> for an example).

1358

(Contributed by Ethan Furman in <a class="reference external" href="https://bugs.python.org/issue19030">issue 19030</a>.)

1359

</div>

1360

1361

<h3>urllib<a class="headerlink" href="#urllib" title="Permalink to this headline">¶</a></h3>

1362

<a class="reference internal" href="../library/urllib.request.html#module-urllib.request" title="urllib.request: Extensible library for opening URLs."><code class="xref py py-mod docutils literal">urllib.request</code></a> now supports <code class="docutils literal">data:</code> URLs via the

1363

<a class="reference internal" href="../library/urllib.request.html#urllib.request.DataHandler" title="urllib.request.DataHandler"><code class="xref py py-class docutils literal">DataHandler</code></a> class. (Contributed by Mathias Panzenböck

1364

in <a class="reference external" href="https://bugs.python.org/issue16423">issue 16423</a>.)

1365

The http method that will be used by a <a class="reference internal" href="../library/urllib.request.html#urllib.request.Request" title="urllib.request.Request"><code class="xref py py-class docutils literal">Request</code></a> class

1366

can now be specified by setting a <a class="reference internal" href="../library/urllib.request.html#urllib.request.Request.method" title="urllib.request.Request.method"><code class="xref py py-class docutils literal">method</code></a>

1367

class attribute on the subclass. (Contributed by Jason R Coombs in

1368

<a class="reference external" href="https://bugs.python.org/issue18978">issue 18978</a>.)

1369

<a class="reference internal" href="../library/urllib.request.html#urllib.request.Request" title="urllib.request.Request"><code class="xref py py-class docutils literal">Request</code></a> objects are now reusable: if the

1370

1371

attributes are modified, all relevant internal properties are updated. This

1372

means, for example, that it is now possible to use the same

1373

<a class="reference internal" href="../library/urllib.request.html#urllib.request.Request" title="urllib.request.Request"><code class="xref py py-class docutils literal">Request</code></a> object in more than one

1374

<a class="reference internal" href="../library/urllib.request.html#urllib.request.OpenerDirector.open" title="urllib.request.OpenerDirector.open"><code class="xref py py-meth docutils literal">OpenerDirector.open()</code></a> call with different data arguments, or to

1375

modify a <a class="reference internal" href="../library/urllib.request.html#urllib.request.Request" title="urllib.request.Request"><code class="xref py py-class docutils literal">Request</code></a>‘s <code class="docutils literal">url</code> rather than recomputing it

1376

from scratch. There is also a new

1377

<a class="reference internal" href="../library/urllib.request.html#urllib.request.Request.remove_header" title="urllib.request.Request.remove_header"><code class="xref py py-meth docutils literal">remove_header()</code></a> method that can be used to remove

1378

headers from a <a class="reference internal" href="../library/urllib.request.html#urllib.request.Request" title="urllib.request.Request"><code class="xref py py-class docutils literal">Request</code></a>. (Contributed by Alexey

1379

Kachayev in <a class="reference external" href="https://bugs.python.org/issue16464">issue 16464</a>, Daniel Wozniak in <a class="reference external" href="https://bugs.python.org/issue17485">issue 17485</a>, and Damien Brecht

1380

and Senthil Kumaran in <a class="reference external" href="https://bugs.python.org/issue17272">issue 17272</a>.)

1381

<a class="reference internal" href="../library/urllib.error.html#urllib.error.HTTPError" title="urllib.error.HTTPError"><code class="xref py py-class docutils literal">HTTPError</code></a> objects now have a

1382

<a class="reference internal" href="../library/urllib.error.html#urllib.error.HTTPError.headers" title="urllib.error.HTTPError.headers"><code class="xref py py-attr docutils literal">headers</code></a> attribute that provides access to the

1383

HTTP response headers associated with the error. (Contributed by

1384

Berker Peksag in <a class="reference external" href="https://bugs.python.org/issue15701">issue 15701</a>.)

1385

</div>

1386

1387

<h3>unittest<a class="headerlink" href="#unittest" title="Permalink to this headline">¶</a></h3>

1388

The <a class="reference internal" href="../library/unittest.html#unittest.TestCase" title="unittest.TestCase"><code class="xref py py-class docutils literal">TestCase</code></a> class has a new method,

1389

<a class="reference internal" href="../library/unittest.html#unittest.TestCase.subTest" title="unittest.TestCase.subTest"><code class="xref py py-meth docutils literal">subTest()</code></a>, that produces a context manager whose

1390

1391

method to dynamically generate subtests by, say, calling the <code class="docutils literal">subTest</code>

1392

context manager inside a loop. A single test method can thereby produce an

1393

indefinite number of separately-identified and separately-counted tests, all of

1394

which will run even if one or more of them fail. For example:

1395

<div class="highlight-python3"><div class="highlight"><pre>class NumbersTest(unittest.TestCase):

1396

def test_even(self):

1397

for i in range(6):

1398

with self.subTest(i=i):

1399

self.assertEqual(i % 2, 0)

1400

</pre></div>

1401

</div>

1402

will result in six subtests, each identified in the unittest verbose output

1403

with a label consisting of the variable name <code class="docutils literal">i</code> and a particular value for

1404

that variable (<code class="docutils literal">i=0</code>, <code class="docutils literal">i=1</code>, etc). See <a class="reference internal" href="../library/unittest.html#subtests">Distinguishing test iterations using subtests</a> for the full

1405

version of this example. (Contributed by Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue16997">issue 16997</a>.)

1406

<a class="reference internal" href="../library/unittest.html#unittest.main" title="unittest.main"><code class="xref py py-func docutils literal">unittest.main()</code></a> now accepts an iterable of test names for

1407

defaultTest, where previously it only accepted a single test name as a

1408

string. (Contributed by Jyrki Pulliainen in <a class="reference external" href="https://bugs.python.org/issue15132">issue 15132</a>.)

1409

If <a class="reference internal" href="../library/unittest.html#unittest.SkipTest" title="unittest.SkipTest"><code class="xref py py-class docutils literal">SkipTest</code></a> is raised during test discovery (that is, at the

1410

module level in the test file), it is now reported as a skip instead of an

1411

error. (Contributed by Zach Ware in <a class="reference external" href="https://bugs.python.org/issue16935">issue 16935</a>.)

1412

<a class="reference internal" href="../library/unittest.html#unittest.TestLoader.discover" title="unittest.TestLoader.discover"><code class="xref py py-meth docutils literal">discover()</code></a> now sorts the discovered files to provide

1413

consistent test ordering. (Contributed by Martin Melin and Jeff Ramnani in

1414

<a class="reference external" href="https://bugs.python.org/issue16709">issue 16709</a>.)

1415

<a class="reference internal" href="../library/unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">TestSuite</code></a> now drops references to tests as soon as the test

1416

has been run, if the test is successful. On Python interpreters that do

1417

garbage collection, this allows the tests to be garbage collected if nothing

1418

else is holding a reference to the test. It is possible to override this

1419

behavior by creating a <a class="reference internal" href="../library/unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">TestSuite</code></a> subclass that defines a

1420

custom <code class="docutils literal">_removeTestAtIndex</code> method. (Contributed by Tom Wardill, Matt

1421

McClure, and Andrew Svetlov in <a class="reference external" href="https://bugs.python.org/issue11798">issue 11798</a>.)

1422

A new test assertion context-manager, <a class="reference internal" href="../library/unittest.html#unittest.TestCase.assertLogs" title="unittest.TestCase.assertLogs"><code class="xref py py-meth docutils literal">assertLogs()</code></a>,

1423

will ensure that a given block of code emits a log message using the

1424

<a class="reference internal" href="../library/logging.html#module-logging" title="logging: Flexible event logging system for applications."><code class="xref py py-mod docutils literal">logging</code></a> module. By default the message can come from any logger and

1425

have a priority of <code class="docutils literal">INFO</code> or higher, but both the logger name and an

1426

alternative minimum logging level may be specified. The object returned by the

1427

context manager can be queried for the <a class="reference internal" href="../library/logging.html#logging.LogRecord" title="logging.LogRecord"><code class="xref py py-class docutils literal">LogRecord</code></a>s and/or

1428

formatted messages that were logged. (Contributed by Antoine Pitrou in

1429

<a class="reference external" href="https://bugs.python.org/issue18937">issue 18937</a>.)

1430

Test discovery now works with namespace packages (Contributed by Claudiu Popa

1431

in <a class="reference external" href="https://bugs.python.org/issue17457">issue 17457</a>.)

1432

<a class="reference internal" href="../library/unittest.mock.html#module-unittest.mock" title="unittest.mock: Mock object library."><code class="xref py py-mod docutils literal">unittest.mock</code></a> objects now inspect their specification signatures when

1433

matching calls, which means an argument can now be matched by either position

1434

or name, instead of only by position. (Contributed by Antoine Pitrou in

1435

<a class="reference external" href="https://bugs.python.org/issue17015">issue 17015</a>.)

1436

<code class="xref py py-func docutils literal">mock_open()</code> objects now have <code class="docutils literal">readline</code> and <code class="docutils literal">readlines</code>

1437

methods. (Contributed by Toshio Kuratomi in <a class="reference external" href="https://bugs.python.org/issue17467">issue 17467</a>.)

1438

</div>

1439

1440

1441

<a class="reference internal" href="../library/venv.html#module-venv" title="venv: Creation of virtual environments."><code class="xref py py-mod docutils literal">venv</code></a> now includes activation scripts for the <code class="docutils literal">csh</code> and <code class="docutils literal">fish</code>

1442

shells. (Contributed by Andrew Svetlov in <a class="reference external" href="https://bugs.python.org/issue15417">issue 15417</a>.)

1443

<a class="reference internal" href="../library/venv.html#venv.EnvBuilder" title="venv.EnvBuilder"><code class="xref py py-class docutils literal">EnvBuilder</code></a> and the <a class="reference internal" href="../library/venv.html#venv.create" title="venv.create"><code class="xref py py-func docutils literal">create()</code></a> convenience function

1444

take a new keyword argument with_pip, which defaults to <code class="docutils literal">False</code>, that

1445

controls whether or not <a class="reference internal" href="../library/venv.html#venv.EnvBuilder" title="venv.EnvBuilder"><code class="xref py py-class docutils literal">EnvBuilder</code></a> ensures that <code class="docutils literal">pip</code> is

1446

installed in the virtual environment. (Contributed by Nick Coghlan in

1447

<a class="reference external" href="https://bugs.python.org/issue19552">issue 19552</a> as part of the <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0453">PEP 453</a> implementation.)

1448

</div>

1449

1450

1451

The <code class="xref py py-meth docutils literal">getparams()</code> method now returns a namedtuple rather than a

1452

plain tuple. (Contributed by Claudiu Popa in <a class="reference external" href="https://bugs.python.org/issue17487">issue 17487</a>.)

1453

<a class="reference internal" href="../library/wave.html#wave.open" title="wave.open"><code class="xref py py-meth docutils literal">wave.open()</code></a> now supports the context management protocol. (Contributed

1454

by Claudiu Popa in <a class="reference external" href="https://bugs.python.org/issue17616">issue 17616</a>.)

1455

<a class="reference internal" href="../library/wave.html#module-wave" title="wave: Provide an interface to the WAV sound format."><code class="xref py py-mod docutils literal">wave</code></a> can now <a class="reference internal" href="../library/wave.html#wave-write-objects">write output to unseekable files</a>. (Contributed by David Jones, Guilherme Polo, and Serhiy

1456

Storchaka in <a class="reference external" href="https://bugs.python.org/issue5202">issue 5202</a>.)

1457

The <a class="reference internal" href="../library/wave.html#wave.Wave_write.writeframesraw" title="wave.Wave_write.writeframesraw"><code class="xref py py-meth docutils literal">writeframesraw()</code></a> and

1458

<a class="reference internal" href="../library/wave.html#wave.Wave_write.writeframes" title="wave.Wave_write.writeframes"><code class="xref py py-meth docutils literal">writeframes()</code></a> methods now accept any <a class="reference internal" href="../glossary.html#term-bytes-like-object">bytes-like

1459

object</a>. (Contributed by Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue8311">issue 8311</a>.)

1460

</div>

1461

1462

<h3>weakref<a class="headerlink" href="#weakref" title="Permalink to this headline">¶</a></h3>

1463

New <a class="reference internal" href="../library/weakref.html#weakref.WeakMethod" title="weakref.WeakMethod"><code class="xref py py-class docutils literal">WeakMethod</code></a> class simulates weak references to bound

1464

methods. (Contributed by Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue14631">issue 14631</a>.)

1465

New <a class="reference internal" href="../library/weakref.html#weakref.finalize" title="weakref.finalize"><code class="xref py py-class docutils literal">finalize</code></a> class makes it possible to register a callback

1466

to be invoked when an object is garbage collected, without needing to

1467

carefully manage the lifecycle of the weak reference itself. (Contributed by

1468

Richard Oudkerk in <a class="reference external" href="https://bugs.python.org/issue15528">issue 15528</a>.)

1469

The callback, if any, associated with a <a class="reference internal" href="../library/weakref.html#weakref.ref" title="weakref.ref"><code class="xref py py-class docutils literal">ref</code></a> is now

1470

exposed via the <a class="reference internal" href="../library/weakref.html#weakref.ref.__callback__" title="weakref.ref.__callback__"><code class="xref py py-attr docutils literal">__callback__</code></a> attribute. (Contributed

1471

by Mark Dickinson in <a class="reference external" href="https://bugs.python.org/issue17643">issue 17643</a>.)

1472

</div>

1473

1474

<h3>xml.etree<a class="headerlink" href="#xml-etree" title="Permalink to this headline">¶</a></h3>

1475

A new parser, <a class="reference internal" href="../library/xml.etree.elementtree.html#xml.etree.ElementTree.XMLPullParser" title="xml.etree.ElementTree.XMLPullParser"><code class="xref py py-class docutils literal">XMLPullParser</code></a>, allows a

1476

non-blocking applications to parse XML documents. An example can be

1477

seen at <a class="reference internal" href="../library/xml.etree.elementtree.html#elementtree-pull-parsing">Pull API for non-blocking parsing</a>. (Contributed by Antoine

1478

Pitrou in <a class="reference external" href="https://bugs.python.org/issue17741">issue 17741</a>.)

1479

The <a class="reference internal" href="../library/xml.etree.elementtree.html#module-xml.etree.ElementTree" title="xml.etree.ElementTree: Implementation of the ElementTree API."><code class="xref py py-mod docutils literal">xml.etree.ElementTree</code></a> <a class="reference internal" href="../library/xml.etree.elementtree.html#xml.etree.ElementTree.tostring" title="xml.etree.ElementTree.tostring"><code class="xref py py-func docutils literal">tostring()</code></a> and

1480

<a class="reference internal" href="../library/xml.etree.elementtree.html#xml.etree.ElementTree.tostringlist" title="xml.etree.ElementTree.tostringlist"><code class="xref py py-func docutils literal">tostringlist()</code></a> functions, and the

1481

<a class="reference internal" href="../library/xml.etree.elementtree.html#xml.etree.ElementTree.ElementTree" title="xml.etree.ElementTree.ElementTree"><code class="xref py py-class docutils literal">ElementTree</code></a>

1482

<a class="reference internal" href="../library/xml.etree.elementtree.html#xml.etree.ElementTree.ElementTree.write" title="xml.etree.ElementTree.ElementTree.write"><code class="xref py py-meth docutils literal">write()</code></a> method, now have a

1483

short_empty_elements <a class="reference internal" href="../glossary.html#keyword-only-parameter">keyword-only parameter</a>

1484

providing control over whether elements with no content are written in

1485

abbreviated (<code class="docutils literal"><tag /></code>) or expanded (<code class="docutils literal"><tag></tag></code>) form. (Contributed by

1486

Ariel Poliak and Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue14377">issue 14377</a>.)

1487

</div>

1488

1489

<h3>zipfile<a class="headerlink" href="#zipfile" title="Permalink to this headline">¶</a></h3>

1490

The <a class="reference internal" href="../library/zipfile.html#zipfile.PyZipFile.writepy" title="zipfile.PyZipFile.writepy"><code class="xref py py-meth docutils literal">writepy()</code></a> method of the

1491

<a class="reference internal" href="../library/zipfile.html#zipfile.PyZipFile" title="zipfile.PyZipFile"><code class="xref py py-class docutils literal">PyZipFile</code></a> class has a new filterfunc option that can be

1492

used to control which directories and files are added to the archive. For

1493

example, this could be used to exclude test files from the archive.

1494

(Contributed by Christian Tismer in <a class="reference external" href="https://bugs.python.org/issue19274">issue 19274</a>.)

1495

The allowZip64 parameter to <a class="reference internal" href="../library/zipfile.html#zipfile.ZipFile" title="zipfile.ZipFile"><code class="xref py py-class docutils literal">ZipFile</code></a> and

1496

<code class="xref py py-class docutils literal">PyZipfile</code> is now <code class="docutils literal">True</code> by default. (Contributed by

1497

William Mallard in <a class="reference external" href="https://bugs.python.org/issue17201">issue 17201</a>.)

1498

</div>

1499

</div>

1500

1501

<h2>CPython Implementation Changes<a class="headerlink" href="#cpython-implementation-changes" title="Permalink to this headline">¶</a></h2>

1502

1503

<h3>PEP 445: Customization of CPython Memory Allocators<a class="headerlink" href="#pep-445-customization-of-cpython-memory-allocators" title="Permalink to this headline">¶</a></h3>

1504

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0445">PEP 445</a> adds new C level interfaces to customize memory allocation in

1505

the CPython interpreter.

1506

1507

See also

1508

1509

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0445">PEP 445</a> – Add new APIs to customize Python memory allocators</dt>

1510

<dd>PEP written and implemented by Victor Stinner.</dd>

1511

</dl>

1512

</div>

1513

</div>

1514

1515

<h3>PEP 442: Safe Object Finalization<a class="headerlink" href="#pep-442-safe-object-finalization" title="Permalink to this headline">¶</a></h3>

1516

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0442">PEP 442</a> removes the current limitations and quirks of object finalization

1517

in CPython. With it, objects with <a class="reference internal" href="../reference/datamodel.html#object.__del__" title="object.__del__"><code class="xref py py-meth docutils literal">__del__()</code></a> methods, as well as

1518

generators with <a class="reference internal" href="../reference/compound_stmts.html#finally"><code class="xref std std-keyword docutils literal">finally</code></a> clauses, can be finalized when they are

1519

part of a reference cycle.

1520

As part of this change, module globals are no longer forcibly set to

1521

<a class="reference internal" href="../library/constants.html#None" title="None"><code class="xref py py-const docutils literal">None</code></a> during interpreter shutdown in most cases, instead relying

1522

on the normal operation of the cyclic garbage collector. This avoids a

1523

whole class of interpreter-shutdown-time errors, usually involving

1524

<code class="docutils literal">__del__</code> methods, that have plagued Python since the cyclic GC

1525

was first introduced.

1526

1527

See also

1528

1529

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0442">PEP 442</a> – Safe object finalization</dt>

1530

<dd>PEP written and implemented by Antoine Pitrou.</dd>

1531

</dl>

1532

</div>

1533

</div>

1534

1535

<h3>PEP 456: Secure and Interchangeable Hash Algorithm<a class="headerlink" href="#pep-456-secure-and-interchangeable-hash-algorithm" title="Permalink to this headline">¶</a></h3>

1536

<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0456">PEP 456</a> follows up on earlier security fix work done on Python’s hash

1537

algorithm to address certain DOS attacks to which public facing APIs backed by

1538

dictionary lookups may be subject. (See <a class="reference external" href="https://bugs.python.org/issue14621">issue 14621</a> for the start of the

1539

current round of improvements.) The PEP unifies CPython’s hash code to make it

1540

easier for a packager to substitute a different hash algorithm, and switches

1541

Python’s default implementation to a SipHash implementation on platforms that

1542

have a 64 bit data type. Any performance differences in comparison with the

1543

older FNV algorithm are trivial.

1544

The PEP adds additional fields to the <a class="reference internal" href="../library/sys.html#sys.hash_info" title="sys.hash_info"><code class="xref py py-attr docutils literal">sys.hash_info</code></a> struct sequence to

1545

describe the hash algorithm in use by the currently executing binary. Otherwise,

1546

the PEP does not alter any existing CPython APIs.

1547

</div>

1548

1549

<h3>PEP 436: Argument Clinic<a class="headerlink" href="#pep-436-argument-clinic" title="Permalink to this headline">¶</a></h3>

1550

“Argument Clinic” (<a class="pep reference external" href="https://www.python.org/dev/peps/pep-0436">PEP 436</a>) is now part of the CPython build process

1551

and can be used to simplify the process of defining and maintaining

1552

accurate signatures for builtins and standard library extension modules

1553

implemented in C.

1554

Some standard library extension modules have been converted to use Argument

1555

Clinic in Python 3.4, and <a class="reference internal" href="../library/pydoc.html#module-pydoc" title="pydoc: Documentation generator and online help system."><code class="xref py py-mod docutils literal">pydoc</code></a> and <a class="reference internal" href="../library/inspect.html#module-inspect" title="inspect: Extract information and source code from live objects."><code class="xref py py-mod docutils literal">inspect</code></a> have been updated

1556

accordingly.

1557

It is expected that signature metadata for programmatic introspection will

1558

be added to additional callables implemented in C as part of Python 3.4

1559

maintenance releases.

1560

1561

Note

1562

The Argument Clinic PEP is not fully up to date with the state of the

1563

implementation. This has been deemed acceptable by the release manager

1564

and core development team in this case, as Argument Clinic will not

1565

be made available as a public API for third party use in Python 3.4.

1566

</div>

1567

1568

See also

1569

1570

<dt><a class="pep reference external" href="https://www.python.org/dev/peps/pep-0436">PEP 436</a> – The Argument Clinic DSL</dt>

1571

<dd>PEP written and implemented by Larry Hastings.</dd>

1572

</dl>

1573

</div>

1574

</div>

1575

1576

<h3>Other Build and C API Changes<a class="headerlink" href="#other-build-and-c-api-changes" title="Permalink to this headline">¶</a></h3>

1577

1578

<li>The new <a class="reference internal" href="../c-api/type.html#c.PyType_GetSlot" title="PyType_GetSlot"><code class="xref c c-func docutils literal">PyType_GetSlot()</code></a> function has been added to the stable ABI,

1579

allowing retrieval of function pointers from named type slots when using

1580

the limited API. (Contributed by Martin von Löwis in <a class="reference external" href="https://bugs.python.org/issue17162">issue 17162</a>.)</li>

1581

<li>The new <a class="reference internal" href="../c-api/init.html#c.Py_SetStandardStreamEncoding" title="Py_SetStandardStreamEncoding"><code class="xref c c-func docutils literal">Py_SetStandardStreamEncoding()</code></a> pre-initialization API

1582

allows applications embedding the CPython interpreter to reliably force

1583

a particular encoding and error handler for the standard streams.

1584

(Contributed by Bastien Montagne and Nick Coghlan in <a class="reference external" href="https://bugs.python.org/issue16129">issue 16129</a>.)</li>

1585

<li>Most Python C APIs that don’t mutate string arguments are now correctly

1586

marked as accepting <code class="docutils literal">const char *</code> rather than <code class="docutils literal">char *</code>. (Contributed

1587

by Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue1772673">issue 1772673</a>.)</li>

1588

<li>A new shell version of <code class="docutils literal">python-config</code> can be used even when a python

1589

interpreter is not available (for example, in cross compilation scenarios).</li>

1590

<li><a class="reference internal" href="../c-api/unicode.html#c.PyUnicode_FromFormat" title="PyUnicode_FromFormat"><code class="xref c c-func docutils literal">PyUnicode_FromFormat()</code></a> now supports width and precision

1591

specifications for <code class="docutils literal">%s</code>, <code class="docutils literal">%A</code>, <code class="docutils literal">%U</code>, <code class="docutils literal">%V</code>, <code class="docutils literal">%S</code>, and <code class="docutils literal">%R</code>.

1592

(Contributed by Ysj Ray and Victor Stinner in <a class="reference external" href="https://bugs.python.org/issue7330">issue 7330</a>.)</li>

1593

<li>New function <a class="reference internal" href="../c-api/tuple.html#c.PyStructSequence_InitType2" title="PyStructSequence_InitType2"><code class="xref c c-func docutils literal">PyStructSequence_InitType2()</code></a> supplements the

1594

existing <a class="reference internal" href="../c-api/tuple.html#c.PyStructSequence_InitType" title="PyStructSequence_InitType"><code class="xref c c-func docutils literal">PyStructSequence_InitType()</code></a> function. The difference

1595

is that it returns <code class="docutils literal">0</code> on success and <code class="docutils literal">-1</code> on failure.</li>

1596

<li>The CPython source can now be compiled using the address sanity checking

1597

features of recent versions of GCC and clang: the false alarms in the small

1598

object allocator have been silenced. (Contributed by Dhiru Kholia in

1599

<a class="reference external" href="https://bugs.python.org/issue18596">issue 18596</a>.)</li>

1600

<li>The Windows build now uses <a class="reference external" href="https://en.wikipedia.org/wiki/Address_space_layout_randomization">Address Space Layout Randomization</a> and <a class="reference external" href="https://en.wikipedia.org/wiki/Data_Execution_Prevention">Data Execution Prevention</a>. (Contributed by

1601

Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue16632">issue 16632</a>.)</li>

1602

<li>New function <a class="reference internal" href="../c-api/object.html#c.PyObject_LengthHint" title="PyObject_LengthHint"><code class="xref c c-func docutils literal">PyObject_LengthHint()</code></a> is the C API equivalent

1603

of <a class="reference internal" href="../library/operator.html#operator.length_hint" title="operator.length_hint"><code class="xref py py-func docutils literal">operator.length_hint()</code></a>. (Contributed by Armin Ronacher in

1604

<a class="reference external" href="https://bugs.python.org/issue16148">issue 16148</a>.)</li>

1605

</ul>

1606

</div>

1607

1608

<h3>Other Improvements<a class="headerlink" href="#other-improvements" title="Permalink to this headline">¶</a></h3>

1609

1610

<li>The <a class="reference internal" href="../using/cmdline.html#using-on-cmdline">python</a> command has a new <a class="reference internal" href="../using/cmdline.html#using-on-misc-options">option</a>, <code class="docutils literal">-I</code>, which causes it to run in “isolated mode”,

1611

which means that <a class="reference internal" href="../library/sys.html#sys.path" title="sys.path"><code class="xref py py-data docutils literal">sys.path</code></a> contains neither the script’s directory nor

1612

the user’s <code class="docutils literal">site-packages</code> directory, and all <code class="xref std std-envvar docutils literal">PYTHON*</code> environment

1613

variables are ignored (it implies both <code class="docutils literal">-s</code> and <code class="docutils literal">-E</code>). Other

1614

restrictions may also be applied in the future, with the goal being to

1615

isolate the execution of a script from the user’s environment. This is

1616

appropriate, for example, when Python is used to run a system script. On

1617

most POSIX systems it can and should be used in the <code class="docutils literal">#!</code> line of system

1618

scripts. (Contributed by Christian Heimes in <a class="reference external" href="https://bugs.python.org/issue16499">issue 16499</a>.)</li>

1619

<li>Tab-completion is now enabled by default in the interactive interpreter

1620

on systems that support <a class="reference internal" href="../library/readline.html#module-readline" title="readline: GNU readline support for Python. (Unix)"><code class="xref py py-mod docutils literal">readline</code></a>. History is also enabled by default,

1621

and is written to (and read from) the file <code class="file docutils literal">~/.python-history</code>.

1622

(Contributed by Antoine Pitrou and Éric Araujo in <a class="reference external" href="https://bugs.python.org/issue5845">issue 5845</a>.)</li>

1623

<li>Invoking the Python interpreter with <code class="docutils literal">--version</code> now outputs the version to

1624

standard output instead of standard error (<a class="reference external" href="https://bugs.python.org/issue18338">issue 18338</a>). Similar changes

1625

were made to <a class="reference internal" href="../library/argparse.html#module-argparse" title="argparse: Command-line option and argument parsing library."><code class="xref py py-mod docutils literal">argparse</code></a> (<a class="reference external" href="https://bugs.python.org/issue18920">issue 18920</a>) and other modules that have

1626

script-like invocation capabilities (<a class="reference external" href="https://bugs.python.org/issue18922">issue 18922</a>).</li>

1627

<li>The CPython Windows installer now adds <code class="docutils literal">.py</code> to the <code class="xref std std-envvar docutils literal">PATHEXT</code>

1628

variable when extensions are registered, allowing users to run a python

1629

script at the windows command prompt by just typing its name without the

1630

<code class="docutils literal">.py</code> extension. (Contributed by Paul Moore in <a class="reference external" href="https://bugs.python.org/issue18569">issue 18569</a>.)</li>

1631

<li>A new <code class="docutils literal">make</code> target <a class="reference external" href="https://docs.python.org/devguide/coverage.html#measuring-coverage-of-c-code-with-gcov-and-lcov">coverage-report</a>

1632

will build python, run the test suite, and generate an HTML coverage report

1633

for the C codebase using <code class="docutils literal">gcov</code> and <a class="reference external" href="http://ltp.sourceforge.net/coverage/lcov.php">lcov</a>.</li>

1634

<li>The <code class="docutils literal">-R</code> option to the <a class="reference internal" href="../library/test.html#regrtest">python regression test suite</a> now

1635

also checks for memory allocation leaks, using

1636

<a class="reference internal" href="../library/sys.html#sys.getallocatedblocks" title="sys.getallocatedblocks"><code class="xref py py-func docutils literal">sys.getallocatedblocks()</code></a>. (Contributed by Antoine Pitrou in

1637

<a class="reference external" href="https://bugs.python.org/issue13390">issue 13390</a>.)</li>

1638

<li><code class="docutils literal">python -m</code> now works with namespace packages.</li>

1639

<li>The <a class="reference internal" href="../library/stat.html#module-stat" title="stat: Utilities for interpreting the results of os.stat(), os.lstat() and os.fstat()."><code class="xref py py-mod docutils literal">stat</code></a> module is now implemented in C, which means it gets the

1640

values for its constants from the C header files, instead of having the

1641

values hard-coded in the python module as was previously the case.</li>

1642

<li>Loading multiple python modules from a single OS module (<code class="docutils literal">.so</code>, <code class="docutils literal">.dll</code>)

1643

now works correctly (previously it silently returned the first python

1644

module in the file). (Contributed by Václav Šmilauer in <a class="reference external" href="https://bugs.python.org/issue16421">issue 16421</a>.)</li>

1645

<li>A new opcode, <a class="reference internal" href="../library/dis.html#opcode-LOAD_CLASSDEREF"><code class="xref std std-opcode docutils literal">LOAD_CLASSDEREF</code></a>, has been added to fix a bug in the

1646

loading of free variables in class bodies that could be triggered by certain

1647

uses of <a class="reference internal" href="../reference/datamodel.html#prepare">__prepare__</a>. (Contributed by Benjamin Peterson in

1648

<a class="reference external" href="https://bugs.python.org/issue17853">issue 17853</a>.)</li>

1649

<li>A number of MemoryError-related crashes were identified and fixed by Victor

1650

Stinner using his <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0445">PEP 445</a>-based <code class="docutils literal">pyfailmalloc</code> tool (<a class="reference external" href="https://bugs.python.org/issue18408">issue 18408</a>,

1651

<a class="reference external" href="https://bugs.python.org/issue18520">issue 18520</a>).</li>

1652

<li>The <a class="reference internal" href="../using/scripts.html#scripts-pyvenv">pyvenv</a> command now accepts a <code class="docutils literal">--copies</code> option

1653

to use copies rather than symlinks even on systems where symlinks are the

1654

default. (Contributed by Vinay Sajip in <a class="reference external" href="https://bugs.python.org/issue18807">issue 18807</a>.)</li>

1655

<li>The <a class="reference internal" href="../using/scripts.html#scripts-pyvenv">pyvenv</a> command also accepts a <code class="docutils literal">--without-pip</code>

1656

option to suppress the otherwise-automatic bootstrapping of pip into

1657

the virtual environment. (Contributed by Nick Coghlan in <a class="reference external" href="https://bugs.python.org/issue19552">issue 19552</a>

1658

as part of the <a class="pep reference external" href="https://www.python.org/dev/peps/pep-0453">PEP 453</a> implementation.)</li>

1659

<li>The encoding name is now optional in the value set for the

1660

<a class="reference internal" href="../using/cmdline.html#envvar-PYTHONIOENCODING"><code class="xref std std-envvar docutils literal">PYTHONIOENCODING</code></a> environment variable. This makes it possible to

1661

set just the error handler, without changing the default encoding.

1662

(Contributed by Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue18818">issue 18818</a>.)</li>

1663

<li>The <a class="reference internal" href="../library/bz2.html#module-bz2" title="bz2: Interfaces for bzip2 compression and decompression."><code class="xref py py-mod docutils literal">bz2</code></a>, <a class="reference internal" href="../library/lzma.html#module-lzma" title="lzma: A Python wrapper for the liblzma compression library."><code class="xref py py-mod docutils literal">lzma</code></a>, and <a class="reference internal" href="../library/gzip.html#module-gzip" title="gzip: Interfaces for gzip compression and decompression using file objects."><code class="xref py py-mod docutils literal">gzip</code></a> module <code class="docutils literal">open</code> functions now

1664

support <code class="docutils literal">x</code> (exclusive creation) mode. (Contributed by Tim Heaney and

1665

Vajrasky Kok in <a class="reference external" href="https://bugs.python.org/issue19201">issue 19201</a>, <a class="reference external" href="https://bugs.python.org/issue19222">issue 19222</a>, and <a class="reference external" href="https://bugs.python.org/issue19223">issue 19223</a>.)</li>

1666

</ul>

1667

</div>

1668

1669

<h3>Significant Optimizations<a class="headerlink" href="#significant-optimizations" title="Permalink to this headline">¶</a></h3>

1670

1671

<li>The UTF-32 decoder is now 3x to 4x faster. (Contributed by Serhiy Storchaka

1672

in <a class="reference external" href="https://bugs.python.org/issue14625">issue 14625</a>.)</li>

1673

<li>The cost of hash collisions for sets is now reduced. Each hash table

1674

probe now checks a series of consecutive, adjacent key/hash pairs before

1675

continuing to make random probes through the hash table. This exploits

1676

cache locality to make collision resolution less expensive.

1677

The collision resolution scheme can be described as a hybrid of linear

1678

probing and open addressing. The number of additional linear probes

1679

defaults to nine. This can be changed at compile-time by defining

1680

LINEAR_PROBES to be any value. Set LINEAR_PROBES=0 to turn-off

1681

linear probing entirely. (Contributed by Raymond Hettinger in

1682

<a class="reference external" href="https://bugs.python.org/issue18771">issue 18771</a>.)</li>

1683

<li>The interpreter starts about 30% faster. A couple of measures lead to the

1684

speedup. The interpreter loads fewer modules on startup, e.g. the <a class="reference internal" href="../library/re.html#module-re" title="re: Regular expression operations."><code class="xref py py-mod docutils literal">re</code></a>,

1685

<a class="reference internal" href="../library/collections.html#module-collections" title="collections: Container datatypes"><code class="xref py py-mod docutils literal">collections</code></a> and <a class="reference internal" href="../library/locale.html#module-locale" title="locale: Internationalization services."><code class="xref py py-mod docutils literal">locale</code></a> modules and their dependencies are no

1686

longer imported by default. The marshal module has been improved to load

1687

compiled Python code faster. (Contributed by Antoine Pitrou, Christian

1688

Heimes and Victor Stinner in <a class="reference external" href="https://bugs.python.org/issue19219">issue 19219</a>, <a class="reference external" href="https://bugs.python.org/issue19218">issue 19218</a>, <a class="reference external" href="https://bugs.python.org/issue19209">issue 19209</a>,

1689

<a class="reference external" href="https://bugs.python.org/issue19205">issue 19205</a> and <a class="reference external" href="https://bugs.python.org/issue9548">issue 9548</a>.)</li>

1690

<li><a class="reference internal" href="../library/bz2.html#bz2.BZ2File" title="bz2.BZ2File"><code class="xref py py-class docutils literal">bz2.BZ2File</code></a> is now as fast or faster than the Python2 version for

1691

most cases. <a class="reference internal" href="../library/lzma.html#lzma.LZMAFile" title="lzma.LZMAFile"><code class="xref py py-class docutils literal">lzma.LZMAFile</code></a> has also been optimized. (Contributed by

1692

Serhiy Storchaka and Nadeem Vawda in <a class="reference external" href="https://bugs.python.org/issue16034">issue 16034</a>.)</li>

1693

<li><a class="reference internal" href="../library/random.html#random.getrandbits" title="random.getrandbits"><code class="xref py py-func docutils literal">random.getrandbits()</code></a> is 20%-40% faster for small integers (the most

1694

common use case). (Contributed by Serhiy Storchaka in <a class="reference external" href="https://bugs.python.org/issue16674">issue 16674</a>.)</li>

1695

<li>By taking advantage of the new storage format for strings, pickling of

1696

strings is now significantly faster. (Contributed by Victor Stinner and

1697

Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue15596">issue 15596</a>.)</li>

1698

<li>A performance issue in <code class="xref py py-meth docutils literal">io.FileIO.readall()</code> has been solved. This

1699

particularly affects Windows, and significantly speeds up the case of piping

1700

significant amounts of data through <a class="reference internal" href="../library/subprocess.html#module-subprocess" title="subprocess: Subprocess management."><code class="xref py py-mod docutils literal">subprocess</code></a>. (Contributed

1701

by Richard Oudkerk in <a class="reference external" href="https://bugs.python.org/issue15758">issue 15758</a>.)</li>

1702

<li><a class="reference internal" href="../library/html.html#html.escape" title="html.escape"><code class="xref py py-func docutils literal">html.escape()</code></a> is now 10x faster. (Contributed by Matt Bryant in

1703

<a class="reference external" href="https://bugs.python.org/issue18020">issue 18020</a>.)</li>

1704

<li>On Windows, the native <code class="docutils literal">VirtualAlloc</code> is now used instead of the CRT

1705

<code class="docutils literal">malloc</code> in <code class="docutils literal">obmalloc</code>. Artificial benchmarks show about a 3% memory

1706

savings.</li>

1707

<li><a class="reference internal" href="../library/os.html#os.urandom" title="os.urandom"><code class="xref py py-func docutils literal">os.urandom()</code></a> now uses a lazily-opened persistent file descriptor

1708

so as to avoid using many file descriptors when run in parallel from

1709

multiple threads. (Contributed by Antoine Pitrou in <a class="reference external" href="https://bugs.python.org/issue18756">issue 18756</a>.)</li>

1710

</ul>

1711

</div>

1712

</div>

1713

1714

<h2>Deprecated<a class="headerlink" href="#deprecated" title="Permalink to this headline">¶</a></h2>

1715

This section covers various APIs and other features that have been deprecated

1716

in Python 3.4, and will be removed in Python 3.5 or later. In most (but not

1717

all) cases, using the deprecated APIs will produce a <a class="reference internal" href="../library/exceptions.html#DeprecationWarning" title="DeprecationWarning"><code class="xref py py-exc docutils literal">DeprecationWarning</code></a>

1718

when the interpreter is run with deprecation warnings enabled (for example, by

1719

using <code class="docutils literal">-Wd</code>).

1720

1721

<h3>Deprecations in the Python API<a class="headerlink" href="#deprecations-in-the-python-api" title="Permalink to this headline">¶</a></h3>

1722

1723

<li>As mentioned in <a class="reference internal" href="#whatsnew-pep-451">PEP 451: A ModuleSpec Type for the Import System</a>, a number of <a class="reference internal" href="../library/importlib.html#module-importlib" title="importlib: The implementation of the import machinery."><code class="xref py py-mod docutils literal">importlib</code></a>

1724

methods and functions are deprecated: <a class="reference internal" href="../library/importlib.html#importlib.find_loader" title="importlib.find_loader"><code class="xref py py-meth docutils literal">importlib.find_loader()</code></a> is

1725

replaced by <a class="reference internal" href="../library/importlib.html#importlib.util.find_spec" title="importlib.util.find_spec"><code class="xref py py-func docutils literal">importlib.util.find_spec()</code></a>;

1726

<a class="reference internal" href="../library/importlib.html#importlib.machinery.PathFinder.find_module" title="importlib.machinery.PathFinder.find_module"><code class="xref py py-meth docutils literal">importlib.machinery.PathFinder.find_module()</code></a> is replaced by

1727

<a class="reference internal" href="../library/importlib.html#importlib.machinery.PathFinder.find_spec" title="importlib.machinery.PathFinder.find_spec"><code class="xref py py-meth docutils literal">importlib.machinery.PathFinder.find_spec()</code></a>;

1728

<a class="reference internal" href="../library/importlib.html#importlib.abc.MetaPathFinder.find_module" title="importlib.abc.MetaPathFinder.find_module"><code class="xref py py-meth docutils literal">importlib.abc.MetaPathFinder.find_module()</code></a> is replaced by

1729

<a class="reference internal" href="../library/importlib.html#importlib.abc.MetaPathFinder.find_spec" title="importlib.abc.MetaPathFinder.find_spec"><code class="xref py py-meth docutils literal">importlib.abc.MetaPathFinder.find_spec()</code></a>;

1730

<a class="reference internal" href="../library/importlib.html#importlib.abc.PathEntryFinder.find_loader" title="importlib.abc.PathEntryFinder.find_loader"><code class="xref py py-meth docutils literal">importlib.abc.PathEntryFinder.find_loader()</code></a> and

1731

<a class="reference internal" href="../library/importlib.html#importlib.abc.PathEntryFinder.find_module" title="importlib.abc.PathEntryFinder.find_module"><code class="xref py py-meth docutils literal">find_module()</code></a> are replaced by

1732

<a class="reference internal" href="../library/importlib.html#importlib.abc.PathEntryFinder.find_spec" title="importlib.abc.PathEntryFinder.find_spec"><code class="xref py py-meth docutils literal">importlib.abc.PathEntryFinder.find_spec()</code></a>; all of the <code class="docutils literal">xxxLoader</code> ABC

1733

<code class="docutils literal">load_module</code> methods (<a class="reference internal" href="../library/importlib.html#importlib.abc.Loader.load_module" title="importlib.abc.Loader.load_module"><code class="xref py py-meth docutils literal">importlib.abc.Loader.load_module()</code></a>,

1734

<a class="reference internal" href="../library/importlib.html#importlib.abc.InspectLoader.load_module" title="importlib.abc.InspectLoader.load_module"><code class="xref py py-meth docutils literal">importlib.abc.InspectLoader.load_module()</code></a>,

1735

<a class="reference internal" href="../library/importlib.html#importlib.abc.FileLoader.load_module" title="importlib.abc.FileLoader.load_module"><code class="xref py py-meth docutils literal">importlib.abc.FileLoader.load_module()</code></a>,

1736

<a class="reference internal" href="../library/importlib.html#importlib.abc.SourceLoader.load_module" title="importlib.abc.SourceLoader.load_module"><code class="xref py py-meth docutils literal">importlib.abc.SourceLoader.load_module()</code></a>) should no longer be

1737

implemented, instead loaders should implement an

1738

<code class="docutils literal">exec_module</code> method

1739

(<a class="reference internal" href="../library/importlib.html#importlib.abc.Loader.exec_module" title="importlib.abc.Loader.exec_module"><code class="xref py py-meth docutils literal">importlib.abc.Loader.exec_module()</code></a>,

1740

<a class="reference internal" href="../library/importlib.html#importlib.abc.InspectLoader.exec_module" title="importlib.abc.InspectLoader.exec_module"><code class="xref py py-meth docutils literal">importlib.abc.InspectLoader.exec_module()</code></a>

1741

<a class="reference internal" href="../library/importlib.html#importlib.abc.SourceLoader.exec_module" title="importlib.abc.SourceLoader.exec_module"><code class="xref py py-meth docutils literal">importlib.abc.SourceLoader.exec_module()</code></a>) and let the import system

1742

take care of the rest; and

1743

<a class="reference internal" href="../library/importlib.html#importlib.abc.Loader.module_repr" title="importlib.abc.Loader.module_repr"><code class="xref py py-meth docutils literal">importlib.abc.Loader.module_repr()</code></a>,

1744

<a class="reference internal" href="../library/importlib.html#importlib.util.module_for_loader" title="importlib.util.module_for_loader"><code class="xref py py-meth docutils literal">importlib.util.module_for_loader()</code></a>, <a class="reference internal" href="../library/importlib.html#importlib.util.set_loader" title="importlib.util.set_loader"><code class="xref py py-meth docutils literal">importlib.util.set_loader()</code></a>,

1745

and <a class="reference internal" href="../library/importlib.html#importlib.util.set_package" title="importlib.util.set_package"><code class="xref py py-meth docutils literal">importlib.util.set_package()</code></a> are no longer needed because their

1746

functions are now handled automatically by the import system.</li>

1747

<li>The <a class="reference internal" href="../library/imp.html#module-imp" title="imp: Access the implementation of the import statement. (deprecated)"><code class="xref py py-mod docutils literal">imp</code></a> module is pending deprecation. To keep compatibility with

1748

Python 2/3 code bases, the module’s removal is currently not scheduled.</li>

1749

<li>The <a class="reference internal" href="../library/formatter.html#module-formatter" title="formatter: Generic output formatter and device interface. (deprecated)"><code class="xref py py-mod docutils literal">formatter</code></a> module is pending deprecation and is slated for removal

1750

in Python 3.6.</li>

1751

<li><code class="docutils literal">MD5</code> as the default digestmod for the <a class="reference internal" href="../library/hmac.html#hmac.new" title="hmac.new"><code class="xref py py-func docutils literal">hmac.new()</code></a> function is

1752

deprecated. Python 3.6 will require an explicit digest name or constructor as

1753

digestmod argument.</li>

1754

<li>The internal <code class="docutils literal">Netrc</code> class in the <a class="reference internal" href="../library/ftplib.html#module-ftplib" title="ftplib: FTP protocol client (requires sockets)."><code class="xref py py-mod docutils literal">ftplib</code></a> module has been documented

1755

as deprecated in its docstring for quite some time. It now emits a

1756

<a class="reference internal" href="../library/exceptions.html#DeprecationWarning" title="DeprecationWarning"><code class="xref py py-exc docutils literal">DeprecationWarning</code></a> and will be removed completely in Python 3.5.</li>

1757

<li>The undocumented endtime argument to <a class="reference internal" href="../library/subprocess.html#subprocess.Popen.wait" title="subprocess.Popen.wait"><code class="xref py py-meth docutils literal">subprocess.Popen.wait()</code></a> should

1758

not have been exposed and is hopefully not in use; it is deprecated and

1759

will mostly likely be removed in Python 3.5.</li>

1760

<li>The strict argument of <a class="reference internal" href="../library/html.parser.html#html.parser.HTMLParser" title="html.parser.HTMLParser"><code class="xref py py-class docutils literal">HTMLParser</code></a> is deprecated.</li>

1761

<li>The <a class="reference internal" href="../library/plistlib.html#module-plistlib" title="plistlib: Generate and parse Mac OS X plist files."><code class="xref py py-mod docutils literal">plistlib</code></a> <a class="reference internal" href="../library/plistlib.html#plistlib.readPlist" title="plistlib.readPlist"><code class="xref py py-func docutils literal">readPlist()</code></a>,

1762

<a class="reference internal" href="../library/plistlib.html#plistlib.writePlist" title="plistlib.writePlist"><code class="xref py py-func docutils literal">writePlist()</code></a>, <a class="reference internal" href="../library/plistlib.html#plistlib.readPlistFromBytes" title="plistlib.readPlistFromBytes"><code class="xref py py-func docutils literal">readPlistFromBytes()</code></a>, and

1763

<a class="reference internal" href="../library/plistlib.html#plistlib.writePlistToBytes" title="plistlib.writePlistToBytes"><code class="xref py py-func docutils literal">writePlistToBytes()</code></a> functions are deprecated in favor of the

1764

corresponding new functions <a class="reference internal" href="../library/plistlib.html#plistlib.load" title="plistlib.load"><code class="xref py py-func docutils literal">load()</code></a>, <a class="reference internal" href="../library/plistlib.html#plistlib.dump" title="plistlib.dump"><code class="xref py py-func docutils literal">dump()</code></a>,

1765

<a class="reference internal" href="../library/plistlib.html#plistlib.loads" title="plistlib.loads"><code class="xref py py-func docutils literal">loads()</code></a>, and <a class="reference internal" href="../library/plistlib.html#plistlib.dumps" title="plistlib.dumps"><code class="xref py py-func docutils literal">dumps()</code></a>. <a class="reference internal" href="../library/plistlib.html#plistlib.Data" title="plistlib.Data"><code class="xref py py-func docutils literal">Data()</code></a>

1766

is deprecated in favor of just using the <a class="reference internal" href="../library/functions.html#bytes" title="bytes"><code class="xref py py-class docutils literal">bytes</code></a> constructor.</li>

1767

<li>The <a class="reference internal" href="../library/sysconfig.html#module-sysconfig" title="sysconfig: Python's configuration information"><code class="xref py py-mod docutils literal">sysconfig</code></a> key <code class="docutils literal">SO</code> is deprecated, it has been replaced by

1768

<code class="docutils literal">EXT_SUFFIX</code>.</li>

1769

<li>The <code class="docutils literal">U</code> mode accepted by various <code class="docutils literal">open</code> functions is deprecated.

1770

In Python3 it does not do anything useful, and should be replaced by

1771

appropriate uses of <a class="reference internal" href="../library/io.html#io.TextIOWrapper" title="io.TextIOWrapper"><code class="xref py py-class docutils literal">io.TextIOWrapper</code></a> (if needed) and its newline

1772

argument.</li>

1773

<li>The parser argument of <a class="reference internal" href="../library/xml.etree.elementtree.html#xml.etree.ElementTree.iterparse" title="xml.etree.ElementTree.iterparse"><code class="xref py py-func docutils literal">xml.etree.ElementTree.iterparse()</code></a> has

1774

been deprecated, as has the html argument of

1775

<a class="reference internal" href="../library/xml.etree.elementtree.html#xml.etree.ElementTree.XMLParser" title="xml.etree.ElementTree.XMLParser"><code class="xref py py-func docutils literal">XMLParser()</code></a>. To prepare for the removal of the

1776

latter, all arguments to <code class="docutils literal">XMLParser</code> should be passed by keyword.</li>

1777

</ul>

1778

</div>

1779

1780

<h3>Deprecated Features<a class="headerlink" href="#deprecated-features" title="Permalink to this headline">¶</a></h3>

1781

1782

<li>Running <a class="reference internal" href="../library/idle.html#idle">IDLE</a> with the <code class="docutils literal">-n</code> flag (no subprocess) is deprecated.

1783

However, the feature will not be removed until <a class="reference external" href="https://bugs.python.org/issue18823">issue 18823</a> is resolved.</li>

1784

<li>The site module adding a “site-python” directory to sys.path, if it

1785

exists, is deprecated (<a class="reference external" href="https://bugs.python.org/issue19375">issue 19375</a>).</li>

1786

</ul>

1787

</div>

1788

</div>

1789

1790

<h2>Removed<a class="headerlink" href="#removed" title="Permalink to this headline">¶</a></h2>

1791

1792

<h3>Operating Systems No Longer Supported<a class="headerlink" href="#operating-systems-no-longer-supported" title="Permalink to this headline">¶</a></h3>

1793

Support for the following operating systems has been removed from the source

1794

and build tools:

1795

1796

<li>OS/2 (<a class="reference external" href="https://bugs.python.org/issue16135">issue 16135</a>).</li>

1797

<li>Windows 2000 (changeset e52df05b496a).</li>

1798

<li>Windows systems where <code class="docutils literal">COMSPEC</code> points to <code class="docutils literal">command.com</code> (<a class="reference external" href="https://bugs.python.org/issue14470">issue 14470</a>).</li>

1799

<li>VMS (<a class="reference external" href="https://bugs.python.org/issue16136">issue 16136</a>).</li>

1800

</ul>

1801

</div>

1802

1803

<h3>API and Feature Removals<a class="headerlink" href="#api-and-feature-removals" title="Permalink to this headline">¶</a></h3>

1804

The following obsolete and previously deprecated APIs and features have been

1805

removed:

1806

1807

<li>The unmaintained <code class="docutils literal">Misc/TextMate</code> and <code class="docutils literal">Misc/vim</code> directories have been

1808

removed (see the <a class="reference external" href="https://docs.python.org/devguide">devguide</a>

1809

for suggestions on what to use instead).</li>

1810

<li>The <code class="docutils literal">SO</code> makefile macro is removed (it was replaced by the

1811

<code class="docutils literal">SHLIB_SUFFIX</code> and <code class="docutils literal">EXT_SUFFIX</code> macros) (<a class="reference external" href="https://bugs.python.org/issue16754">issue 16754</a>).</li>

1812

<li>The <code class="docutils literal">PyThreadState.tick_counter</code> field has been removed; its value has

1813

been meaningless since Python 3.2, when the “new GIL” was introduced

1814

(<a class="reference external" href="https://bugs.python.org/issue19199">issue 19199</a>).</li>

1815

<li><code class="docutils literal">PyLoader</code> and <code class="docutils literal">PyPycLoader</code> have been removed from <a class="reference internal" href="../library/importlib.html#module-importlib" title="importlib: The implementation of the import machinery."><code class="xref py py-mod docutils literal">importlib</code></a>.

1816

(Contributed by Taras Lyapun in <a class="reference external" href="https://bugs.python.org/issue15641">issue 15641</a>.)</li>

1817

<li>The strict argument to <a class="reference internal" href="../library/http.client.html#http.client.HTTPConnection" title="http.client.HTTPConnection"><code class="xref py py-class docutils literal">HTTPConnection</code></a> and

1818

<a class="reference internal" href="../library/http.client.html#http.client.HTTPSConnection" title="http.client.HTTPSConnection"><code class="xref py py-class docutils literal">HTTPSConnection</code></a> has been removed. HTTP 0.9-style

1819

“Simple Responses” are no longer supported.</li>

1820

<li>The deprecated <a class="reference internal" href="../library/urllib.request.html#urllib.request.Request" title="urllib.request.Request"><code class="xref py py-mod docutils literal">urllib.request.Request</code></a> getter and setter methods

1821

<code class="docutils literal">add_data</code>, <code class="docutils literal">has_data</code>, <code class="docutils literal">get_data</code>, <code class="docutils literal">get_type</code>, <code class="docutils literal">get_host</code>,

1822

<code class="docutils literal">get_selector</code>, <code class="docutils literal">set_proxy</code>, <code class="docutils literal">get_origin_req_host</code>, and

1823

<code class="docutils literal">is_unverifiable</code> have been removed (use direct attribute access instead).</li>

1824

<li>Support for loading the deprecated <code class="docutils literal">TYPE_INT64</code> has been removed from

1825

<a class="reference internal" href="../library/marshal.html#module-marshal" title="marshal: Convert Python objects to streams of bytes and back (with different constraints)."><code class="xref py py-mod docutils literal">marshal</code></a>. (Contributed by Dan Riti in <a class="reference external" href="https://bugs.python.org/issue15480">issue 15480</a>.)</li>

1826

<li><a class="reference internal" href="../library/inspect.html#inspect.Signature" title="inspect.Signature"><code class="xref py py-class docutils literal">inspect.Signature</code></a>: positional-only parameters are now required

1827

to have a valid name.</li>

1828

<li><a class="reference internal" href="../reference/datamodel.html#object.__format__" title="object.__format__"><code class="xref py py-meth docutils literal">object.__format__()</code></a> no longer accepts non-empty format strings, it now

1829

raises a <a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a> instead. Using a non-empty string has been

1830

deprecated since Python 3.2. This change has been made to prevent a

1831

situation where previously working (but incorrect) code would start failing

1832

if an object gained a __format__ method, which means that your code may now

1833

raise a <a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a> if you are using an <code class="docutils literal">'s'</code> format code with objects

1834

that do not have a __format__ method that handles it. See <a class="reference external" href="https://bugs.python.org/issue7994">issue 7994</a> for

1835

background.</li>

1836

<li><code class="xref py py-meth docutils literal">difflib.SequenceMatcher.isbjunk()</code> and

1837

<code class="xref py py-meth docutils literal">difflib.SequenceMatcher.isbpopular()</code> were deprecated in 3.2, and have

1838

now been removed: use <code class="docutils literal">x in sm.bjunk</code> and

1839

<code class="docutils literal">x in sm.bpopular</code>, where sm is a <a class="reference internal" href="../library/difflib.html#difflib.SequenceMatcher" title="difflib.SequenceMatcher"><code class="xref py py-class docutils literal">SequenceMatcher</code></a> object

1840

(<a class="reference external" href="https://bugs.python.org/issue13248">issue 13248</a>).</li>

1841

</ul>

1842

</div>

1843

1844

<h3>Code Cleanups<a class="headerlink" href="#code-cleanups" title="Permalink to this headline">¶</a></h3>

1845

1846

<li>The unused and undocumented internal <code class="docutils literal">Scanner</code> class has been removed from

1847

the <a class="reference internal" href="../library/pydoc.html#module-pydoc" title="pydoc: Documentation generator and online help system."><code class="xref py py-mod docutils literal">pydoc</code></a> module.</li>

1848

<li>The private and effectively unused <code class="docutils literal">_gestalt</code> module has been removed,

1849

along with the private <a class="reference internal" href="../library/platform.html#module-platform" title="platform: Retrieves as much platform identifying data as possible."><code class="xref py py-mod docutils literal">platform</code></a> functions <code class="docutils literal">_mac_ver_lookup</code>,

1850

<code class="docutils literal">_mac_ver_gstalt</code>, and <code class="docutils literal">_bcd2str</code>, which would only have ever been called

1851

on badly broken OSX systems (see <a class="reference external" href="https://bugs.python.org/issue18393">issue 18393</a>).</li>

1852

<li>The hardcoded copies of certain <a class="reference internal" href="../library/stat.html#module-stat" title="stat: Utilities for interpreting the results of os.stat(), os.lstat() and os.fstat()."><code class="xref py py-mod docutils literal">stat</code></a> constants that were included in

1853

the <a class="reference internal" href="../library/tarfile.html#module-tarfile" title="tarfile: Read and write tar-format archive files."><code class="xref py py-mod docutils literal">tarfile</code></a> module namespace have been removed.</li>

1854

</ul>

1855

</div>

1856

</div>

1857

1858

<h2>Porting to Python 3.4<a class="headerlink" href="#porting-to-python-3-4" title="Permalink to this headline">¶</a></h2>

1859

This section lists previously described changes and other bugfixes

1860

that may require changes to your code.

1861

1862

<h3>Changes in ‘python’ Command Behavior<a class="headerlink" href="#changes-in-python-command-behavior" title="Permalink to this headline">¶</a></h3>

1863

1864

<li>In a posix shell, setting the <code class="xref std std-envvar docutils literal">PATH</code> environment variable to

1865

an empty value is equivalent to not setting it at all. However, setting

1866

<a class="reference internal" href="../using/cmdline.html#envvar-PYTHONPATH"><code class="xref std std-envvar docutils literal">PYTHONPATH</code></a> to an empty value was not equivalent to not setting it

1867

at all: setting <a class="reference internal" href="../using/cmdline.html#envvar-PYTHONPATH"><code class="xref std std-envvar docutils literal">PYTHONPATH</code></a> to an empty value was equivalent to

1868

setting it to <code class="docutils literal">.</code>, which leads to confusion when reasoning by analogy to

1869

how <code class="xref std std-envvar docutils literal">PATH</code> works. The behavior now conforms to the posix convention

1870

for <code class="xref std std-envvar docutils literal">PATH</code>.</li>

1871

<li>The [X refs, Y blocks] output of a debug (<code class="docutils literal">--with-pydebug</code>) build of the

1872

CPython interpreter is now off by default. It can be re-enabled using the

1873

<code class="docutils literal">-X showrefcount</code> option. (Contributed by Ezio Melotti in <a class="reference external" href="https://bugs.python.org/issue17323">issue 17323</a>.)</li>

1874

<li>The python command and most stdlib scripts (as well as <a class="reference internal" href="../library/argparse.html#module-argparse" title="argparse: Command-line option and argument parsing library."><code class="xref py py-mod docutils literal">argparse</code></a>) now

1875

output <code class="docutils literal">--version</code> information to <code class="docutils literal">stdout</code> instead of <code class="docutils literal">stderr</code> (for

1876

issue list see <a class="reference internal" href="#other-improvements-3-4">Other Improvements</a> above).</li>

1877

</ul>

1878

</div>

1879

1880

<h3>Changes in the Python API<a class="headerlink" href="#changes-in-the-python-api" title="Permalink to this headline">¶</a></h3>

1881

1882

<li>The ABCs defined in <a class="reference internal" href="../library/importlib.html#module-importlib.abc" title="importlib.abc: Abstract base classes related to import"><code class="xref py py-mod docutils literal">importlib.abc</code></a> now either raise the appropriate

1883

exception or return a default value instead of raising

1884

<a class="reference internal" href="../library/exceptions.html#NotImplementedError" title="NotImplementedError"><code class="xref py py-exc docutils literal">NotImplementedError</code></a> blindly. This will only affect code calling

1885

<a class="reference internal" href="../library/functions.html#super" title="super"><code class="xref py py-func docutils literal">super()</code></a> and falling through all the way to the ABCs. For compatibility,

1886

catch both <a class="reference internal" href="../library/exceptions.html#NotImplementedError" title="NotImplementedError"><code class="xref py py-exc docutils literal">NotImplementedError</code></a> or the appropriate exception as needed.</li>

1887

<li>The module type now initializes the <a class="reference internal" href="../reference/import.html#__package__" title="__package__"><code class="xref py py-attr docutils literal">__package__</code></a> and <a class="reference internal" href="../reference/import.html#__loader__" title="__loader__"><code class="xref py py-attr docutils literal">__loader__</code></a>

1888

attributes to <code class="docutils literal">None</code> by default. To determine if these attributes were set

1889

in a backwards-compatible fashion, use e.g.

1890

<code class="docutils literal">getattr(module, '__loader__', None) is not None</code>. (<a class="reference external" href="https://bugs.python.org/issue17115">issue 17115</a>.)</li>

1891

<li><a class="reference internal" href="../library/importlib.html#importlib.util.module_for_loader" title="importlib.util.module_for_loader"><code class="xref py py-meth docutils literal">importlib.util.module_for_loader()</code></a> now sets <code class="docutils literal">__loader__</code> and

1892

<code class="docutils literal">__package__</code> unconditionally to properly support reloading. If this is not

1893

desired then you will need to set these attributes manually. You can use

1894

<code class="xref py py-func docutils literal">importlib.util.module_to_load()</code> for module management.</li>

1895

<li>Import now resets relevant attributes (e.g. <code class="docutils literal">__name__</code>, <code class="docutils literal">__loader__</code>,

1896

<code class="docutils literal">__package__</code>, <code class="docutils literal">__file__</code>, <code class="docutils literal">__cached__</code>) unconditionally when reloading.

1897

Note that this restores a pre-3.3 behavior in that it means a module is

1898

re-found when re-loaded (<a class="reference external" href="https://bugs.python.org/issue19413">issue 19413</a>).</li>

1899

<li>Frozen packages no longer set <code class="docutils literal">__path__</code> to a list containing the package

1900

name, they now set it to an empty list. The previous behavior could cause

1901

the import system to do the wrong thing on submodule imports if there was

1902

also a directory with the same name as the frozen package. The correct way

1903

to determine if a module is a package or not is to use <code class="docutils literal">hasattr(module,

1904

'__path__')</code> (<a class="reference external" href="https://bugs.python.org/issue18065">issue 18065</a>).</li>

1905

<li>Frozen modules no longer define a <code class="docutils literal">__file__</code> attribute. It’s semantically

1906

incorrect for frozen modules to set the attribute as they are not loaded from

1907

any explicit location. If you must know that a module comes from frozen code

1908

then you can see if the module’s <code class="docutils literal">__spec__.location</code> is set to <code class="docutils literal">'frozen'</code>,

1909

check if the loader is a subclass of

1910

<a class="reference internal" href="../library/importlib.html#importlib.machinery.FrozenImporter" title="importlib.machinery.FrozenImporter"><code class="xref py py-class docutils literal">importlib.machinery.FrozenImporter</code></a>,

1911

or if Python 2 compatibility is necessary you can use <code class="xref py py-func docutils literal">imp.is_frozen()</code>.</li>

1912

<li><a class="reference internal" href="../library/py_compile.html#py_compile.compile" title="py_compile.compile"><code class="xref py py-func docutils literal">py_compile.compile()</code></a> now raises <a class="reference internal" href="../library/exceptions.html#FileExistsError" title="FileExistsError"><code class="xref py py-exc docutils literal">FileExistsError</code></a> if the file path

1913

it would write to is a symlink or a non-regular file. This is to act as a

1914

warning that import will overwrite those files with a regular file regardless

1915

of what type of file path they were originally.</li>

1916

<li><a class="reference internal" href="../library/importlib.html#importlib.abc.SourceLoader.get_source" title="importlib.abc.SourceLoader.get_source"><code class="xref py py-meth docutils literal">importlib.abc.SourceLoader.get_source()</code></a> no longer raises

1917

<a class="reference internal" href="../library/exceptions.html#ImportError" title="ImportError"><code class="xref py py-exc docutils literal">ImportError</code></a> when the source code being loaded triggers a

1918

<a class="reference internal" href="../library/exceptions.html#SyntaxError" title="SyntaxError"><code class="xref py py-exc docutils literal">SyntaxError</code></a> or <a class="reference internal" href="../library/exceptions.html#UnicodeDecodeError" title="UnicodeDecodeError"><code class="xref py py-exc docutils literal">UnicodeDecodeError</code></a>. As <a class="reference internal" href="../library/exceptions.html#ImportError" title="ImportError"><code class="xref py py-exc docutils literal">ImportError</code></a> is

1919

meant to be raised only when source code cannot be found but it should, it was

1920

felt to be over-reaching/overloading of that meaning when the source code is

1921

found but improperly structured. If you were catching ImportError before and

1922

wish to continue to ignore syntax or decoding issues, catch all three

1923

exceptions now.</li>

1924

<li><a class="reference internal" href="../library/functools.html#functools.update_wrapper" title="functools.update_wrapper"><code class="xref py py-func docutils literal">functools.update_wrapper()</code></a> and <a class="reference internal" href="../library/functools.html#functools.wraps" title="functools.wraps"><code class="xref py py-func docutils literal">functools.wraps()</code></a> now correctly

1925

set the <code class="docutils literal">__wrapped__</code> attribute to the function being wrapped, even if

1926

that function also had its <code class="docutils literal">__wrapped__</code> attribute set. This means

1927

<code class="docutils literal">__wrapped__</code> attributes now correctly link a stack of decorated

1928

functions rather than every <code class="docutils literal">__wrapped__</code> attribute in the chain

1929

referring to the innermost function. Introspection libraries that

1930

assumed the previous behaviour was intentional can use

1931

<a class="reference internal" href="../library/inspect.html#inspect.unwrap" title="inspect.unwrap"><code class="xref py py-func docutils literal">inspect.unwrap()</code></a> to access the first function in the chain that has

1932

no <code class="docutils literal">__wrapped__</code> attribute.</li>

1933

<li><a class="reference internal" href="../library/inspect.html#inspect.getfullargspec" title="inspect.getfullargspec"><code class="xref py py-func docutils literal">inspect.getfullargspec()</code></a> has been reimplemented on top of

1934

<a class="reference internal" href="../library/inspect.html#inspect.signature" title="inspect.signature"><code class="xref py py-func docutils literal">inspect.signature()</code></a> and hence handles a much wider variety of callable

1935

objects than it did in the past. It is expected that additional builtin and

1936

extension module callables will gain signature metadata over the course of

1937

the Python 3.4 series. Code that assumes that

1938

<a class="reference internal" href="../library/inspect.html#inspect.getfullargspec" title="inspect.getfullargspec"><code class="xref py py-func docutils literal">inspect.getfullargspec()</code></a> will fail on non-Python callables may need

1939

to be adjusted accordingly.</li>

1940

<li><a class="reference internal" href="../library/importlib.html#importlib.machinery.PathFinder" title="importlib.machinery.PathFinder"><code class="xref py py-class docutils literal">importlib.machinery.PathFinder</code></a> now passes on the current working

1941

directory to objects in <a class="reference internal" href="../library/sys.html#sys.path_hooks" title="sys.path_hooks"><code class="xref py py-data docutils literal">sys.path_hooks</code></a> for the empty string. This

1942

results in <a class="reference internal" href="../library/sys.html#sys.path_importer_cache" title="sys.path_importer_cache"><code class="xref py py-data docutils literal">sys.path_importer_cache</code></a> never containing <code class="docutils literal">''</code>, thus

1943

iterating through <a class="reference internal" href="../library/sys.html#sys.path_importer_cache" title="sys.path_importer_cache"><code class="xref py py-data docutils literal">sys.path_importer_cache</code></a> based on <a class="reference internal" href="../library/sys.html#sys.path" title="sys.path"><code class="xref py py-data docutils literal">sys.path</code></a>

1944

will not find all keys. A module’s <code class="docutils literal">__file__</code> when imported in the current

1945

working directory will also now have an absolute path, including when using

1946

<code class="docutils literal">-m</code> with the interpreter (except for <code class="docutils literal">__main__.__file__</code> when a script

1947

has been executed directly using a relative path) (Contributed by Brett

1948

Cannon in <a class="reference external" href="https://bugs.python.org/issue18416">issue 18416</a>). is specified on the command-line)

1949

(<a class="reference external" href="https://bugs.python.org/issue18416">issue 18416</a>).</li>

1950

<li>The removal of the strict argument to <a class="reference internal" href="../library/http.client.html#http.client.HTTPConnection" title="http.client.HTTPConnection"><code class="xref py py-class docutils literal">HTTPConnection</code></a>

1951

and <a class="reference internal" href="../library/http.client.html#http.client.HTTPSConnection" title="http.client.HTTPSConnection"><code class="xref py py-class docutils literal">HTTPSConnection</code></a> changes the meaning of the

1952

remaining arguments if you are specifying them positionally rather than by

1953

keyword. If you’ve been paying attention to deprecation warnings your code

1954

should already be specifying any additional arguments via keywords.</li>

1955

<li>Strings between <code class="docutils literal">from __future__ import ...</code> statements now always raise

1956

a <a class="reference internal" href="../library/exceptions.html#SyntaxError" title="SyntaxError"><code class="xref py py-exc docutils literal">SyntaxError</code></a>. Previously if there was no leading docstring, an

1957

interstitial string would sometimes be ignored. This brings CPython into

1958

compliance with the language spec; Jython and PyPy already were.

1959

(<a class="reference external" href="https://bugs.python.org/issue17434">issue 17434</a>).</li>

1960

<li><a class="reference internal" href="../library/ssl.html#ssl.SSLSocket.getpeercert" title="ssl.SSLSocket.getpeercert"><code class="xref py py-meth docutils literal">ssl.SSLSocket.getpeercert()</code></a> and <a class="reference internal" href="../library/ssl.html#ssl.SSLSocket.do_handshake" title="ssl.SSLSocket.do_handshake"><code class="xref py py-meth docutils literal">ssl.SSLSocket.do_handshake()</code></a>

1961

now raise an <a class="reference internal" href="../library/exceptions.html#OSError" title="OSError"><code class="xref py py-exc docutils literal">OSError</code></a> with <code class="docutils literal">ENOTCONN</code> when the <code class="docutils literal">SSLSocket</code> is not

1962

connected, instead of the previous behavior of raising an

1963

<a class="reference internal" href="../library/exceptions.html#AttributeError" title="AttributeError"><code class="xref py py-exc docutils literal">AttributeError</code></a>. In addition, <a class="reference internal" href="../library/ssl.html#ssl.SSLSocket.getpeercert" title="ssl.SSLSocket.getpeercert"><code class="xref py py-meth docutils literal">getpeercert()</code></a>

1964

will raise a <a class="reference internal" href="../library/exceptions.html#ValueError" title="ValueError"><code class="xref py py-exc docutils literal">ValueError</code></a> if the handshake has not yet been done.</li>

1965

<li><a class="reference internal" href="../library/base64.html#base64.b32decode" title="base64.b32decode"><code class="xref py py-func docutils literal">base64.b32decode()</code></a> now raises a <a class="reference internal" href="../library/binascii.html#binascii.Error" title="binascii.Error"><code class="xref py py-exc docutils literal">binascii.Error</code></a> when the

1966

input string contains non-b32-alphabet characters, instead of a

1967

<a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a>. This particular <a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a> was missed when the other

1968

1969

<a class="reference external" href="https://bugs.python.org/issue18011">issue 18011</a>.) Note: this change was also inadvertently applied in Python

1970

3.3.3.</li>

1971

<li>The <code class="xref py py-attr docutils literal">file</code> attribute is now automatically closed when

1972

the creating <code class="xref py py-class docutils literal">cgi.FieldStorage</code> instance is garbage collected. If you

1973

were pulling the file object out separately from the <code class="xref py py-class docutils literal">cgi.FieldStorage</code>

1974

instance and not keeping the instance alive, then you should either store the

1975

entire <code class="xref py py-class docutils literal">cgi.FieldStorage</code> instance or read the contents of the file

1976

before the <code class="xref py py-class docutils literal">cgi.FieldStorage</code> instance is garbage collected.</li>

1977

<li>Calling <code class="docutils literal">read</code> or <code class="docutils literal">write</code> on a closed SSL socket now raises an

1978

informative <a class="reference internal" href="../library/exceptions.html#ValueError" title="ValueError"><code class="xref py py-exc docutils literal">ValueError</code></a> rather than the previous more mysterious

1979

1980

<li><a class="reference internal" href="../reference/datamodel.html#slice.indices" title="slice.indices"><code class="xref py py-meth docutils literal">slice.indices()</code></a> no longer produces an <a class="reference internal" href="../library/exceptions.html#OverflowError" title="OverflowError"><code class="xref py py-exc docutils literal">OverflowError</code></a> for huge

1981

values. As a consequence of this fix, <a class="reference internal" href="../reference/datamodel.html#slice.indices" title="slice.indices"><code class="xref py py-meth docutils literal">slice.indices()</code></a> now raises a

1982

<a class="reference internal" href="../library/exceptions.html#ValueError" title="ValueError"><code class="xref py py-exc docutils literal">ValueError</code></a> if given a negative length; previously it returned nonsense

1983

values (<a class="reference external" href="https://bugs.python.org/issue14794">issue 14794</a>).</li>

1984

<li>The <a class="reference internal" href="../library/functions.html#complex" title="complex"><code class="xref py py-class docutils literal">complex</code></a> constructor, unlike the <a class="reference internal" href="../library/cmath.html#module-cmath" title="cmath: Mathematical functions for complex numbers."><code class="xref py py-mod docutils literal">cmath</code></a> functions, was

1985

incorrectly accepting <a class="reference internal" href="../library/functions.html#float" title="float"><code class="xref py py-class docutils literal">float</code></a> values if an object’s <code class="docutils literal">__complex__</code>

1986

special method returned one. This now raises a <a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a>.

1987

(<a class="reference external" href="https://bugs.python.org/issue16290">issue 16290</a>.)</li>

1988

<li>The <a class="reference internal" href="../library/functions.html#int" title="int"><code class="xref py py-class docutils literal">int</code></a> constructor in 3.2 and 3.3 erroneously accepts <a class="reference internal" href="../library/functions.html#float" title="float"><code class="xref py py-class docutils literal">float</code></a>

1989

values for the base parameter. It is unlikely anyone was doing this, but

1990

if so, it will now raise a <a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a> (<a class="reference external" href="https://bugs.python.org/issue16772">issue 16772</a>).</li>

1991

<li>Defaults for keyword-only arguments are now evaluated after defaults for

1992

regular keyword arguments, instead of before. Hopefully no one wrote any

1993

code that depends on the previous buggy behavior (<a class="reference external" href="https://bugs.python.org/issue16967">issue 16967</a>).</li>

1994

<li>Stale thread states are now cleared after <a class="reference internal" href="../library/os.html#os.fork" title="os.fork"><code class="xref py py-func docutils literal">fork()</code></a>. This may cause

1995

some system resources to be released that previously were incorrectly kept

1996

perpetually alive (for example, database connections kept in thread-local

1997

storage). (<a class="reference external" href="https://bugs.python.org/issue17094">issue 17094</a>.)</li>

1998

<li>Parameter names in <code class="docutils literal">__annotations__</code> dicts are now mangled properly,

1999

similarly to <code class="docutils literal">__kwdefaults__</code>. (Contributed by Yury Selivanov in

2000

<a class="reference external" href="https://bugs.python.org/issue20625">issue 20625</a>.)</li>

2001

<li><a class="reference internal" href="../library/hashlib.html#hashlib.hash.name" title="hashlib.hash.name"><code class="xref py py-attr docutils literal">hashlib.hash.name</code></a> now always returns the identifier in lower case.

2002

Previously some builtin hashes had uppercase names, but now that it is a

2003

formal public interface the naming has been made consistent (<a class="reference external" href="https://bugs.python.org/issue18532">issue 18532</a>).</li>

2004

<li>Because <a class="reference internal" href="../library/unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-mod docutils literal">unittest.TestSuite</code></a> now drops references to tests after they

2005

are run, test harnesses that re-use a <a class="reference internal" href="../library/unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-class docutils literal">TestSuite</code></a> to re-run

2006

a set of tests may fail. Test suites should not be re-used in this fashion

2007

since it means state is retained between test runs, breaking the test

2008

isolation that <a class="reference internal" href="../library/unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> is designed to provide. However, if the lack

2009

of isolation is considered acceptable, the old behavior can be restored by

2010

creating a <a class="reference internal" href="../library/unittest.html#unittest.TestSuite" title="unittest.TestSuite"><code class="xref py py-mod docutils literal">TestSuite</code></a> subclass that defines a

2011

<code class="docutils literal">_removeTestAtIndex</code> method that does nothing (see

2012

<a class="reference internal" href="../library/unittest.html#unittest.TestSuite.__iter__" title="unittest.TestSuite.__iter__"><code class="xref py py-meth docutils literal">TestSuite.__iter__()</code></a>) (<a class="reference external" href="https://bugs.python.org/issue11798">issue 11798</a>).</li>

2013

<li><a class="reference internal" href="../library/unittest.html#module-unittest" title="unittest: Unit testing framework for Python."><code class="xref py py-mod docutils literal">unittest</code></a> now uses <a class="reference internal" href="../library/argparse.html#module-argparse" title="argparse: Command-line option and argument parsing library."><code class="xref py py-mod docutils literal">argparse</code></a> for command line parsing. There are

2014

certain invalid command forms that used to work that are no longer allowed;

2015

in theory this should not cause backward compatibility issues since the

2016

disallowed command forms didn’t make any sense and are unlikely to be in use.</li>

2017

<li>The <a class="reference internal" href="../library/re.html#re.split" title="re.split"><code class="xref py py-func docutils literal">re.split()</code></a>, <a class="reference internal" href="../library/re.html#re.findall" title="re.findall"><code class="xref py py-func docutils literal">re.findall()</code></a>, and <a class="reference internal" href="../library/re.html#re.sub" title="re.sub"><code class="xref py py-func docutils literal">re.sub()</code></a> functions, and

2018

the <a class="reference internal" href="../library/re.html#re.match.group" title="re.match.group"><code class="xref py py-meth docutils literal">group()</code></a> and <a class="reference internal" href="../library/re.html#re.match.groups" title="re.match.groups"><code class="xref py py-meth docutils literal">groups()</code></a> methods of

2019

<code class="docutils literal">match</code> objects now always return a bytes object when the string

2020

to be matched is a <a class="reference internal" href="../glossary.html#term-bytes-like-object">bytes-like object</a>. Previously the return type

2021

matched the input type, so if your code was depending on the return value

2022

being, say, a <code class="docutils literal">bytearray</code>, you will need to change your code.</li>

2023

<li><a class="reference internal" href="../library/audioop.html#module-audioop" title="audioop: Manipulate raw audio data."><code class="xref py py-mod docutils literal">audioop</code></a> functions now raise an error immediately if passed string

2024

input, instead of failing randomly later on (<a class="reference external" href="https://bugs.python.org/issue16685">issue 16685</a>).</li>

2025

<li>The new convert_charrefs argument to <a class="reference internal" href="../library/html.parser.html#html.parser.HTMLParser" title="html.parser.HTMLParser"><code class="xref py py-class docutils literal">HTMLParser</code></a>

2026

currently defaults to <code class="docutils literal">False</code> for backward compatibility, but will

2027

eventually be changed to default to <code class="docutils literal">True</code>. It is recommended that you add

2028

this keyword, with the appropriate value, to any

2029

<a class="reference internal" href="../library/html.parser.html#html.parser.HTMLParser" title="html.parser.HTMLParser"><code class="xref py py-class docutils literal">HTMLParser</code></a> calls in your code (<a class="reference external" href="https://bugs.python.org/issue13633">issue 13633</a>).</li>

2030

<li>Since the digestmod argument to the <a class="reference internal" href="../library/hmac.html#hmac.new" title="hmac.new"><code class="xref py py-func docutils literal">hmac.new()</code></a> function will in the

2031

future have no default, all calls to <a class="reference internal" href="../library/hmac.html#hmac.new" title="hmac.new"><code class="xref py py-func docutils literal">hmac.new()</code></a> should be changed to

2032

explicitly specify a digestmod (<a class="reference external" href="https://bugs.python.org/issue17276">issue 17276</a>).</li>

2033

<li>Calling <a class="reference internal" href="../library/sysconfig.html#sysconfig.get_config_var" title="sysconfig.get_config_var"><code class="xref py py-func docutils literal">sysconfig.get_config_var()</code></a> with the <code class="docutils literal">SO</code> key, or looking

2034

<code class="docutils literal">SO</code> up in the results of a call to <a class="reference internal" href="../library/sysconfig.html#sysconfig.get_config_vars" title="sysconfig.get_config_vars"><code class="xref py py-func docutils literal">sysconfig.get_config_vars()</code></a>

2035

is deprecated. This key should be replaced by <code class="docutils literal">EXT_SUFFIX</code> or

2036

<code class="docutils literal">SHLIB_SUFFIX</code>, depending on the context (<a class="reference external" href="https://bugs.python.org/issue19555">issue 19555</a>).</li>

2037

<li>Any calls to <code class="docutils literal">open</code> functions that specify <code class="docutils literal">U</code> should be modified.

2038

<code class="docutils literal">U</code> is ineffective in Python3 and will eventually raise an error if used.

2039

Depending on the function, the equivalent of its old Python2 behavior can be

2040

achieved using either a newline argument, or if necessary by wrapping the

2041

stream in <a class="reference internal" href="../library/io.html#io.TextIOWrapper" title="io.TextIOWrapper"><code class="xref py py-mod docutils literal">TextIOWrapper</code></a> to use its newline argument

2042

(<a class="reference external" href="https://bugs.python.org/issue15204">issue 15204</a>).</li>

2043

<li>If you use <a class="reference internal" href="../using/scripts.html#scripts-pyvenv">pyvenv</a> in a script and desire that pip

2044

not be installed, you must add <code class="docutils literal">--without-pip</code> to your command

2045

invocation.</li>

2046

<li>The default behavior of <a class="reference internal" href="../library/json.html#json.dump" title="json.dump"><code class="xref py py-func docutils literal">json.dump()</code></a> and <a class="reference internal" href="../library/json.html#json.dumps" title="json.dumps"><code class="xref py py-func docutils literal">json.dumps()</code></a> when

2047

an indent is specified has changed: it no longer produces trailing

2048

spaces after the item separating commas at the ends of lines. This

2049

will matter only if you have tests that are doing white-space-sensitive

2050

comparisons of such output (<a class="reference external" href="https://bugs.python.org/issue16333">issue 16333</a>).</li>

2051

<li><a class="reference internal" href="../library/doctest.html#module-doctest" title="doctest: Test pieces of code within docstrings."><code class="xref py py-mod docutils literal">doctest</code></a> now looks for doctests in extension module <code class="docutils literal">__doc__</code>

2052

strings, so if your doctest test discovery includes extension modules that

2053

have things that look like doctests in them you may see test failures you’ve

2054

never seen before when running your tests (<a class="reference external" href="https://bugs.python.org/issue3158">issue 3158</a>).</li>

2055

<li>The <a class="reference internal" href="../library/collections.abc.html#module-collections.abc" title="collections.abc: Abstract base classes for containers"><code class="xref py py-mod docutils literal">collections.abc</code></a> module has been slightly refactored as

2056

part of the Python startup improvements. As a consequence of this, it is no

2057

longer the case that importing <a class="reference internal" href="../library/collections.html#module-collections" title="collections: Container datatypes"><code class="xref py py-mod docutils literal">collections</code></a> automatically imports

2058

<a class="reference internal" href="../library/collections.abc.html#module-collections.abc" title="collections.abc: Abstract base classes for containers"><code class="xref py py-mod docutils literal">collections.abc</code></a>. If your program depended on the (undocumented)

2059

implicit import, you will need to add an explicit <code class="docutils literal">import collections.abc</code>

2060

(<a class="reference external" href="https://bugs.python.org/issue20784">issue 20784</a>).</li>

2061

</ul>

2062

</div>

2063

2064

<h3>Changes in the C API<a class="headerlink" href="#changes-in-the-c-api" title="Permalink to this headline">¶</a></h3>

2065

2066

<li><a class="reference internal" href="../c-api/veryhigh.html#c.PyEval_EvalFrameEx" title="PyEval_EvalFrameEx"><code class="xref c c-func docutils literal">PyEval_EvalFrameEx()</code></a>, <a class="reference internal" href="../c-api/object.html#c.PyObject_Repr" title="PyObject_Repr"><code class="xref c c-func docutils literal">PyObject_Repr()</code></a>, and

2067

<a class="reference internal" href="../c-api/object.html#c.PyObject_Str" title="PyObject_Str"><code class="xref c c-func docutils literal">PyObject_Str()</code></a>, along with some other internal C APIs, now include

2068

a debugging assertion that ensures they are not used in situations where

2069

they may silently discard a currently active exception. In cases where

2070

discarding the active exception is expected and desired (for example,

2071

because it has already been saved locally with <a class="reference internal" href="../c-api/exceptions.html#c.PyErr_Fetch" title="PyErr_Fetch"><code class="xref c c-func docutils literal">PyErr_Fetch()</code></a> or

2072

is being deliberately replaced with a different exception), an explicit

2073

<a class="reference internal" href="../c-api/exceptions.html#c.PyErr_Clear" title="PyErr_Clear"><code class="xref c c-func docutils literal">PyErr_Clear()</code></a> call will be needed to avoid triggering the

2074

assertion when invoking these operations (directly or indirectly) and

2075

running against a version of Python that is compiled with assertions

2076

enabled.</li>

2077

<li><a class="reference internal" href="../c-api/exceptions.html#c.PyErr_SetImportError" title="PyErr_SetImportError"><code class="xref c c-func docutils literal">PyErr_SetImportError()</code></a> now sets <a class="reference internal" href="../library/exceptions.html#TypeError" title="TypeError"><code class="xref py py-exc docutils literal">TypeError</code></a> when its msg

2078

argument is not set. Previously only <code class="docutils literal">NULL</code> was returned with no exception

2079

set.</li>

2080

<li>The result of the <a class="reference internal" href="../c-api/veryhigh.html#c.PyOS_ReadlineFunctionPointer" title="PyOS_ReadlineFunctionPointer"><code class="xref c c-data docutils literal">PyOS_ReadlineFunctionPointer</code></a> callback must

2081

now be a string allocated by <a class="reference internal" href="../c-api/memory.html#c.PyMem_RawMalloc" title="PyMem_RawMalloc"><code class="xref c c-func docutils literal">PyMem_RawMalloc()</code></a> or

2082

<a class="reference internal" href="../c-api/memory.html#c.PyMem_RawRealloc" title="PyMem_RawRealloc"><code class="xref c c-func docutils literal">PyMem_RawRealloc()</code></a>, or NULL if an error occurred, instead of a

2083

string allocated by <a class="reference internal" href="../c-api/memory.html#c.PyMem_Malloc" title="PyMem_Malloc"><code class="xref c c-func docutils literal">PyMem_Malloc()</code></a> or <a class="reference internal" href="../c-api/memory.html#c.PyMem_Realloc" title="PyMem_Realloc"><code class="xref c c-func docutils literal">PyMem_Realloc()</code></a>

2084

(<a class="reference external" href="https://bugs.python.org/issue16742">issue 16742</a>)</li>

2085

<li><code class="xref c c-func docutils literal">PyThread_set_key_value()</code> now always set the value. In Python

2086

3.3, the function did nothing if the key already exists (if the current

2087

value is a non-NULL pointer).</li>

2088

<li>The <code class="docutils literal">f_tstate</code> (thread state) field of the <a class="reference internal" href="../c-api/veryhigh.html#c.PyFrameObject" title="PyFrameObject"><code class="xref c c-type docutils literal">PyFrameObject</code></a>

2089

structure has been removed to fix a bug: see <a class="reference external" href="https://bugs.python.org/issue14432">issue 14432</a> for the

2090

rationale.</li>

2091

</ul>

2092

</div>

2093

</div>

2094

2095

<h2>Changed in 3.4.3<a class="headerlink" href="#changed-in-3-4-3" title="Permalink to this headline">¶</a></h2>

2096

2097

<h3>PEP 476: Enabling certificate verification by default for stdlib http clients<a class="headerlink" href="#pep-476-enabling-certificate-verification-by-default-for-stdlib-http-clients" title="Permalink to this headline">¶</a></h3>

2098

<a class="reference internal" href="../library/http.client.html#module-http.client" title="http.client: HTTP and HTTPS protocol client (requires sockets)."><code class="xref py py-mod docutils literal">http.client</code></a> and modules which use it, such as <a class="reference internal" href="../library/urllib.request.html#module-urllib.request" title="urllib.request: Extensible library for opening URLs."><code class="xref py py-mod docutils literal">urllib.request</code></a> and

2099

<a class="reference internal" href="../library/xmlrpc.client.html#module-xmlrpc.client" title="xmlrpc.client: XML-RPC client access."><code class="xref py py-mod docutils literal">xmlrpc.client</code></a>, will now verify that the server presents a certificate

2100

which is signed by a CA in the platform trust store and whose hostname matches

2101

the hostname being requested by default, significantly improving security for

2102

many applications.

2103

For applications which require the old previous behavior, they can pass an

2104

alternate context:

2105

<div class="highlight-python3"><div class="highlight"><pre>import urllib.request

2106

import ssl

2107

2108

# This disables all verification

2109

context = ssl._create_unverified_context()

2110

2111

# This allows using a specific certificate for the host, which doesn't need

2112

# to be in the trust store

2113

context = ssl.create_default_context(cafile="/path/to/file.crt")

2114

2115

urllib.request.urlopen("https://invalid-cert", context=context)

2116

</pre></div>

2117

</div>

2118

</div>

2119

</div>

2120

</div>

2121

2122

2123

</div>

2124

</div>

2125

</div>

2126

2127

2128

<h3><a href="../contents.html">Table Of Contents</a></h3>

2129

<ul>

2130

<li><a class="reference internal" href="#">What’s New In Python 3.4</a><ul>

2131

<li><a class="reference internal" href="#summary-release-highlights">Summary – Release Highlights</a></li>

2132

<li><a class="reference internal" href="#new-features">New Features</a><ul>

2133

<li><a class="reference internal" href="#pep-453-explicit-bootstrapping-of-pip-in-python-installations">PEP 453: Explicit Bootstrapping of PIP in Python Installations</a><ul>

2134

<li><a class="reference internal" href="#bootstrapping-pip-by-default">Bootstrapping pip By Default</a></li>

2135

<li><a class="reference internal" href="#documentation-changes">Documentation Changes</a></li>

2136

</ul>

2137

</li>

2138

<li><a class="reference internal" href="#pep-446-newly-created-file-descriptors-are-non-inheritable">PEP 446: Newly Created File Descriptors Are Non-Inheritable</a></li>

2139

<li><a class="reference internal" href="#improvements-to-codec-handling">Improvements to Codec Handling</a></li>

2140

<li><a class="reference internal" href="#pep-451-a-modulespec-type-for-the-import-system">PEP 451: A ModuleSpec Type for the Import System</a></li>

2141

<li><a class="reference internal" href="#other-language-changes">Other Language Changes</a></li>

2142

</ul>

2143

</li>

2144

<li><a class="reference internal" href="#new-modules">New Modules</a><ul>

2145

<li><a class="reference internal" href="#asyncio">asyncio</a></li>

2146

<li><a class="reference internal" href="#ensurepip">ensurepip</a></li>

2147

2148

<li><a class="reference internal" href="#pathlib">pathlib</a></li>

2149

<li><a class="reference internal" href="#selectors">selectors</a></li>

2150

<li><a class="reference internal" href="#statistics">statistics</a></li>

2151

<li><a class="reference internal" href="#tracemalloc">tracemalloc</a></li>

2152

</ul>

2153

</li>

2154

<li><a class="reference internal" href="#improved-modules">Improved Modules</a><ul>

2155

2156

2157

<li><a class="reference internal" href="#argparse">argparse</a></li>

2158

<li><a class="reference internal" href="#audioop">audioop</a></li>

2159

2160

<li><a class="reference internal" href="#collections">collections</a></li>

2161

<li><a class="reference internal" href="#colorsys">colorsys</a></li>

2162

<li><a class="reference internal" href="#contextlib">contextlib</a></li>

2163

2164

2165

<li><a class="reference internal" href="#doctest">doctest</a></li>

2166

<li><a class="reference internal" href="#email">email</a></li>

2167

<li><a class="reference internal" href="#filecmp">filecmp</a></li>

2168

<li><a class="reference internal" href="#functools">functools</a></li>

2169

2170

2171

<li><a class="reference internal" href="#hashlib">hashlib</a></li>

2172

2173

2174

2175

<li><a class="reference internal" href="#idlelib-and-idle">idlelib and IDLE</a></li>

2176

<li><a class="reference internal" href="#importlib">importlib</a></li>

2177

<li><a class="reference internal" href="#inspect">inspect</a></li>

2178

<li><a class="reference internal" href="#ipaddress">ipaddress</a></li>

2179

<li><a class="reference internal" href="#logging">logging</a></li>

2180

<li><a class="reference internal" href="#marshal">marshal</a></li>

2181

2182

<li><a class="reference internal" href="#multiprocessing">multiprocessing</a></li>

2183

<li><a class="reference internal" href="#operator">operator</a></li>

2184

2185

2186

<li><a class="reference internal" href="#pickle">pickle</a></li>

2187

<li><a class="reference internal" href="#plistlib">plistlib</a></li>

2188

<li><a class="reference internal" href="#poplib">poplib</a></li>

2189

<li><a class="reference internal" href="#pprint">pprint</a></li>

2190

2191

<li><a class="reference internal" href="#pydoc">pydoc</a></li>

2192

2193

<li><a class="reference internal" href="#resource">resource</a></li>

2194

<li><a class="reference internal" href="#select">select</a></li>

2195

<li><a class="reference internal" href="#shelve">shelve</a></li>

2196

<li><a class="reference internal" href="#shutil">shutil</a></li>

2197

<li><a class="reference internal" href="#smtpd">smtpd</a></li>

2198

<li><a class="reference internal" href="#smtplib">smtplib</a></li>

2199

<li><a class="reference internal" href="#socket">socket</a></li>

2200

<li><a class="reference internal" href="#sqlite3">sqlite3</a></li>

2201

2202

2203

<li><a class="reference internal" href="#struct">struct</a></li>

2204

<li><a class="reference internal" href="#subprocess">subprocess</a></li>

2205

<li><a class="reference internal" href="#sunau">sunau</a></li>

2206

2207

<li><a class="reference internal" href="#tarfile">tarfile</a></li>

2208

<li><a class="reference internal" href="#textwrap">textwrap</a></li>

2209

<li><a class="reference internal" href="#threading">threading</a></li>

2210

<li><a class="reference internal" href="#traceback">traceback</a></li>

2211

<li><a class="reference internal" href="#types">types</a></li>

2212

<li><a class="reference internal" href="#urllib">urllib</a></li>

2213

<li><a class="reference internal" href="#unittest">unittest</a></li>

2214

2215

2216

<li><a class="reference internal" href="#weakref">weakref</a></li>

2217

<li><a class="reference internal" href="#xml-etree">xml.etree</a></li>

2218

<li><a class="reference internal" href="#zipfile">zipfile</a></li>

2219

</ul>

2220

</li>

2221

<li><a class="reference internal" href="#cpython-implementation-changes">CPython Implementation Changes</a><ul>

2222

<li><a class="reference internal" href="#pep-445-customization-of-cpython-memory-allocators">PEP 445: Customization of CPython Memory Allocators</a></li>

2223

<li><a class="reference internal" href="#pep-442-safe-object-finalization">PEP 442: Safe Object Finalization</a></li>

2224

<li><a class="reference internal" href="#pep-456-secure-and-interchangeable-hash-algorithm">PEP 456: Secure and Interchangeable Hash Algorithm</a></li>

2225

<li><a class="reference internal" href="#pep-436-argument-clinic">PEP 436: Argument Clinic</a></li>

2226

<li><a class="reference internal" href="#other-build-and-c-api-changes">Other Build and C API Changes</a></li>

2227

<li><a class="reference internal" href="#other-improvements">Other Improvements</a></li>

2228

<li><a class="reference internal" href="#significant-optimizations">Significant Optimizations</a></li>

2229

</ul>

2230

</li>

2231

<li><a class="reference internal" href="#deprecated">Deprecated</a><ul>

2232

<li><a class="reference internal" href="#deprecations-in-the-python-api">Deprecations in the Python API</a></li>

2233

<li><a class="reference internal" href="#deprecated-features">Deprecated Features</a></li>

2234

</ul>

2235

</li>

2236

<li><a class="reference internal" href="#removed">Removed</a><ul>

2237

<li><a class="reference internal" href="#operating-systems-no-longer-supported">Operating Systems No Longer Supported</a></li>

2238

<li><a class="reference internal" href="#api-and-feature-removals">API and Feature Removals</a></li>

2239

<li><a class="reference internal" href="#code-cleanups">Code Cleanups</a></li>

2240

</ul>

2241

</li>

2242

<li><a class="reference internal" href="#porting-to-python-3-4">Porting to Python 3.4</a><ul>

2243

<li><a class="reference internal" href="#changes-in-python-command-behavior">Changes in ‘python’ Command Behavior</a></li>

2244

<li><a class="reference internal" href="#changes-in-the-python-api">Changes in the Python API</a></li>

2245

<li><a class="reference internal" href="#changes-in-the-c-api">Changes in the C API</a></li>

2246

</ul>

2247

</li>

2248

<li><a class="reference internal" href="#changed-in-3-4-3">Changed in 3.4.3</a><ul>

2249

<li><a class="reference internal" href="#pep-476-enabling-certificate-verification-by-default-for-stdlib-http-clients">PEP 476: Enabling certificate verification by default for stdlib http clients</a></li>

2250

</ul>

2251

</li>

2252

</ul>

2253

</li>

2254

</ul>

2255

2256

<h4>Previous topic</h4>

2257

<a href="3.5.html"

2258

title="previous chapter">What’s New In Python 3.5</a>

2259

<h4>Next topic</h4>

2260

<a href="3.3.html"

2261

title="next chapter">What’s New In Python 3.3</a>

2262

2263

2264

2265

<li><a href="../bugs.html">Report a Bug</a></li>

2266

<li><a href="../_sources/whatsnew/3.4.txt"

2267

rel="nofollow">Show Source</a></li>

2268

</ul>

2269

</div>

2270

</div>

2271

</div>

2272

2273

</div>

2274

2275

<h3>Navigation</h3>

2276

<ul>

2277

2278

<a href="../genindex.html" title="General Index"

2279

>index</a></li>

2280

2281

<a href="../py-modindex.html" title="Python Module Index"

2282

>modules</a> |</li>

2283

2284

<a href="3.3.html" title="What’s New In Python 3.3"

2285

>next</a> |</li>

2286

2287

<a href="3.5.html" title="What’s New In Python 3.5"

2288

>previous</a> |</li>

2289

<li><img src="../_static/py.png" alt=""

2290

style="vertical-align: middle; margin-top: -1px"/></li>

2291

<li><a href="https://www.python.org/">Python</a> »</li>

2292

<li>

2293

3.5.2

2294

<a href="../index.html">Documentation </a> »

2295

</li>

2296

2297

<li class="nav-item nav-item-1"><a href="index.html" >What’s New in Python</a> »</li>

2298

2299

2300

2301

2302

2303

2304

2305

2306

2307

</form>

2308

</div>

2309

2310

2311

</li>

2312

2313

</ul>

2314

</div>

2315

2316

2317

2318

The Python Software Foundation is a non-profit corporation.

2319

<a href="https://www.python.org/psf/donations/">Please donate.</a>

2320

2321

Last updated on Oct 19, 2016.

2322

<a href="../bugs.html">Found a bug</a>?

2323

2324

Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.3.3.

2325

</div>

2326

2327

</body>

2328

</html>

b'\\ No newline at end of file'

Older »