~ubuntu-branches/ubuntu/quantal/python3.2/quantal-proposed : revision 34

2

3

# hg diff -r v3.2.3rc1 | filterdiff --exclude=.hgignore --exclude=.hgeol --exclude=.hgtags --remove-timestamps

4

5

diff -r 7085403daf43 .hgtags

6

--- a/.hgtags

7

+++ b/.hgtags

8

@@ -94,3 +94,4 @@

9

ac1f7e5c05104d557d5acd922e95625ba5d1fe10 v3.2.1

10

c860feaa348d663e598986894ee4680480577e15 v3.2.2rc1

11

137e45f15c0bd262c9ad4c032d97425bc0589456 v3.2.2

12

+7085403daf439adb3f9e70ef13f6bedb1c447376 v3.2.3rc1

13

diff -r 7085403daf43 Doc/howto/advocacy.rst

14

--- a/Doc/howto/advocacy.rst

15

+++ b/Doc/howto/advocacy.rst

16

@@ -264,8 +264,7 @@

17

18

**What are the restrictions on Python's use?**

19

20

-They're practically nonexistent. Consult the :file:`Misc/COPYRIGHT` file in the

21

-source distribution, or the section :ref:`history-and-license` for the full

22

+They're practically nonexistent. Consult :ref:`history-and-license` for the full

23

language, but it boils down to three conditions:

24

25

* You have to leave the copyright notice on the software; if you don't include

26

diff -r 7085403daf43 Doc/howto/cporting.rst

27

--- a/Doc/howto/cporting.rst

28

+++ b/Doc/howto/cporting.rst

29

@@ -2,27 +2,28 @@

30

31

.. _cporting-howto:

32

33

-********************************

34

-Porting Extension Modules to 3.0

35

-********************************

36

+*************************************

37

+Porting Extension Modules to Python 3

38

+*************************************

39

40

:author: Benjamin Peterson

41

42

43

.. topic:: Abstract

44

45

- Although changing the C-API was not one of Python 3.0's objectives, the many

46

- Python level changes made leaving 2.x's API intact impossible. In fact, some

47

- changes such as :func:`int` and :func:`long` unification are more obvious on

48

- the C level. This document endeavors to document incompatibilities and how

49

- they can be worked around.

50

+ Although changing the C-API was not one of Python 3's objectives,

51

+ the many Python-level changes made leaving Python 2's API intact

52

+ impossible. In fact, some changes such as :func:`int` and

53

+ :func:`long` unification are more obvious on the C level. This

54

+ document endeavors to document incompatibilities and how they can

55

+ be worked around.

56

57

58

Conditional compilation

59

=======================

60

61

-The easiest way to compile only some code for 3.0 is to check if

62

-:c:macro:`PY_MAJOR_VERSION` is greater than or equal to 3. ::

63

+The easiest way to compile only some code for Python 3 is to check

64

+if :c:macro:`PY_MAJOR_VERSION` is greater than or equal to 3. ::

65

66

#if PY_MAJOR_VERSION >= 3

67

#define IS_PY3K

68

@@ -35,7 +36,7 @@

69

Changes to Object APIs

70

======================

71

72

-Python 3.0 merged together some types with similar functions while cleanly

73

+Python 3 merged together some types with similar functions while cleanly

74

separating others.

75

76

77

@@ -43,14 +44,14 @@

78

-----------------------

79

80

81

-Python 3.0's :func:`str` (``PyString_*`` functions in C) type is equivalent to

82

-2.x's :func:`unicode` (``PyUnicode_*``). The old 8-bit string type has become

83

-:func:`bytes`. Python 2.6 and later provide a compatibility header,

84

+Python 3's :func:`str` (``PyString_*`` functions in C) type is equivalent to

85

+Python 2's :func:`unicode` (``PyUnicode_*``). The old 8-bit string type has

86

+become :func:`bytes`. Python 2.6 and later provide a compatibility header,

87

:file:`bytesobject.h`, mapping ``PyBytes`` names to ``PyString`` ones. For best

88

-compatibility with 3.0, :c:type:`PyUnicode` should be used for textual data and

89

+compatibility with Python 3, :c:type:`PyUnicode` should be used for textual data and

90

:c:type:`PyBytes` for binary data. It's also important to remember that

91

-:c:type:`PyBytes` and :c:type:`PyUnicode` in 3.0 are not interchangeable like

92

-:c:type:`PyString` and :c:type:`PyUnicode` are in 2.x. The following example

93

+:c:type:`PyBytes` and :c:type:`PyUnicode` in Python 3 are not interchangeable like

94

+:c:type:`PyString` and :c:type:`PyUnicode` are in Python 2. The following example

95

shows best practices with regards to :c:type:`PyUnicode`, :c:type:`PyString`,

96

and :c:type:`PyBytes`. ::

97

98

@@ -94,10 +95,12 @@

99

long/int Unification

100

--------------------

101

102

-In Python 3.0, there is only one integer type. It is called :func:`int` on the

103

-Python level, but actually corresponds to 2.x's :func:`long` type. In the

104

-C-API, ``PyInt_*`` functions are replaced by their ``PyLong_*`` neighbors. The

105

-best course of action here is using the ``PyInt_*`` functions aliased to

106

+Python 3 has only one integer type, :func:`int`. But it actually

107

+corresponds to Python 2's :func:`long` type--the :func:`int` type

108

+used in Python 2 was removed. In the C-API, ``PyInt_*`` functions

109

+are replaced by their ``PyLong_*`` equivalents.

110

+

111

+The best course of action here is using the ``PyInt_*`` functions aliased to

112

``PyLong_*`` found in :file:`intobject.h`. The abstract ``PyNumber_*`` APIs

113

can also be used in some cases. ::

114

115

@@ -120,10 +123,11 @@

116

Module initialization and state

117

===============================

118

119

-Python 3.0 has a revamped extension module initialization system. (See

120

-:pep:`3121`.) Instead of storing module state in globals, they should be stored

121

-in an interpreter specific structure. Creating modules that act correctly in

122

-both 2.x and 3.0 is tricky. The following simple example demonstrates how. ::

123

+Python 3 has a revamped extension module initialization system. (See

124

+:pep:`3121`.) Instead of storing module state in globals, they should

125

+be stored in an interpreter specific structure. Creating modules that

126

+act correctly in both Python 2 and Python 3 is tricky. The following

127

+simple example demonstrates how. ::

128

129

#include "Python.h"

130

131

@@ -209,10 +213,65 @@

132

}

133

134

135

+CObject replaced with Capsule

136

+=============================

137

+

138

+The :c:type:`Capsule` object was introduced in Python 3.1 and 2.7 to replace

139

+:c:type:`CObject`. CObjects were useful,

140

+but the :c:type:`CObject` API was problematic: it didn't permit distinguishing

141

+between valid CObjects, which allowed mismatched CObjects to crash the

142

+interpreter, and some of its APIs relied on undefined behavior in C.

143

+(For further reading on the rationale behind Capsules, please see :issue:`5630`.)

144

+

145

+If you're currently using CObjects, and you want to migrate to 3.1 or newer,

146

+you'll need to switch to Capsules.

147

+:c:type:`CObject` was deprecated in 3.1 and 2.7 and completely removed in

148

+Python 3.2. If you only support 2.7, or 3.1 and above, you

149

+can simply switch to :c:type:`Capsule`. If you need to support Python 3.0,

150

+or versions of Python earlier than 2.7,

151

+you'll have to support both CObjects and Capsules.

152

+(Note that Python 3.0 is no longer supported, and it is not recommended

153

+for production use.)

154

+

155

+The following example header file :file:`capsulethunk.h` may

156

+solve the problem for you. Simply write your code against the

157

+:c:type:`Capsule` API and include this header file after

158

+:file:`Python.h`. Your code will automatically use Capsules

159

+in versions of Python with Capsules, and switch to CObjects

160

+when Capsules are unavailable.

161

+

162

+:file:`capsulethunk.h` simulates Capsules using CObjects. However,

163

+:c:type:`CObject` provides no place to store the capsule's "name". As a

164

+result the simulated :c:type:`Capsule` objects created by :file:`capsulethunk.h`

165

+behave slightly differently from real Capsules. Specifically:

166

+

167

+ * The name parameter passed in to :c:func:`PyCapsule_New` is ignored.

168

+

169

+ * The name parameter passed in to :c:func:`PyCapsule_IsValid` and

170

+ :c:func:`PyCapsule_GetPointer` is ignored, and no error checking

171

+ of the name is performed.

172

+

173

+ * :c:func:`PyCapsule_GetName` always returns NULL.

174

+

175

+ * :c:func:`PyCapsule_SetName` always throws an exception and

176

+ returns failure. (Since there's no way to store a name

177

+ in a CObject, noisy failure of :c:func:`PyCapsule_SetName`

178

+ was deemed preferable to silent failure here. If this is

179

+ inconveient, feel free to modify your local

180

+ copy as you see fit.)

181

+

182

+You can find :file:`capsulethunk.h` in the Python source distribution

183

+as :source:`Doc/includes/capsulethunk.h`. We also include it here for

184

+your convenience:

185

+

186

+.. literalinclude:: ../includes/capsulethunk.h

187

+

188

+

189

+

190

Other options

191

=============

192

193

If you are writing a new extension module, you might consider `Cython

194

<http://www.cython.org>`_. It translates a Python-like language to C. The

195

-extension modules it creates are compatible with Python 3.x and 2.x.

196

+extension modules it creates are compatible with Python 3 and Python 2.

197

198

diff -r 7085403daf43 Doc/howto/logging-cookbook.rst

199

--- a/Doc/howto/logging-cookbook.rst

200

+++ b/Doc/howto/logging-cookbook.rst

201

@@ -268,12 +268,12 @@

202

.. currentmodule:: logging.handlers

203

204

Sometimes you have to get your logging handlers to do their work without

205

-blocking the thread you’re logging from. This is common in Web applications,

206

+blocking the thread you're logging from. This is common in Web applications,

207

though of course it also occurs in other scenarios.

208

209

A common culprit which demonstrates sluggish behaviour is the

210

:class:`SMTPHandler`: sending emails can take a long time, for a

211

-number of reasons outside the developer’s control (for example, a poorly

212

+number of reasons outside the developer's control (for example, a poorly

213

performing mail or network infrastructure). But almost any network-based

214

handler can block: Even a :class:`SocketHandler` operation may do a

215

DNS query under the hood which is too slow (and this query can be deep in the

216

@@ -292,7 +292,7 @@

217

218

The second part of the solution is :class:`QueueListener`, which has been

219

designed as the counterpart to :class:`QueueHandler`. A

220

-:class:`QueueListener` is very simple: it’s passed a queue and some handlers,

221

+:class:`QueueListener` is very simple: it's passed a queue and some handlers,

222

and it fires up an internal thread which listens to its queue for LogRecords

223

sent from ``QueueHandlers`` (or any other source of ``LogRecords``, for that

224

matter). The ``LogRecords`` are removed from the queue and passed to the

225

@@ -745,7 +745,7 @@

226

raise

227

except:

228

import sys, traceback

229

- print >> sys.stderr, 'Whoops! Problem:'

230

+ print('Whoops! Problem:', file=sys.stderr)

231

traceback.print_exc(file=sys.stderr)

232

233

# Arrays used for random selections in this demo

234

@@ -964,6 +964,219 @@

235

Obviously this example sets the log length much too small as an extreme

236

example. You would want to set *maxBytes* to an appropriate value.

237

238

+.. _format-styles:

239

+

240

+Use of alternative formatting styles

241

+------------------------------------

242

+

243

+When logging was added to the Python standard library, the only way of

244

+formatting messages with variable content was to use the %-formatting

245

+method. Since then, Python has gained two new formatting approaches:

246

+:class:`string.Template` (added in Python 2.4) and :meth:`str.format`

247

+(added in Python 2.6).

248

+

249

+Logging (as of 3.2) provides improved support for these two additional

250

+formatting styles. The :class:`Formatter` class been enhanced to take an

251

+additional, optional keyword parameter named ``style``. This defaults to

252

+``'%'``, but other possible values are ``'{'`` and ``'$'``, which correspond

253

+to the other two formatting styles. Backwards compatibility is maintained by

254

+default (as you would expect), but by explicitly specifying a style parameter,

255

+you get the ability to specify format strings which work with

256

+:meth:`str.format` or :class:`string.Template`. Here's an example console

257

+session to show the possibilities:

258

+

259

+.. code-block:: pycon

260

+

261

+ >>> import logging

262

+ >>> root = logging.getLogger()

263

+ >>> root.setLevel(logging.DEBUG)

264

+ >>> handler = logging.StreamHandler()

265

+ >>> bf = logging.Formatter('{asctime} {name} {levelname:8s} {message}',

266

+ ... style='{')

267

+ >>> handler.setFormatter(bf)

268

+ >>> root.addHandler(handler)

269

+ >>> logger = logging.getLogger('foo.bar')

270

+ >>> logger.debug('This is a DEBUG message')

271

+ 2010-10-28 15:11:55,341 foo.bar DEBUG This is a DEBUG message

272

+ >>> logger.critical('This is a CRITICAL message')

273

+ 2010-10-28 15:12:11,526 foo.bar CRITICAL This is a CRITICAL message

274

+ >>> df = logging.Formatter('$asctime $name ${levelname} $message',

275

+ ... style='$')

276

+ >>> handler.setFormatter(df)

277

+ >>> logger.debug('This is a DEBUG message')

278

+ 2010-10-28 15:13:06,924 foo.bar DEBUG This is a DEBUG message

279

+ >>> logger.critical('This is a CRITICAL message')

280

+ 2010-10-28 15:13:11,494 foo.bar CRITICAL This is a CRITICAL message

281

+ >>>

282

+

283

+Note that the formatting of logging messages for final output to logs is

284

+completely independent of how an individual logging message is constructed.

285

+That can still use %-formatting, as shown here::

286

+

287

+ >>> logger.error('This is an%s %s %s', 'other,', 'ERROR,', 'message')

288

+ 2010-10-28 15:19:29,833 foo.bar ERROR This is another, ERROR, message

289

+ >>>

290

+

291

+Logging calls (``logger.debug()``, ``logger.info()`` etc.) only take

292

+positional parameters for the actual logging message itself, with keyword

293

+parameters used only for determining options for how to handle the actual

294

+logging call (e.g. the ``exc_info`` keyword parameter to indicate that

295

+traceback information should be logged, or the ``extra`` keyword parameter

296

+to indicate additional contextual information to be added to the log). So

297

+you cannot directly make logging calls using :meth:`str.format` or

298

+:class:`string.Template` syntax, because internally the logging package

299

+uses %-formatting to merge the format string and the variable arguments.

300

+There would no changing this while preserving backward compatibility, since

301

+all logging calls which are out there in existing code will be using %-format

302

+strings.

303

+

304

+There is, however, a way that you can use {}- and $- formatting to construct

305

+your individual log messages. Recall that for a message you can use an

306

+arbitrary object as a message format string, and that the logging package will

307

+call ``str()`` on that object to get the actual format string. Consider the

308

+following two classes::

309

+

310

+ class BraceMessage(object):

311

+ def __init__(self, fmt, *args, **kwargs):

312

+ self.fmt = fmt

313

+ self.args = args

314

+ self.kwargs = kwargs

315

+

316

+ def __str__(self):

317

+ return self.fmt.format(*self.args, **self.kwargs)

318

+

319

+ class DollarMessage(object):

320

+ def __init__(self, fmt, **kwargs):

321

+ self.fmt = fmt

322

+ self.kwargs = kwargs

323

+

324

+ def __str__(self):

325

+ from string import Template

326

+ return Template(self.fmt).substitute(**self.kwargs)

327

+

328

+Either of these can be used in place of a format string, to allow {}- or

329

+$-formatting to be used to build the actual "message" part which appears in the

330

+formatted log output in place of "%(message)s" or "{message}" or "$message".

331

+It's a little unwieldy to use the class names whenever you want to log

332

+something, but it's quite palatable if you use an alias such as __ (double

333

+underscore – not to be confused with _, the single underscore used as a

334

+synonym/alias for :func:`gettext.gettext` or its brethren).

335

+

336

+The above classes are not included in Python, though they're easy enough to

337

+copy and paste into your own code. They can be used as follows (assuming that

338

+they're declared in a module called ``wherever``):

339

+

340

+.. code-block:: pycon

341

+

342

+ >>> from wherever import BraceMessage as __

343

+ >>> print(__('Message with {0} {name}', 2, name='placeholders'))

344

+ Message with 2 placeholders

345

+ >>> class Point: pass

346

+ ...

347

+ >>> p = Point()

348

+ >>> p.x = 0.5

349

+ >>> p.y = 0.5

350

+ >>> print(__('Message with coordinates: ({point.x:.2f}, {point.y:.2f})',

351

+ ... point=p))

352

+ Message with coordinates: (0.50, 0.50)

353

+ >>> from wherever import DollarMessage as __

354

+ >>> print(__('Message with $num $what', num=2, what='placeholders'))

355

+ Message with 2 placeholders

356

+ >>>

357

+

358

+While the above examples use ``print()`` to show how the formatting works, you

359

+would of course use ``logger.debug()`` or similar to actually log using this

360

+approach.

361

+

362

+One thing to note is that you pay no significant performance penalty with this

363

+approach: the actual formatting happens not when you make the logging call, but

364

+when (and if) the logged message is actually about to be output to a log by a

365

+handler. So the only slightly unusual thing which might trip you up is that the

366

+parentheses go around the format string and the arguments, not just the format

367

+string. That's because the __ notation is just syntax sugar for a constructor

368

+call to one of the XXXMessage classes.

369

+

370

+

371

+.. currentmodule:: logging

372

+

373

+.. _custom-logrecord:

374

+

375

+Customising ``LogRecord``

376

+-------------------------

377

+

378

+Every logging event is represented by a :class:`LogRecord` instance.

379

+When an event is logged and not filtered out by a logger's level, a

380

+:class:`LogRecord` is created, populated with information about the event and

381

+then passed to the handlers for that logger (and its ancestors, up to and

382

+including the logger where further propagation up the hierarchy is disabled).

383

+Before Python 3.2, there were only two places where this creation was done:

384

+

385

+* :meth:`Logger.makeRecord`, which is called in the normal process of

386

+ logging an event. This invoked :class:`LogRecord` directly to create an

387

+ instance.

388

+* :func:`makeLogRecord`, which is called with a dictionary containing

389

+ attributes to be added to the LogRecord. This is typically invoked when a

390

+ suitable dictionary has been received over the network (e.g. in pickle form

391

+ via a :class:`~handlers.SocketHandler`, or in JSON form via an

392

+ :class:`~handlers.HTTPHandler`).

393

+

394

+This has usually meant that if you need to do anything special with a

395

+:class:`LogRecord`, you've had to do one of the following.

396

+

397

+* Create your own :class:`Logger` subclass, which overrides

398

+ :meth:`Logger.makeRecord`, and set it using :func:`~logging.setLoggerClass`

399

+ before any loggers that you care about are instantiated.

400

+* Add a :class:`Filter` to a logger or handler, which does the

401

+ necessary special manipulation you need when its

402

+ :meth:`~Filter.filter` method is called.

403

+

404

+The first approach would be a little unwieldy in the scenario where (say)

405

+several different libraries wanted to do different things. Each would attempt

406

+to set its own :class:`Logger` subclass, and the one which did this last would

407

+win.

408

+

409

+The second approach works reasonably well for many cases, but does not allow

410

+you to e.g. use a specialized subclass of :class:`LogRecord`. Library

411

+developers can set a suitable filter on their loggers, but they would have to

412

+remember to do this every time they introduced a new logger (which they would

413

+do simply by adding new packages or modules and doing ::

414

+

415

+ logger = logging.getLogger(__name__)

416

+

417

+at module level). It's probably one too many things to think about. Developers

418

+could also add the filter to a :class:`~logging.NullHandler` attached to their

419

+top-level logger, but this would not be invoked if an application developer

420

+attached a handler to a lower-level library logger – so output from that

421

+handler would not reflect the intentions of the library developer.

422

+

423

+In Python 3.2 and later, :class:`~logging.LogRecord` creation is done through a

424

+factory, which you can specify. The factory is just a callable you can set with

425

+:func:`~logging.setLogRecordFactory`, and interrogate with

426

+:func:`~logging.getLogRecordFactory`. The factory is invoked with the same

427

+signature as the :class:`~logging.LogRecord` constructor, as :class:`LogRecord`

428

+is the default setting for the factory.

429

+

430

+This approach allows a custom factory to control all aspects of LogRecord

431

+creation. For example, you could return a subclass, or just add some additional

432

+attributes to the record once created, using a pattern similar to this::

433

+

434

+ old_factory = logging.getLogRecordFactory()

435

+

436

+ def record_factory(*args, **kwargs):

437

+ record = old_factory(*args, **kwargs)

438

+ record.custom_attribute = 0xdecafbad

439

+ return record

440

+

441

+ logging.setLogRecordFactory(record_factory)

442

+

443

+This pattern allows different libraries to chain factories together, and as

444

+long as they don't overwrite each other's attributes or unintentionally

445

+overwrite the attributes provided as standard, there should be no surprises.

446

+However, it should be borne in mind that each link in the chain adds run-time

447

+overhead to all logging operations, and the technique should only be used when

448

+the use of a :class:`Filter` does not provide the desired result.

449

+

450

+

451

.. _zeromq-handlers:

452

453

Subclassing QueueHandler - a ZeroMQ example

454

diff -r 7085403daf43 Doc/howto/regex.rst

455

--- a/Doc/howto/regex.rst

456

+++ b/Doc/howto/regex.rst

457

@@ -360,7 +360,7 @@

458

459

You can learn about this by interactively experimenting with the :mod:`re`

460

module. If you have :mod:`tkinter` available, you may also want to look at

461

-:file:`Tools/demo/redemo.py`, a demonstration program included with the

462

+:source:`Tools/demo/redemo.py`, a demonstration program included with the

463

Python distribution. It allows you to enter REs and strings, and displays

464

whether the RE matches or fails. :file:`redemo.py` can be quite useful when

465

trying to debug a complicated RE. Phil Schwartz's `Kodos

466

@@ -495,7 +495,7 @@

467

the same ones in several locations, then it might be worthwhile to collect all

468

the definitions in one place, in a section of code that compiles all the REs

469

ahead of time. To take an example from the standard library, here's an extract

470

-from the now deprecated :file:`xmllib.py`::

471

+from the now-defunct Python 2 standard :mod:`xmllib` module::

472

473

ref = re.compile( ... )

474

entityref = re.compile( ... )

475

diff -r 7085403daf43 Doc/includes/capsulethunk.h

476

--- /dev/null

477

+++ b/Doc/includes/capsulethunk.h

478

@@ -0,0 +1,134 @@

479

+#ifndef __CAPSULETHUNK_H

480

+#define __CAPSULETHUNK_H

481

+

482

+#if ( (PY_VERSION_HEX < 0x02070000) \

483

+ || ((PY_VERSION_HEX >= 0x03000000) \

484

+ && (PY_VERSION_HEX < 0x03010000)) )

485

+

486

+#define __PyCapsule_GetField(capsule, field, default_value) \

487

+ ( PyCapsule_CheckExact(capsule) \

488

+ ? (((PyCObject *)capsule)->field) \

489

+ : (default_value) \

490

+ ) \

491

+

492

+#define __PyCapsule_SetField(capsule, field, value) \

493

+ ( PyCapsule_CheckExact(capsule) \

494

+ ? (((PyCObject *)capsule)->field = value), 1 \

495

+ : 0 \

496

+ ) \

497

+

498

+

499

+#define PyCapsule_Type PyCObject_Type

500

+

501

+#define PyCapsule_CheckExact(capsule) (PyCObject_Check(capsule))

502

+#define PyCapsule_IsValid(capsule, name) (PyCObject_Check(capsule))

503

+

504

+

505

+#define PyCapsule_New(pointer, name, destructor) \

506

+ (PyCObject_FromVoidPtr(pointer, destructor))

507

+

508

+

509

+#define PyCapsule_GetPointer(capsule, name) \

510

+ (PyCObject_AsVoidPtr(capsule))

511

+

512

+/* Don't call PyCObject_SetPointer here, it fails if there's a destructor */

513

+#define PyCapsule_SetPointer(capsule, pointer) \

514

+ __PyCapsule_SetField(capsule, cobject, pointer)

515

+

516

+

517

+#define PyCapsule_GetDestructor(capsule) \

518

+ __PyCapsule_GetField(capsule, destructor)

519

+

520

+#define PyCapsule_SetDestructor(capsule, dtor) \

521

+ __PyCapsule_SetField(capsule, destructor, dtor)

522

+

523

+

524

+/*

525

+ * Sorry, there's simply no place

526

+ * to store a Capsule "name" in a CObject.

527

+ */

528

+#define PyCapsule_GetName(capsule) NULL

529

+

530

+static int

531

+PyCapsule_SetName(PyObject *capsule, const char *unused)

532

+{

533

+ unused = unused;

534

+ PyErr_SetString(PyExc_NotImplementedError,

535

+ "can't use PyCapsule_SetName with CObjects");

536

+ return 1;

537

+}

538

+

539

+

540

+

541

+#define PyCapsule_GetContext(capsule) \

542

+ __PyCapsule_GetField(capsule, descr)

543

+

544

+#define PyCapsule_SetContext(capsule, context) \

545

+ __PyCapsule_SetField(capsule, descr, context)

546

+

547

+

548

+static void *

549

+PyCapsule_Import(const char *name, int no_block)

550

+{

551

+ PyObject *object = NULL;

552

+ void *return_value = NULL;

553

+ char *trace;

554

+ size_t name_length = (strlen(name) + 1) * sizeof(char);

555

+ char *name_dup = (char *)PyMem_MALLOC(name_length);

556

+

557

+ if (!name_dup) {

558

+ return NULL;

559

+ }

560

+

561

+ memcpy(name_dup, name, name_length);

562

+

563

+ trace = name_dup;

564

+ while (trace) {

565

+ char *dot = strchr(trace, '.');

566

+ if (dot) {

567

+ *dot++ = '\0';

568

+ }

569

+

570

+ if (object == NULL) {

571

+ if (no_block) {

572

+ object = PyImport_ImportModuleNoBlock(trace);

573

+ } else {

574

+ object = PyImport_ImportModule(trace);

575

+ if (!object) {

576

+ PyErr_Format(PyExc_ImportError,

577

+ "PyCapsule_Import could not "

578

+ "import module \"%s\"", trace);

579

+ }

580

+ }

581

+ } else {

582

+ PyObject *object2 = PyObject_GetAttrString(object, trace);

583

+ Py_DECREF(object);

584

+ object = object2;

585

+ }

586

+ if (!object) {

587

+ goto EXIT;

588

+ }

589

+

590

+ trace = dot;

591

+ }

592

+

593

+ if (PyCObject_Check(object)) {

594

+ PyCObject *cobject = (PyCObject *)object;

595

+ return_value = cobject->cobject;

596

+ } else {

597

+ PyErr_Format(PyExc_AttributeError,

598

+ "PyCapsule_Import \"%s\" is not valid",

599

+ name);

600

+ }

601

+

602

+EXIT:

603

+ Py_XDECREF(object);

604

+ if (name_dup) {

605

+ PyMem_FREE(name_dup);

606

+ }

607

+ return return_value;

608

+}

609

+

610

+#endif /* #if PY_VERSION_HEX < 0x02070000 */

611

+

612

+#endif /* __CAPSULETHUNK_H */

613

diff -r 7085403daf43 Doc/library/logging.rst

614

--- a/Doc/library/logging.rst

615

+++ b/Doc/library/logging.rst

616

@@ -159,7 +159,7 @@

617

618

FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s'

619

logging.basicConfig(format=FORMAT)

620

- d = { 'clientip' : '192.168.0.1', 'user' : 'fbloggs' }

621

+ d = {'clientip': '192.168.0.1', 'user': 'fbloggs'}

622

logger = logging.getLogger('tcpserver')

623

logger.warning('Protocol problem: %s', 'connection reset', extra=d)

624

625

@@ -1077,7 +1077,7 @@

626

If *capture* is ``True``, warnings issued by the :mod:`warnings` module will

627

be redirected to the logging system. Specifically, a warning will be

628

formatted using :func:`warnings.formatwarning` and the resulting string

629

- logged to a logger named ``'py.warnings'`` with a severity of ``'WARNING'``.

630

+ logged to a logger named ``'py.warnings'`` with a severity of :const:`WARNING`.

631

632

If *capture* is ``False``, the redirection of warnings to the logging system

633

will stop, and warnings will be redirected to their original destinations

634

diff -r 7085403daf43 Doc/library/markup.rst

635

--- a/Doc/library/markup.rst

636

+++ b/Doc/library/markup.rst

637

@@ -23,7 +23,7 @@

638

html.rst

639

html.parser.rst

640

html.entities.rst

641

- pyexpat.rst

642

+ xml.etree.elementtree.rst

643

xml.dom.rst

644

xml.dom.minidom.rst

645

xml.dom.pulldom.rst

646

@@ -31,4 +31,4 @@

647

xml.sax.handler.rst

648

xml.sax.utils.rst

649

xml.sax.reader.rst

650

- xml.etree.elementtree.rst

651

+ pyexpat.rst

652

diff -r 7085403daf43 Doc/library/operator.rst

653

--- a/Doc/library/operator.rst

654

+++ b/Doc/library/operator.rst

655

@@ -340,7 +340,7 @@

656

+-----------------------+-------------------------+---------------------------------------+

657

| Containment Test | ``obj in seq`` | ``contains(seq, obj)`` |

658

+-----------------------+-------------------------+---------------------------------------+

659

-| Division | ``a / b`` | ``div(a, b)`` |

660

+| Division | ``a / b`` | ``truediv(a, b)`` |

661

+-----------------------+-------------------------+---------------------------------------+

662

| Division | ``a // b`` | ``floordiv(a, b)`` |

663

+-----------------------+-------------------------+---------------------------------------+

664

diff -r 7085403daf43 Doc/library/re.rst

665

--- a/Doc/library/re.rst

666

+++ b/Doc/library/re.rst

667

@@ -330,16 +330,22 @@

668

Matches the empty string, but only at the beginning or end of a word.

669

A word is defined as a sequence of Unicode alphanumeric or underscore

670

characters, so the end of a word is indicated by whitespace or a

671

- non-alphanumeric, non-underscore Unicode character. Note that

672

- formally, ``\b`` is defined as the boundary between a ``\w`` and a

673

- ``\W`` character (or vice versa). By default Unicode alphanumerics

674

- are the ones used, but this can be changed by using the :const:`ASCII`

675

- flag. Inside a character range, ``\b`` represents the backspace

676

- character, for compatibility with Python's string literals.

677

+ non-alphanumeric, non-underscore Unicode character. Note that formally,

678

+ ``\b`` is defined as the boundary between a ``\w`` and a ``\W`` character

679

+ (or vice versa), or between ``\w`` and the beginning/end of the string.

680

+ This means that ``r'\bfoo\b'`` matches ``'foo'``, ``'foo.'``, ``'(foo)'``,

681

+ ``'bar foo baz'`` but not ``'foobar'`` or ``'foo3'``.

682

+

683

+ By default Unicode alphanumerics are the ones used, but this can be changed

684

+ by using the :const:`ASCII` flag. Inside a character range, ``\b``

685

+ represents the backspace character, for compatibility with Python's string

686

+ literals.

687

688

``\B``

689

- Matches the empty string, but only when it is *not* at the beginning or end of a

690

- word. This is just the opposite of ``\b``, so word characters are

691

+ Matches the empty string, but only when it is *not* at the beginning or end

692

+ of a word. This means that ``r'py\B'`` matches ``'python'``, ``'py3'``,

693

+ ``'py2'``, but not ``'py'``, ``'py.'``, or ``'py!'``.

694

+ ``\B`` is just the opposite of ``\b``, so word characters are

695

Unicode alphanumerics or the underscore, although this can be changed

696

by using the :const:`ASCII` flag.

697

698

@@ -417,31 +423,6 @@

699

three digits in length.

700

701

702

-.. _matching-searching:

703

-

704

-Matching vs. Searching

705

-----------------------

706

-

707

-.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>

708

-

709

-

710

-Python offers two different primitive operations based on regular expressions:

711

-**match** checks for a match only at the beginning of the string, while

712

-**search** checks for a match anywhere in the string (this is what Perl does

713

-by default).

714

-

715

-Note that match may differ from search even when using a regular expression

716

-beginning with ``'^'``: ``'^'`` matches only at the start of the string, or in

717

-:const:`MULTILINE` mode also immediately following a newline. The "match"

718

-operation succeeds only if the pattern matches at the start of the string

719

-regardless of mode, or at the starting position given by the optional *pos*

720

-argument regardless of whether a newline precedes it.

721

-

722

- >>> re.match("c", "abcdef") # No match

723

- >>> re.search("c", "abcdef") # Match

724

- <_sre.SRE_Match object at ...>

725

-

726

-

727

.. _contents-of-module-re:

728

729

Module Contents

730

@@ -575,10 +556,11 @@

731

<match-objects>`. Return ``None`` if the string does not match the pattern;

732

note that this is different from a zero-length match.

733

734

- .. note::

735

+ Note that even in :const:`MULTILINE` mode, :func:`re.match` will only match

736

+ at the beginning of the string and not at the beginning of each line.

737

738

- If you want to locate a match anywhere in *string*, use :func:`search`

739

- instead.

740

+ If you want to locate a match anywhere in *string*, use :func:`search`

741

+ instead (see also :ref:`search-vs-match`).

742

743

744

.. function:: split(pattern, string, maxsplit=0, flags=0)

745

@@ -762,16 +744,14 @@

746

The optional *pos* and *endpos* parameters have the same meaning as for the

747

:meth:`~regex.search` method.

748

749

- .. note::

750

-

751

- If you want to locate a match anywhere in *string*, use

752

- :meth:`~regex.search` instead.

753

-

754

>>> pattern = re.compile("o")

755

>>> pattern.match("dog") # No match as "o" is not at the start of "dog".

756

>>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog".

757

<_sre.SRE_Match object at ...>

758

759

+ If you want to locate a match anywhere in *string*, use

760

+ :meth:`~regex.search` instead (see also :ref:`search-vs-match`).

761

+

762

763

.. method:: regex.split(string, maxsplit=0)

764

765

@@ -1133,37 +1113,39 @@

766

[a-zA-Z0-9_ ]*?end``. As a further benefit, such regular expressions will run

767

faster than their recursive equivalents.

768

769

+.. _search-vs-match:

770

771

search() vs. match()

772

^^^^^^^^^^^^^^^^^^^^

773

774

-In a nutshell, :func:`match` only attempts to match a pattern at the beginning

775

-of a string where :func:`search` will match a pattern anywhere in a string.

776

-For example:

777

+.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>

778

779

- >>> re.match("o", "dog") # No match as "o" is not the first letter of "dog".

780

- >>> re.search("o", "dog") # Match as search() looks everywhere in the string.

781

+Python offers two different primitive operations based on regular expressions:

782

+:func:`re.match` checks for a match only at the beginning of the string, while

783

+:func:`re.search` checks for a match anywhere in the string (this is what Perl

784

+does by default).

785

+

786

+For example::

787

+

788

+ >>> re.match("c", "abcdef") # No match

789

+ >>> re.search("c", "abcdef") # Match

790

<_sre.SRE_Match object at ...>

791

792

-.. note::

793

+Regular expressions beginning with ``'^'`` can be used with :func:`search` to

794

+restrict the match at the beginning of the string::

795

796

- The following applies only to regular expression objects like those created

797

- with ``re.compile("pattern")``, not the primitives ``re.match(pattern,

798

- string)`` or ``re.search(pattern, string)``.

799

+ >>> re.match("c", "abcdef") # No match

800

+ >>> re.search("^c", "abcdef") # No match

801

+ >>> re.search("^a", "abcdef") # Match

802

+ <_sre.SRE_Match object at ...>

803

804

-:func:`match` has an optional second parameter that gives an index in the string

805

-where the search is to start::

806

+Note however that in :const:`MULTILINE` mode :func:`match` only matches at the

807

+beginning of the string, whereas using :func:`search` with a regular expression

808

+beginning with ``'^'`` will match at the beginning of each line.

809

810

- >>> pattern = re.compile("o")

811

- >>> pattern.match("dog") # No match as "o" is not at the start of "dog."

812

-

813

- # Equivalent to the above expression as 0 is the default starting index:

814

- >>> pattern.match("dog", 0)

815

-

816

- # Match as "o" is the 2nd character of "dog" (index 0 is the first):

817

- >>> pattern.match("dog", 1)

818

+ >>> re.match('X', 'A\nB\nX', re.MULTILINE) # No match

819

+ >>> re.search('^X', 'A\nB\nX', re.MULTILINE) # Match

820

<_sre.SRE_Match object at ...>

821

- >>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog."

822

823

824

Making a Phonebook

825

diff -r 7085403daf43 Doc/library/sqlite3.rst

826

--- a/Doc/library/sqlite3.rst

827

+++ b/Doc/library/sqlite3.rst

828

@@ -3,7 +3,7 @@

829

830

.. module:: sqlite3

831

:synopsis: A DB-API 2.0 implementation using SQLite 3.x.

832

-.. sectionauthor:: Gerhard HÃ¤ring <gh@ghaering.de>

833

+.. sectionauthor:: Gerhard Häring <gh@ghaering.de>

834

835

836

SQLite is a C library that provides a lightweight disk-based database that

837

@@ -20,6 +20,7 @@

838

represents the database. Here the data will be stored in the

839

:file:`/tmp/example` file::

840

841

+ import sqlite3

842

conn = sqlite3.connect('/tmp/example')

843

844

You can also supply the special name ``:memory:`` to create a database in RAM.

845

@@ -56,7 +57,7 @@

846

847

# Never do this -- insecure!

848

symbol = 'IBM'

849

- c.execute("... where symbol = '%s'" % symbol)

850

+ c.execute("select * from stocks where symbol = '%s'" % symbol)

851

852

# Do this instead

853

t = (symbol,)

854

@@ -64,7 +65,7 @@

855

856

# Larger example

857

for t in [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),

858

- ('2006-04-05', 'BUY', 'MSOFT', 1000, 72.00),

859

+ ('2006-04-05', 'BUY', 'MSFT', 1000, 72.00),

860

('2006-04-06', 'SELL', 'IBM', 500, 53.00),

861

]:

862

c.execute('insert into stocks values (?,?,?,?,?)', t)

863

@@ -271,7 +272,6 @@

864

calling the cursor method, then calls the cursor's :meth:`executemany

865

<Cursor.executemany>` method with the parameters given.

866

867

-

868

.. method:: Connection.executescript(sql_script)

869

870

This is a nonstandard shortcut that creates an intermediate cursor object by

871

@@ -376,22 +376,22 @@

872

aggregates or whole new virtual table implementations. One well-known

873

extension is the fulltext-search extension distributed with SQLite.

874

875

+ Loadable extensions are disabled by default. See [#f1]_.

876

+

877

.. versionadded:: 3.2

878

879

.. literalinclude:: ../includes/sqlite3/load_extension.py

880

881

- Loadable extensions are disabled by default. See [#f1]_.

882

-

883

.. method:: Connection.load_extension(path)

884

885

This routine loads a SQLite extension from a shared library. You have to

886

enable extension loading with :meth:`enable_load_extension` before you can

887

use this routine.

888

889

+ Loadable extensions are disabled by default. See [#f1]_.

890

+

891

.. versionadded:: 3.2

892

893

- Loadable extensions are disabled by default. See [#f1]_.

894

-

895

.. attribute:: Connection.row_factory

896

897

You can change this attribute to a callable that accepts the cursor and the

898

diff -r 7085403daf43 Doc/library/stdtypes.rst

899

--- a/Doc/library/stdtypes.rst

900

+++ b/Doc/library/stdtypes.rst

901

@@ -1437,8 +1437,13 @@

902

903

.. note::

904

905

- The formatting operations described here are obsolete and may go away in future

906

- versions of Python. Use the new :ref:`string-formatting` in new code.

907

+ The formatting operations described here are modelled on C's printf()

908

+ syntax. They only support formatting of certain builtin types. The

909

+ use of a binary operator means that care may be needed in order to

910

+ format tuples and dictionaries correctly. As the new

911

+ :ref:`string-formatting` syntax is more flexible and handles tuples and

912

+ dictionaries naturally, it is recommended for new code. However, there

913

+ are no current plans to deprecate printf-style formatting.

914

915

String objects have one unique built-in operation: the ``%`` operator (modulo).

916

This is also known as the string *formatting* or *interpolation* operator.

917

diff -r 7085403daf43 Doc/library/xml.dom.minidom.rst

918

--- a/Doc/library/xml.dom.minidom.rst

919

+++ b/Doc/library/xml.dom.minidom.rst

920

@@ -15,6 +15,14 @@

921

Model interface. It is intended to be simpler than the full DOM and also

922

significantly smaller.

923

924

+.. note::

925

+

926

+ The :mod:`xml.dom.minidom` module provides an implementation of the W3C-DOM,

927

+ with an API similar to that in other programming languages. Users who are

928

+ unfamiliar with the W3C-DOM interface or who would like to write less code

929

+ for processing XML files should consider using the

930

+ :mod:`xml.etree.ElementTree` module instead.

931

+

932

DOM applications typically start by parsing some XML into a DOM. With

933

:mod:`xml.dom.minidom`, this is done through the parse functions::

934

935

diff -r 7085403daf43 Doc/tools/sphinxext/layout.html

936

--- a/Doc/tools/sphinxext/layout.html

937

+++ b/Doc/tools/sphinxext/layout.html

938

@@ -6,7 +6,7 @@

939

{% endblock %}

940

{% block extrahead %}

941

942

- <script type="text/javascript" src="{{ pathto('_static/copybutton.js', 1) }}"></script>

943

+ {% if not embedded %}<script type="text/javascript" src="{{ pathto('_static/copybutton.js', 1) }}"></script>{% endif %}

944

945

{% endblock %}

946

{% block footer %}

947

diff -r 7085403daf43 Doc/tutorial/classes.rst

948

--- a/Doc/tutorial/classes.rst

949

+++ b/Doc/tutorial/classes.rst

950

@@ -180,7 +180,10 @@

951

scope_test()

952

print("In global scope:", spam)

953

954

-The output of the example code is::

955

+The output of the example code is:

956

+

957

+.. code-block:: none

958

+

959

960

After local assignment: test spam

961

After nonlocal assignment: nonlocal spam

962

diff -r 7085403daf43 Lib/_weakrefset.py

963

--- a/Lib/_weakrefset.py

964

+++ b/Lib/_weakrefset.py

965

@@ -63,7 +63,7 @@

966

yield item

967

968

def __len__(self):

969

- return sum(x() is not None for x in self.data)

970

+ return len(self.data) - len(self._pending_removals)

971

972

def __contains__(self, item):

973

try:

974

@@ -114,36 +114,21 @@

975

def update(self, other):

976

if self._pending_removals:

977

self._commit_removals()

978

- if isinstance(other, self.__class__):

979

- self.data.update(other.data)

980

- else:

981

- for element in other:

982

- self.add(element)

983

+ for element in other:

984

+ self.add(element)

985

986

def __ior__(self, other):

987

self.update(other)

988

return self

989

990

- # Helper functions for simple delegating methods.

991

- def _apply(self, other, method):

992

- if not isinstance(other, self.__class__):

993

- other = self.__class__(other)

994

- newdata = method(other.data)

995

- newset = self.__class__()

996

- newset.data = newdata

997

+ def difference(self, other):

998

+ newset = self.copy()

999

+ newset.difference_update(other)

1000

return newset

1001

-

1002

- def difference(self, other):

1003

- return self._apply(other, self.data.difference)

1004

__sub__ = difference

1005

1006

def difference_update(self, other):

1007

- if self._pending_removals:

1008

- self._commit_removals()

1009

- if self is other:

1010

- self.data.clear()

1011

- else:

1012

- self.data.difference_update(ref(item) for item in other)

1013

+ self.__isub__(other)

1014

def __isub__(self, other):

1015

if self._pending_removals:

1016

self._commit_removals()

1017

@@ -154,13 +139,11 @@

1018

return self

1019

1020

def intersection(self, other):

1021

- return self._apply(other, self.data.intersection)

1022

+ return self.__class__(item for item in other if item in self)

1023

__and__ = intersection

1024

1025

def intersection_update(self, other):

1026

- if self._pending_removals:

1027

- self._commit_removals()

1028

- self.data.intersection_update(ref(item) for item in other)

1029

+ self.__iand__(other)

1030

def __iand__(self, other):

1031

if self._pending_removals:

1032

self._commit_removals()

1033

@@ -169,17 +152,17 @@

1034

1035

def issubset(self, other):

1036

return self.data.issubset(ref(item) for item in other)

1037

- __lt__ = issubset

1038

+ __le__ = issubset

1039

1040

- def __le__(self, other):

1041

- return self.data <= set(ref(item) for item in other)

1042

+ def __lt__(self, other):

1043

+ return self.data < set(ref(item) for item in other)

1044

1045

def issuperset(self, other):

1046

return self.data.issuperset(ref(item) for item in other)

1047

- __gt__ = issuperset

1048

+ __ge__ = issuperset

1049

1050

- def __ge__(self, other):

1051

- return self.data >= set(ref(item) for item in other)

1052

+ def __gt__(self, other):

1053

+ return self.data > set(ref(item) for item in other)

1054

1055

def __eq__(self, other):

1056

if not isinstance(other, self.__class__):

1057

@@ -187,27 +170,24 @@

1058

return self.data == set(ref(item) for item in other)

1059

1060

def symmetric_difference(self, other):

1061

- return self._apply(other, self.data.symmetric_difference)

1062

+ newset = self.copy()

1063

+ newset.symmetric_difference_update(other)

1064

+ return newset

1065

__xor__ = symmetric_difference

1066

1067

def symmetric_difference_update(self, other):

1068

- if self._pending_removals:

1069

- self._commit_removals()

1070

- if self is other:

1071

- self.data.clear()

1072

- else:

1073

- self.data.symmetric_difference_update(ref(item) for item in other)

1074

+ self.__ixor__(other)

1075

def __ixor__(self, other):

1076

if self._pending_removals:

1077

self._commit_removals()

1078

if self is other:

1079

self.data.clear()

1080

else:

1081

- self.data.symmetric_difference_update(ref(item) for item in other)

1082

+ self.data.symmetric_difference_update(ref(item, self._remove) for item in other)

1083

return self

1084

1085

def union(self, other):

1086

- return self._apply(other, self.data.union)

1087

+ return self.__class__(e for s in (self, other) for e in s)

1088

__or__ = union

1089

1090

def isdisjoint(self, other):

1091

diff -r 7085403daf43 Lib/distutils/filelist.py

1092

--- a/Lib/distutils/filelist.py

1093

+++ b/Lib/distutils/filelist.py

1094

@@ -201,6 +201,7 @@

1095

1096

Return True if files are found, False otherwise.

1097

"""

1098

+ # XXX docstring lying about what the special chars are?

1099

files_found = False

1100

pattern_re = translate_pattern(pattern, anchor, prefix, is_regex)

1101

self.debug_print("include_pattern: applying regex r'%s'" %

1102

@@ -284,11 +285,14 @@

1103

# IMHO is wrong -- '?' and '*' aren't supposed to match slash in Unix,

1104

# and by extension they shouldn't match such "special characters" under

1105

# any OS. So change all non-escaped dots in the RE to match any

1106

- # character except the special characters.

1107

- # XXX currently the "special characters" are just slash -- i.e. this is

1108

- # Unix-only.

1109

- pattern_re = re.sub(r'((?<!\\)(\\\\)*)\.', r'\1[^/]', pattern_re)

1110

-

1111

+ # character except the special characters (currently: just os.sep).

1112

+ sep = os.sep

1113

+ if os.sep == '\\':

1114

+ # we're using a regex to manipulate a regex, so we need

1115

+ # to escape the backslash twice

1116

+ sep = r'\\\\'

1117

+ escaped = r'\1[^%s]' % sep

1118

+ pattern_re = re.sub(r'((?<!\\)(\\\\)*)\.', escaped, pattern_re)

1119

return pattern_re

1120

1121

1122

@@ -312,9 +316,11 @@

1123

if prefix is not None:

1124

# ditch end of pattern character

1125

empty_pattern = glob_to_re('')

1126

- prefix_re = (glob_to_re(prefix))[:-len(empty_pattern)]

1127

- # paths should always use / in manifest templates

1128

- pattern_re = "^%s/.*%s" % (prefix_re, pattern_re)

1129

+ prefix_re = glob_to_re(prefix)[:-len(empty_pattern)]

1130

+ sep = os.sep

1131

+ if os.sep == '\\':

1132

+ sep = r'\\'

1133

+ pattern_re = "^" + sep.join((prefix_re, ".*" + pattern_re))

1134

else: # no prefix -- respect anchor flag

1135

if anchor:

1136

pattern_re = "^" + pattern_re

1137

diff -r 7085403daf43 Lib/distutils/tests/test_bdist_msi.py

1138

--- a/Lib/distutils/tests/test_bdist_msi.py

1139

+++ b/Lib/distutils/tests/test_bdist_msi.py

1140

@@ -1,12 +1,11 @@

1141

"""Tests for distutils.command.bdist_msi."""

1142

+import sys

1143

import unittest

1144

-import sys

1145

-

1146

from test.support import run_unittest

1147

-

1148

from distutils.tests import support

1149

1150

-@unittest.skipUnless(sys.platform=="win32", "These tests are only for win32")

1151

+

1152

+@unittest.skipUnless(sys.platform == 'win32', 'these tests require Windows')

1153

class BDistMSITestCase(support.TempdirManager,

1154

support.LoggingSilencer,

1155

unittest.TestCase):

1156

@@ -14,10 +13,11 @@

1157

def test_minimal(self):

1158

# minimal test XXX need more tests

1159

from distutils.command.bdist_msi import bdist_msi

1160

- pkg_pth, dist = self.create_dist()

1161

+ project_dir, dist = self.create_dist()

1162

cmd = bdist_msi(dist)

1163

cmd.ensure_finalized()

1164

1165

+

1166

def test_suite():

1167

return unittest.makeSuite(BDistMSITestCase)

1168

1169

diff -r 7085403daf43 Lib/distutils/tests/test_filelist.py

1170

--- a/Lib/distutils/tests/test_filelist.py

1171

+++ b/Lib/distutils/tests/test_filelist.py

1172

@@ -1,4 +1,5 @@

1173

"""Tests for distutils.filelist."""

1174

+import os

1175

import re

1176

import unittest

1177

from distutils import debug

1178

@@ -9,6 +10,26 @@

1179

from test.support import captured_stdout, run_unittest

1180

from distutils.tests import support

1181

1182

+MANIFEST_IN = """\

1183

+include ok

1184

+include xo

1185

+exclude xo

1186

+include foo.tmp

1187

+include buildout.cfg

1188

+global-include *.x

1189

+global-include *.txt

1190

+global-exclude *.tmp

1191

+recursive-include f *.oo

1192

+recursive-exclude global *.x

1193

+graft dir

1194

+prune dir3

1195

+"""

1196

+

1197

+

1198

+def make_local_path(s):

1199

+ """Converts '/' in a string to os.sep"""

1200

+ return s.replace('/', os.sep)

1201

+

1202

1203

class FileListTestCase(support.LoggingSilencer,

1204

unittest.TestCase):

1205

@@ -22,16 +43,62 @@

1206

self.clear_logs()

1207

1208

def test_glob_to_re(self):

1209

- # simple cases

1210

- self.assertEqual(glob_to_re('foo*'), 'foo[^/]*\\Z(?ms)')

1211

- self.assertEqual(glob_to_re('foo?'), 'foo[^/]\\Z(?ms)')

1212

- self.assertEqual(glob_to_re('foo??'), 'foo[^/][^/]\\Z(?ms)')

1213

+ sep = os.sep

1214

+ if os.sep == '\\':

1215

+ sep = re.escape(os.sep)

1216

1217

- # special cases

1218

- self.assertEqual(glob_to_re(r'foo\\*'), r'foo\\\\[^/]*\Z(?ms)')

1219

- self.assertEqual(glob_to_re(r'foo\\\*'), r'foo\\\\\\[^/]*\Z(?ms)')

1220

- self.assertEqual(glob_to_re('foo????'), r'foo[^/][^/][^/][^/]\Z(?ms)')

1221

- self.assertEqual(glob_to_re(r'foo\\??'), r'foo\\\\[^/][^/]\Z(?ms)')

1222

+ for glob, regex in (

1223

+ # simple cases

1224

+ ('foo*', r'foo[^%(sep)s]*\Z(?ms)'),

1225

+ ('foo?', r'foo[^%(sep)s]\Z(?ms)'),

1226

+ ('foo??', r'foo[^%(sep)s][^%(sep)s]\Z(?ms)'),

1227

+ # special cases

1228

+ (r'foo\\*', r'foo\\\\[^%(sep)s]*\Z(?ms)'),

1229

+ (r'foo\\\*', r'foo\\\\\\[^%(sep)s]*\Z(?ms)'),

1230

+ ('foo????', r'foo[^%(sep)s][^%(sep)s][^%(sep)s][^%(sep)s]\Z(?ms)'),

1231

+ (r'foo\\??', r'foo\\\\[^%(sep)s][^%(sep)s]\Z(?ms)')):

1232

+ regex = regex % {'sep': sep}

1233

+ self.assertEqual(glob_to_re(glob), regex)

1234

+

1235

+ def test_process_template_line(self):

1236

+ # testing all MANIFEST.in template patterns

1237

+ file_list = FileList()

1238

+ l = make_local_path

1239

+

1240

+ # simulated file list

1241

+ file_list.allfiles = ['foo.tmp', 'ok', 'xo', 'four.txt',

1242

+ 'buildout.cfg',

1243

+ # filelist does not filter out VCS directories,

1244

+ # it's sdist that does

1245

+ l('.hg/last-message.txt'),

1246

+ l('global/one.txt'),

1247

+ l('global/two.txt'),

1248

+ l('global/files.x'),

1249

+ l('global/here.tmp'),

1250

+ l('f/o/f.oo'),

1251

+ l('dir/graft-one'),

1252

+ l('dir/dir2/graft2'),

1253

+ l('dir3/ok'),

1254

+ l('dir3/sub/ok.txt'),

1255

+ ]

1256

+

1257

+ for line in MANIFEST_IN.split('\n'):

1258

+ if line.strip() == '':

1259

+ continue

1260

+ file_list.process_template_line(line)

1261

+

1262

+ wanted = ['ok',

1263

+ 'buildout.cfg',

1264

+ 'four.txt',

1265

+ l('.hg/last-message.txt'),

1266

+ l('global/one.txt'),

1267

+ l('global/two.txt'),

1268

+ l('f/o/f.oo'),

1269

+ l('dir/graft-one'),

1270

+ l('dir/dir2/graft2'),

1271

+ ]

1272

+

1273

+ self.assertEqual(file_list.files, wanted)

1274

1275

def test_debug_print(self):

1276

file_list = FileList()

1277

@@ -117,6 +184,7 @@

1278

self.assertEqual(file_list.allfiles, ['a.py', 'b.txt'])

1279

1280

def test_process_template(self):

1281

+ l = make_local_path

1282

# invalid lines

1283

file_list = FileList()

1284

for action in ('include', 'exclude', 'global-include',

1285

@@ -127,7 +195,7 @@

1286

1287

# include

1288

file_list = FileList()

1289

- file_list.set_allfiles(['a.py', 'b.txt', 'd/c.py'])

1290

+ file_list.set_allfiles(['a.py', 'b.txt', l('d/c.py')])

1291

1292

file_list.process_template_line('include *.py')

1293

self.assertEqual(file_list.files, ['a.py'])

1294

@@ -139,31 +207,31 @@

1295

1296

# exclude

1297

file_list = FileList()

1298

- file_list.files = ['a.py', 'b.txt', 'd/c.py']

1299

+ file_list.files = ['a.py', 'b.txt', l('d/c.py')]

1300

1301

file_list.process_template_line('exclude *.py')

1302

- self.assertEqual(file_list.files, ['b.txt', 'd/c.py'])

1303

+ self.assertEqual(file_list.files, ['b.txt', l('d/c.py')])

1304

self.assertNoWarnings()

1305

1306

file_list.process_template_line('exclude *.rb')

1307

- self.assertEqual(file_list.files, ['b.txt', 'd/c.py'])

1308

+ self.assertEqual(file_list.files, ['b.txt', l('d/c.py')])

1309

self.assertWarnings()

1310

1311

# global-include

1312

file_list = FileList()

1313

- file_list.set_allfiles(['a.py', 'b.txt', 'd/c.py'])

1314

+ file_list.set_allfiles(['a.py', 'b.txt', l('d/c.py')])

1315

1316

file_list.process_template_line('global-include *.py')

1317

- self.assertEqual(file_list.files, ['a.py', 'd/c.py'])

1318

+ self.assertEqual(file_list.files, ['a.py', l('d/c.py')])

1319

self.assertNoWarnings()

1320

1321

file_list.process_template_line('global-include *.rb')

1322

- self.assertEqual(file_list.files, ['a.py', 'd/c.py'])

1323

+ self.assertEqual(file_list.files, ['a.py', l('d/c.py')])

1324

self.assertWarnings()

1325

1326

# global-exclude

1327

file_list = FileList()

1328

- file_list.files = ['a.py', 'b.txt', 'd/c.py']

1329

+ file_list.files = ['a.py', 'b.txt', l('d/c.py')]

1330

1331

file_list.process_template_line('global-exclude *.py')

1332

self.assertEqual(file_list.files, ['b.txt'])

1333

@@ -175,50 +243,52 @@

1334

1335

# recursive-include

1336

file_list = FileList()

1337

- file_list.set_allfiles(['a.py', 'd/b.py', 'd/c.txt', 'd/d/e.py'])

1338

+ file_list.set_allfiles(['a.py', l('d/b.py'), l('d/c.txt'),

1339

+ l('d/d/e.py')])

1340

1341

file_list.process_template_line('recursive-include d *.py')

1342

- self.assertEqual(file_list.files, ['d/b.py', 'd/d/e.py'])

1343

+ self.assertEqual(file_list.files, [l('d/b.py'), l('d/d/e.py')])

1344

self.assertNoWarnings()

1345

1346

file_list.process_template_line('recursive-include e *.py')

1347

- self.assertEqual(file_list.files, ['d/b.py', 'd/d/e.py'])

1348

+ self.assertEqual(file_list.files, [l('d/b.py'), l('d/d/e.py')])

1349

self.assertWarnings()

1350

1351

# recursive-exclude

1352

file_list = FileList()

1353

- file_list.files = ['a.py', 'd/b.py', 'd/c.txt', 'd/d/e.py']

1354

+ file_list.files = ['a.py', l('d/b.py'), l('d/c.txt'), l('d/d/e.py')]

1355

1356

file_list.process_template_line('recursive-exclude d *.py')

1357

- self.assertEqual(file_list.files, ['a.py', 'd/c.txt'])

1358

+ self.assertEqual(file_list.files, ['a.py', l('d/c.txt')])

1359

self.assertNoWarnings()

1360

1361

file_list.process_template_line('recursive-exclude e *.py')

1362

- self.assertEqual(file_list.files, ['a.py', 'd/c.txt'])

1363

+ self.assertEqual(file_list.files, ['a.py', l('d/c.txt')])

1364

self.assertWarnings()

1365

1366

# graft

1367

file_list = FileList()

1368

- file_list.set_allfiles(['a.py', 'd/b.py', 'd/d/e.py', 'f/f.py'])

1369

+ file_list.set_allfiles(['a.py', l('d/b.py'), l('d/d/e.py'),

1370

+ l('f/f.py')])

1371

1372

file_list.process_template_line('graft d')

1373

- self.assertEqual(file_list.files, ['d/b.py', 'd/d/e.py'])

1374

+ self.assertEqual(file_list.files, [l('d/b.py'), l('d/d/e.py')])

1375

self.assertNoWarnings()

1376

1377

file_list.process_template_line('graft e')

1378

- self.assertEqual(file_list.files, ['d/b.py', 'd/d/e.py'])

1379

+ self.assertEqual(file_list.files, [l('d/b.py'), l('d/d/e.py')])

1380

self.assertWarnings()

1381

1382

# prune

1383

file_list = FileList()

1384

- file_list.files = ['a.py', 'd/b.py', 'd/d/e.py', 'f/f.py']

1385

+ file_list.files = ['a.py', l('d/b.py'), l('d/d/e.py'), l('f/f.py')]

1386

1387

file_list.process_template_line('prune d')

1388

- self.assertEqual(file_list.files, ['a.py', 'f/f.py'])

1389

+ self.assertEqual(file_list.files, ['a.py', l('f/f.py')])

1390

self.assertNoWarnings()

1391

1392

file_list.process_template_line('prune e')

1393

- self.assertEqual(file_list.files, ['a.py', 'f/f.py'])

1394

+ self.assertEqual(file_list.files, ['a.py', l('f/f.py')])

1395

self.assertWarnings()

1396

1397

1398

diff -r 7085403daf43 Lib/distutils/tests/test_sdist.py

1399

--- a/Lib/distutils/tests/test_sdist.py

1400

+++ b/Lib/distutils/tests/test_sdist.py

1401

@@ -6,8 +6,14 @@

1402

import zipfile

1403

from os.path import join

1404

from textwrap import dedent

1405

+from test.support import captured_stdout, check_warnings, run_unittest

1406

1407

-from test.support import captured_stdout, check_warnings, run_unittest

1408

+try:

1409

+ import zlib

1410

+ ZLIB_SUPPORT = True

1411

+except ImportError:

1412

+ ZLIB_SUPPORT = False

1413

+

1414

1415

from distutils.command.sdist import sdist, show_formats

1416

from distutils.core import Distribution

1417

@@ -28,6 +34,7 @@

1418

MANIFEST = """\

1419

# file GENERATED by distutils, do NOT edit

1420

README

1421

+buildout.cfg

1422

inroot.txt

1423

setup.py

1424

data%(sep)sdata.dt

1425

@@ -39,13 +46,6 @@

1426

somecode%(sep)sdoc.txt

1427

"""

1428

1429

-try:

1430

- import zlib

1431

- ZLIB_SUPPORT = True

1432

-except ImportError:

1433

- ZLIB_SUPPORT = False

1434

-

1435

-

1436

class SDistTestCase(PyPIRCCommandTestCase):

1437

1438

def setUp(self):

1439

@@ -143,7 +143,7 @@

1440

dist_folder = join(self.tmp_dir, 'dist')

1441

result = os.listdir(dist_folder)

1442

result.sort()

1443

- self.assertEqual(result, ['fake-1.0.tar', 'fake-1.0.tar.gz'] )

1444

+ self.assertEqual(result, ['fake-1.0.tar', 'fake-1.0.tar.gz'])

1445

1446

os.remove(join(dist_folder, 'fake-1.0.tar'))

1447

os.remove(join(dist_folder, 'fake-1.0.tar.gz'))

1448

@@ -180,11 +180,18 @@

1449

self.write_file((data_dir, 'data.dt'), '#')

1450

some_dir = join(self.tmp_dir, 'some')

1451

os.mkdir(some_dir)

1452

+ # make sure VCS directories are pruned (#14004)

1453

+ hg_dir = join(self.tmp_dir, '.hg')

1454

+ os.mkdir(hg_dir)

1455

+ self.write_file((hg_dir, 'last-message.txt'), '#')

1456

+ # a buggy regex used to prevent this from working on windows (#6884)

1457

+ self.write_file((self.tmp_dir, 'buildout.cfg'), '#')

1458

self.write_file((self.tmp_dir, 'inroot.txt'), '#')

1459

self.write_file((some_dir, 'file.txt'), '#')

1460

self.write_file((some_dir, 'other_file.txt'), '#')

1461

1462

dist.data_files = [('data', ['data/data.dt',

1463

+ 'buildout.cfg',

1464

'inroot.txt',

1465

'notexisting']),

1466

'some/file.txt',

1467

@@ -214,15 +221,15 @@

1468

zip_file.close()

1469

1470

# making sure everything was added

1471

- self.assertEqual(len(content), 11)

1472

+ self.assertEqual(len(content), 12)

1473

1474

# checking the MANIFEST

1475

f = open(join(self.tmp_dir, 'MANIFEST'))

1476

try:

1477

manifest = f.read()

1478

- self.assertEqual(manifest, MANIFEST % {'sep': os.sep})

1479

finally:

1480

f.close()

1481

+ self.assertEqual(manifest, MANIFEST % {'sep': os.sep})

1482

1483

@unittest.skipUnless(ZLIB_SUPPORT, 'Need zlib support to run')

1484

def test_metadata_check_option(self):

1485

@@ -319,6 +326,7 @@

1486

# filling data_files by pointing files in package_data

1487

dist.package_data = {'somecode': ['*.txt']}

1488

self.write_file((self.tmp_dir, 'somecode', 'doc.txt'), '#')

1489

+ cmd.formats = ['gztar']

1490

cmd.ensure_finalized()

1491

cmd.run()

1492

1493

diff -r 7085403daf43 Lib/lib2to3/tests/test_parser.py

1494

--- a/Lib/lib2to3/tests/test_parser.py

1495

+++ b/Lib/lib2to3/tests/test_parser.py

1496

@@ -11,10 +11,14 @@

1497

# Testing imports

1498

from . import support

1499

from .support import driver, test_dir

1500

+from test.support import verbose

1501

1502

# Python imports

1503

import os

1504

+import sys

1505

import unittest

1506

+import warnings

1507

+import subprocess

1508

1509

# Local imports

1510

from lib2to3.pgen2 import tokenize

1511

@@ -171,10 +175,12 @@

1512

try:

1513

tree = driver.parse_string(source)

1514

except ParseError as err:

1515

- print('ParseError on file', filepath, err)

1516

+ if verbose > 0:

1517

+ warnings.warn('ParseError on file %s (%s)' % (filepath, err))

1518

continue

1519

new = str(tree)

1520

- if diff(filepath, new):

1521

+ x = diff(filepath, new)

1522

+ if x:

1523

self.fail("Idempotency failed: %s" % filepath)

1524

1525

def test_extended_unpacking(self):

1526

@@ -183,6 +189,7 @@

1527

driver.parse_string("(z, *y, w) = m\n")

1528

driver.parse_string("for *z, m in d: pass\n")

1529

1530

+

1531

class TestLiterals(GrammarTest):

1532

1533

def validate(self, s):

1534

@@ -221,7 +228,7 @@

1535

with open('@', 'w') as f:

1536

f.write(str(result))

1537

fn = fn.replace('"', '\\"')

1538

- return os.system('diff -u "%s" @' % fn)

1539

+ return subprocess.call(['diff', '-u', fn, '@'], stdout=(subprocess.DEVNULL if verbose < 1 else None))

1540

finally:

1541

try:

1542

os.remove("@")

1543

diff -r 7085403daf43 Lib/logging/__init__.py

1544

--- a/Lib/logging/__init__.py

1545

+++ b/Lib/logging/__init__.py

1546

@@ -1,4 +1,4 @@

1547

1548

1549

#

1550

# Permission to use, copy, modify, and distribute this software and its

1551

# documentation for any purpose and without fee is hereby granted,

1552

@@ -16,9 +16,9 @@

1553

1554

"""

1555

Logging package for Python. Based on PEP 282 and comments thereto in

1556

-comp.lang.python, and influenced by Apache's log4j system.

1557

+comp.lang.python.

1558

1559

1560

1561

1562

To use, simply 'import logging' and log away!

1563

"""

1564

@@ -917,8 +917,12 @@

1565

"""

1566

Flushes the stream.

1567

"""

1568

- if self.stream and hasattr(self.stream, "flush"):

1569

- self.stream.flush()

1570

+ self.acquire()

1571

+ try:

1572

+ if self.stream and hasattr(self.stream, "flush"):

1573

+ self.stream.flush()

1574

+ finally:

1575

+ self.release()

1576

1577

def emit(self, record):

1578

"""

1579

@@ -969,12 +973,16 @@

1580

"""

1581

Closes the stream.

1582

"""

1583

- if self.stream:

1584

- self.flush()

1585

- if hasattr(self.stream, "close"):

1586

- self.stream.close()

1587

- StreamHandler.close(self)

1588

- self.stream = None

1589

+ self.acquire()

1590

+ try:

1591

+ if self.stream:

1592

+ self.flush()

1593

+ if hasattr(self.stream, "close"):

1594

+ self.stream.close()

1595

+ StreamHandler.close(self)

1596

+ self.stream = None

1597

+ finally:

1598

+ self.release()

1599

1600

def _open(self):

1601

"""

1602

diff -r 7085403daf43 Lib/logging/handlers.py

1603

--- a/Lib/logging/handlers.py

1604

+++ b/Lib/logging/handlers.py

1605

@@ -1,4 +1,4 @@

1606

1607

1608

#

1609

# Permission to use, copy, modify, and distribute this software and its

1610

# documentation for any purpose and without fee is hereby granted,

1611

@@ -16,10 +16,9 @@

1612

1613

"""

1614

Additional handlers for the logging package for Python. The core package is

1615

-based on PEP 282 and comments thereto in comp.lang.python, and influenced by

1616

-Apache's log4j system.

1617

+based on PEP 282 and comments thereto in comp.lang.python.

1618

1619

1620

1621

1622

To use, simply 'import logging.handlers' and log away!

1623

"""

1624

@@ -554,10 +553,14 @@

1625

"""

1626

Closes the socket.

1627

"""

1628

- if self.sock:

1629

- self.sock.close()

1630

- self.sock = None

1631

- logging.Handler.close(self)

1632

+ self.acquire()

1633

+ try:

1634

+ if self.sock:

1635

+ self.sock.close()

1636

+ self.sock = None

1637

+ logging.Handler.close(self)

1638

+ finally:

1639

+ self.release()

1640

1641

class DatagramHandler(SocketHandler):

1642

"""

1643

@@ -752,9 +755,13 @@

1644

"""

1645

Closes the socket.

1646

"""

1647

- if self.unixsocket:

1648

- self.socket.close()

1649

- logging.Handler.close(self)

1650

+ self.acquire()

1651

+ try:

1652

+ if self.unixsocket:

1653

+ self.socket.close()

1654

+ logging.Handler.close(self)

1655

+ finally:

1656

+ self.release()

1657

1658

def mapPriority(self, levelName):

1659

"""

1660

@@ -1095,7 +1102,11 @@

1661

1662

This version just zaps the buffer to empty.

1663

"""

1664

- self.buffer = []

1665

+ self.acquire()

1666

+ try:

1667

+ self.buffer = []

1668

+ finally:

1669

+ self.release()

1670

1671

def close(self):

1672

"""

1673

@@ -1145,18 +1156,26 @@

1674

1675

The record buffer is also cleared by this operation.

1676

"""

1677

- if self.target:

1678

- for record in self.buffer:

1679

- self.target.handle(record)

1680

- self.buffer = []

1681

+ self.acquire()

1682

+ try:

1683

+ if self.target:

1684

+ for record in self.buffer:

1685

+ self.target.handle(record)

1686

+ self.buffer = []

1687

+ finally:

1688

+ self.release()

1689

1690

def close(self):

1691

"""

1692

Flush, set the target to None and lose the buffer.

1693

"""

1694

self.flush()

1695

- self.target = None

1696

- BufferingHandler.close(self)

1697

+ self.acquire()

1698

+ try:

1699

+ self.target = None

1700

+ BufferingHandler.close(self)

1701

+ finally:

1702

+ self.release()

1703

1704

1705

class QueueHandler(logging.Handler):

1706

diff -r 7085403daf43 Lib/test/regrtest.py

1707

--- a/Lib/test/regrtest.py

1708

+++ b/Lib/test/regrtest.py

1709

@@ -677,10 +677,10 @@

1710

if bad:

1711

print(count(len(bad), "test"), "failed:")

1712

printlist(bad)

1713

- if environment_changed:

1714

- print("{} altered the execution environment:".format(

1715

- count(len(environment_changed), "test")))

1716

- printlist(environment_changed)

1717

+ if environment_changed:

1718

+ print("{} altered the execution environment:".format(

1719

+ count(len(environment_changed), "test")))

1720

+ printlist(environment_changed)

1721

if skipped and not quiet:

1722

print(count(len(skipped), "test"), "skipped:")

1723

printlist(skipped)

1724

@@ -890,7 +890,9 @@

1725

'logging._handlers', 'logging._handlerList',

1726

'shutil.archive_formats', 'shutil.unpack_formats',

1727

'sys.warnoptions', 'threading._dangling',

1728

- 'multiprocessing.process._dangling')

1729

+ 'multiprocessing.process._dangling',

1730

+ 'support.TESTFN',

1731

+ )

1732

1733

def get_sys_argv(self):

1734

return id(sys.argv), sys.argv, sys.argv[:]

1735

@@ -1020,6 +1022,21 @@

1736

multiprocessing.process._dangling.clear()

1737

multiprocessing.process._dangling.update(saved)

1738

1739

+ def get_support_TESTFN(self):

1740

+ if os.path.isfile(support.TESTFN):

1741

+ result = 'f'

1742

+ elif os.path.isdir(support.TESTFN):

1743

+ result = 'd'

1744

+ else:

1745

+ result = None

1746

+ return result

1747

+ def restore_support_TESTFN(self, saved_value):

1748

+ if saved_value is None:

1749

+ if os.path.isfile(support.TESTFN):

1750

+ os.unlink(support.TESTFN)

1751

+ elif os.path.isdir(support.TESTFN):

1752

+ shutil.rmtree(support.TESTFN)

1753

+

1754

def resource_info(self):

1755

for name in self.resources:

1756

method_suffix = name.replace('.', '_')

1757

diff -r 7085403daf43 Lib/test/test_base64.py

1758

--- a/Lib/test/test_base64.py

1759

+++ b/Lib/test/test_base64.py

1760

@@ -2,6 +2,7 @@

1761

from test import support

1762

import base64

1763

import binascii

1764

+import os

1765

import sys

1766

import subprocess

1767

1768

@@ -227,6 +228,10 @@

1769

1770

1771

class TestMain(unittest.TestCase):

1772

+ def tearDown(self):

1773

+ if os.path.exists(support.TESTFN):

1774

+ os.unlink(support.TESTFN)

1775

+

1776

def get_output(self, *args, **options):

1777

args = (sys.executable, '-m', 'base64') + args

1778

return subprocess.check_output(args, **options)

1779

diff -r 7085403daf43 Lib/test/test_descr.py

1780

--- a/Lib/test/test_descr.py

1781

+++ b/Lib/test/test_descr.py

1782

@@ -1,8 +1,10 @@

1783

import builtins

1784

+import gc

1785

import sys

1786

import types

1787

import math

1788

import unittest

1789

+import weakref

1790

1791

from copy import deepcopy

1792

from test import support

1793

@@ -1186,7 +1188,6 @@

1794

self.assertEqual(Counted.counter, 0)

1795

1796

# Test lookup leaks [SF bug 572567]

1797

- import gc

1798

if hasattr(gc, 'get_objects'):

1799

class G(object):

1800

def __eq__(self, other):

1801

@@ -4380,7 +4381,6 @@

1802

self.assertRaises(AttributeError, getattr, C(), "attr")

1803

self.assertEqual(descr.counter, 4)

1804

1805

- import gc

1806

class EvilGetattribute(object):

1807

# This used to segfault

1808

def __getattr__(self, name):

1809

@@ -4429,6 +4429,21 @@

1810

foo = Foo()

1811

str(foo)

1812

1813

+ def test_cycle_through_dict(self):

1814

+ # See bug #1469629

1815

+ class X(dict):

1816

+ def __init__(self):

1817

+ dict.__init__(self)

1818

+ self.__dict__ = self

1819

+ x = X()

1820

+ x.attr = 42

1821

+ wr = weakref.ref(x)

1822

+ del x

1823

+ support.gc_collect()

1824

+ self.assertIsNone(wr())

1825

+ for o in gc.get_objects():

1826

+ self.assertIsNot(type(o), X)

1827

+

1828

class DictProxyTests(unittest.TestCase):

1829

def setUp(self):

1830

class C(object):

1831

diff -r 7085403daf43 Lib/test/test_dict.py

1832

--- a/Lib/test/test_dict.py

1833

+++ b/Lib/test/test_dict.py

1834

@@ -299,6 +299,26 @@

1835

x.fail = True

1836

self.assertRaises(Exc, d.setdefault, x, [])

1837

1838

+ def test_setdefault_atomic(self):

1839

+ # Issue #13521: setdefault() calls __hash__ and __eq__ only once.

1840

+ class Hashed(object):

1841

+ def __init__(self):

1842

+ self.hash_count = 0

1843

+ self.eq_count = 0

1844

+ def __hash__(self):

1845

+ self.hash_count += 1

1846

+ return 42

1847

+ def __eq__(self, other):

1848

+ self.eq_count += 1

1849

+ return id(self) == id(other)

1850

+ hashed1 = Hashed()

1851

+ y = {hashed1: 5}

1852

+ hashed2 = Hashed()

1853

+ y.setdefault(hashed2, [])

1854

+ self.assertEqual(hashed1.hash_count, 1)

1855

+ self.assertEqual(hashed2.hash_count, 1)

1856

+ self.assertEqual(hashed1.eq_count + hashed2.eq_count, 1)

1857

+

1858

def test_popitem(self):

1859

# dict.popitem()

1860

for copymode in -1, +1:

1861

diff -r 7085403daf43 Lib/test/test_exceptions.py

1862

--- a/Lib/test/test_exceptions.py

1863

+++ b/Lib/test/test_exceptions.py

1864

@@ -38,7 +38,7 @@

1865

try:

1866

try:

1867

import marshal

1868

- marshal.loads('')

1869

+ marshal.loads(b'')

1870

except EOFError:

1871

pass

1872

finally:

1873

diff -r 7085403daf43 Lib/test/test_fractions.py

1874

--- a/Lib/test/test_fractions.py

1875

+++ b/Lib/test/test_fractions.py

1876

@@ -6,6 +6,7 @@

1877

import numbers

1878

import operator

1879

import fractions

1880

+import sys

1881

import unittest

1882

from copy import copy, deepcopy

1883

from pickle import dumps, loads

1884

@@ -76,6 +77,9 @@

1885

def __float__(self):

1886

assert False, "__float__ should not be invoked"

1887

1888

+class DummyFraction(fractions.Fraction):

1889

+ """Dummy Fraction subclass for copy and deepcopy testing."""

1890

+

1891

class GcdTest(unittest.TestCase):

1892

1893

def testMisc(self):

1894

@@ -286,9 +290,14 @@

1895

self.assertEqual(F(201, 200).limit_denominator(100), F(1))

1896

self.assertEqual(F(201, 200).limit_denominator(101), F(102, 101))

1897

self.assertEqual(F(0).limit_denominator(10000), F(0))

1898

+ for i in (0, -1):

1899

+ self.assertRaisesMessage(

1900

+ ValueError, "max_denominator should be at least 1",

1901

+ F(1).limit_denominator, i)

1902

1903

def testConversions(self):

1904

self.assertTypedEquals(-1, math.trunc(F(-11, 10)))

1905

+ self.assertTypedEquals(1, math.trunc(F(11, 10)))

1906

self.assertTypedEquals(-2, math.floor(F(-11, 10)))

1907

self.assertTypedEquals(-1, math.ceil(F(-11, 10)))

1908

self.assertTypedEquals(-1, math.ceil(F(-10, 10)))

1909

@@ -329,6 +338,7 @@

1910

self.assertEqual(F(8, 27), F(2, 3) ** F(3))

1911

self.assertEqual(F(27, 8), F(2, 3) ** F(-3))

1912

self.assertTypedEquals(2.0, F(4) ** F(1, 2))

1913

+ self.assertEqual(F(1, 1), +F(1, 1))

1914

z = pow(F(-1), F(1, 2))

1915

self.assertAlmostEqual(z.real, 0)

1916

self.assertEqual(z.imag, 1)

1917

@@ -395,6 +405,10 @@

1918

TypeError,

1919

"unsupported operand type(s) for +: 'Fraction' and 'Decimal'",

1920

operator.add, F(3,11), Decimal('3.1415926'))

1921

+ self.assertRaisesMessage(

1922

+ TypeError,

1923

+ "unsupported operand type(s) for +: 'Decimal' and 'Fraction'",

1924

+ operator.add, Decimal('3.1415926'), F(3,11))

1925

1926

def testComparisons(self):

1927

self.assertTrue(F(1, 2) < F(2, 3))

1928

@@ -538,9 +552,12 @@

1929

self.assertEqual("7", str(F(7, 1)))

1930

1931

def testHash(self):

1932

+ hmod = sys.hash_info.modulus

1933

+ hinf = sys.hash_info.inf

1934

self.assertEqual(hash(2.5), hash(F(5, 2)))

1935

self.assertEqual(hash(10**50), hash(F(10**50)))

1936

self.assertNotEqual(hash(float(10**23)), hash(F(10**23)))

1937

+ self.assertEqual(hinf, hash(F(1, hmod)))

1938

# Check that __hash__ produces the same value as hash(), for

1939

# consistency with int and Decimal. (See issue #10356.)

1940

self.assertEqual(hash(F(-1)), F(-1).__hash__())

1941

@@ -574,9 +591,14 @@

1942

1943

def test_copy_deepcopy_pickle(self):

1944

r = F(13, 7)

1945

+ dr = DummyFraction(13, 7)

1946

self.assertEqual(r, loads(dumps(r)))

1947

self.assertEqual(id(r), id(copy(r)))

1948

self.assertEqual(id(r), id(deepcopy(r)))

1949

+ self.assertNotEqual(id(dr), id(copy(dr)))

1950

+ self.assertNotEqual(id(dr), id(deepcopy(dr)))

1951

+ self.assertTypedEquals(dr, copy(dr))

1952

+ self.assertTypedEquals(dr, deepcopy(dr))

1953

1954

def test_slots(self):

1955

# Issue 4998

1956

diff -r 7085403daf43 Lib/test/test_mailbox.py

1957

--- a/Lib/test/test_mailbox.py

1958

+++ b/Lib/test/test_mailbox.py

1959

@@ -7,6 +7,7 @@

1960

import email.message

1961

import re

1962

import io

1963

+import shutil

1964

import tempfile

1965

from test import support

1966

import unittest

1967

@@ -38,12 +39,7 @@

1968

def _delete_recursively(self, target):

1969

# Delete a file or delete a directory recursively

1970

if os.path.isdir(target):

1971

- for path, dirs, files in os.walk(target, topdown=False):

1972

- for name in files:

1973

- os.remove(os.path.join(path, name))

1974

- for name in dirs:

1975

- os.rmdir(os.path.join(path, name))

1976

- os.rmdir(target)

1977

+ shutil.rmtree(target)

1978

elif os.path.exists(target):

1979

os.remove(target)

1980

1981

@@ -2029,6 +2025,10 @@

1982

def setUp(self):

1983

# create a new maildir mailbox to work with:

1984

self._dir = support.TESTFN

1985

+ if os.path.isdir(self._dir):

1986

+ shutil.rmtree(self._dir)

1987

+ elif os.path.isfile(self._dir):

1988

+ os.unlink(self._dir)

1989

os.mkdir(self._dir)

1990

os.mkdir(os.path.join(self._dir, "cur"))

1991

os.mkdir(os.path.join(self._dir, "tmp"))

1992

diff -r 7085403daf43 Lib/test/test_marshal.py

1993

--- a/Lib/test/test_marshal.py

1994

+++ b/Lib/test/test_marshal.py

1995

@@ -1,6 +1,7 @@

1996

#!/usr/bin/env python3

1997

1998

from test import support

1999

+import array

2000

import marshal

2001

import sys

2002

import unittest

2003

@@ -137,6 +138,27 @@

2004

for constructor in (set, frozenset):

2005

self.helper(constructor(self.d.keys()))

2006

2007

+

2008

+class BufferTestCase(unittest.TestCase, HelperMixin):

2009

+

2010

+ def test_bytearray(self):

2011

+ b = bytearray(b"abc")

2012

+ self.helper(b)

2013

+ new = marshal.loads(marshal.dumps(b))

2014

+ self.assertEqual(type(new), bytes)

2015

+

2016

+ def test_memoryview(self):

2017

+ b = memoryview(b"abc")

2018

+ self.helper(b)

2019

+ new = marshal.loads(marshal.dumps(b))

2020

+ self.assertEqual(type(new), bytes)

2021

+

2022

+ def test_array(self):

2023

+ a = array.array('B', b"abc")

2024

+ new = marshal.loads(marshal.dumps(a))

2025

+ self.assertEqual(new, b"abc")

2026

+

2027

+

2028

class BugsTestCase(unittest.TestCase):

2029

def test_bug_5888452(self):

2030

# Simple-minded check for SF 588452: Debug build crashes

2031

@@ -162,7 +184,7 @@

2032

pass

2033

2034

def test_loads_recursion(self):

2035

- s = 'c' + ('X' * 4*4) + '{' * 2**20

2036

+ s = b'c' + (b'X' * 4*4) + b'{' * 2**20

2037

self.assertRaises(ValueError, marshal.loads, s)

2038

2039

def test_recursion_limit(self):

2040

@@ -235,6 +257,11 @@

2041

finally:

2042

support.unlink(support.TESTFN)

2043

2044

+ def test_loads_reject_unicode_strings(self):

2045

+ # Issue #14177: marshal.loads() should not accept unicode strings

2046

+ unicode_string = 'T'

2047

+ self.assertRaises(TypeError, marshal.loads, unicode_string)

2048

+

2049

2050

def test_main():

2051

support.run_unittest(IntTestCase,

2052

@@ -243,6 +270,7 @@

2053

CodeTestCase,

2054

ContainerTestCase,

2055

ExceptionTestCase,

2056

+ BufferTestCase,

2057

BugsTestCase)

2058

2059

if __name__ == "__main__":

2060

diff -r 7085403daf43 Lib/test/test_minidom.py

2061

--- a/Lib/test/test_minidom.py

2062

+++ b/Lib/test/test_minidom.py

2063

@@ -350,13 +350,31 @@

2064

def testGetAttrList(self):

2065

pass

2066

2067

- def testGetAttrValues(self): pass

2068

+ def testGetAttrValues(self):

2069

+ pass

2070

2071

- def testGetAttrLength(self): pass

2072

+ def testGetAttrLength(self):

2073

+ pass

2074

2075

- def testGetAttribute(self): pass

2076

+ def testGetAttribute(self):

2077

+ dom = Document()

2078

+ child = dom.appendChild(

2079

+ dom.createElementNS("http://www.python.org", "python:abc"))

2080

+ self.assertEqual(child.getAttribute('missing'), '')

2081

2082

- def testGetAttributeNS(self): pass

2083

+ def testGetAttributeNS(self):

2084

+ dom = Document()

2085

+ child = dom.appendChild(

2086

+ dom.createElementNS("http://www.python.org", "python:abc"))

2087

+ child.setAttributeNS("http://www.w3.org", "xmlns:python",

2088

+ "http://www.python.org")

2089

+ self.assertEqual(child.getAttributeNS("http://www.w3.org", "python"),

2090

+ 'http://www.python.org')

2091

+ self.assertEqual(child.getAttributeNS("http://www.w3.org", "other"),

2092

+ '')

2093

+ child2 = child.appendChild(dom.createElement('abc'))

2094

+ self.assertEqual(child2.getAttributeNS("http://www.python.org", "missing"),

2095

+ '')

2096

2097

def testGetAttributeNode(self): pass

2098

2099

diff -r 7085403daf43 Lib/test/test_re.py

2100

--- a/Lib/test/test_re.py

2101

+++ b/Lib/test/test_re.py

2102

@@ -1,4 +1,5 @@

2103

-from test.support import verbose, run_unittest

2104

+from test.support import verbose, run_unittest, gc_collect

2105

+import io

2106

import re

2107

from re import Scanner

2108

import sys

2109

@@ -16,6 +17,17 @@

2110

2111

class ReTests(unittest.TestCase):

2112

2113

+ def test_keep_buffer(self):

2114

+ # See bug 14212

2115

+ b = bytearray(b'x')

2116

+ it = re.finditer(b'a', b)

2117

+ with self.assertRaises(BufferError):

2118

+ b.extend(b'x'*400)

2119

+ list(it)

2120

+ del it

2121

+ gc_collect()

2122

+ b.extend(b'x'*400)

2123

+

2124

def test_weakref(self):

2125

s = 'QabbbcR'

2126

x = re.compile('ab+c')

2127

@@ -355,6 +367,32 @@

2128

self.assertEqual(re.search(r"\d\D\w\W\s\S",

2129

"1aa! a", re.UNICODE).group(0), "1aa! a")

2130

2131

+ def test_string_boundaries(self):

2132

+ # See http://bugs.python.org/issue10713

2133

+ self.assertEqual(re.search(r"\b(abc)\b", "abc").group(1),

2134

+ "abc")

2135

+ # There's a word boundary at the start of a string.

2136

+ self.assertTrue(re.match(r"\b", "abc"))

2137

+ # A non-empty string includes a non-boundary zero-length match.

2138

+ self.assertTrue(re.search(r"\B", "abc"))

2139

+ # There is no non-boundary match at the start of a string.

2140

+ self.assertFalse(re.match(r"\B", "abc"))

2141

+ # However, an empty string contains no word boundaries, and also no

2142

+ # non-boundaries.

2143

+ self.assertEqual(re.search(r"\B", ""), None)

2144

+ # This one is questionable and different from the perlre behaviour,

2145

+ # but describes current behavior.

2146

+ self.assertEqual(re.search(r"\b", ""), None)

2147

+ # A single word-character string has two boundaries, but no

2148

+ # non-boundary gaps.

2149

+ self.assertEqual(len(re.findall(r"\b", "a")), 2)

2150

+ self.assertEqual(len(re.findall(r"\B", "a")), 0)

2151

+ # If there are no words, there are no boundaries

2152

+ self.assertEqual(len(re.findall(r"\b", " ")), 0)

2153

+ self.assertEqual(len(re.findall(r"\b", " ")), 0)

2154

+ # Can match around the whitespace.

2155

+ self.assertEqual(len(re.findall(r"\B", " ")), 2)

2156

+

2157

def test_bigcharset(self):

2158

self.assertEqual(re.match("([\u2222\u2223])",

2159

"\u2222").group(1), "\u2222")

2160

diff -r 7085403daf43 Lib/test/test_strptime.py

2161

--- a/Lib/test/test_strptime.py

2162

+++ b/Lib/test/test_strptime.py

2163

@@ -38,9 +38,9 @@

2164

comparison = testing[self.time_tuple[tuple_position]]

2165

self.assertIn(strftime_output, testing,

2166

"%s: not found in tuple" % error_msg)

2167

- self.assertTrue(comparison == strftime_output,

2168

- "%s: position within tuple incorrect; %s != %s" %

2169

- (error_msg, comparison, strftime_output))

2170

+ self.assertEqual(comparison, strftime_output,

2171

+ "%s: position within tuple incorrect; %s != %s" %

2172

+ (error_msg, comparison, strftime_output))

2173

2174

def test_weekday(self):

2175

# Make sure that full and abbreviated weekday names are correct in

2176

@@ -65,8 +65,8 @@

2177

"AM/PM representation not in tuple")

2178

if self.time_tuple[3] < 12: position = 0

2179

else: position = 1

2180

- self.assertTrue(strftime_output == self.LT_ins.am_pm[position],

2181

- "AM/PM representation in the wrong position within the tuple")

2182

+ self.assertEqual(self.LT_ins.am_pm[position], strftime_output,

2183

+ "AM/PM representation in the wrong position within the tuple")

2184

2185

def test_timezone(self):

2186

# Make sure timezone is correct

2187

@@ -86,17 +86,14 @@

2188

# output.

2189

magic_date = (1999, 3, 17, 22, 44, 55, 2, 76, 0)

2190

strftime_output = time.strftime("%c", magic_date)

2191

- self.assertTrue(strftime_output == time.strftime(self.LT_ins.LC_date_time,

2192

- magic_date),

2193

- "LC_date_time incorrect")

2194

+ self.assertEqual(time.strftime(self.LT_ins.LC_date_time, magic_date),

2195

+ strftime_output, "LC_date_time incorrect")

2196

strftime_output = time.strftime("%x", magic_date)

2197

- self.assertTrue(strftime_output == time.strftime(self.LT_ins.LC_date,

2198

- magic_date),

2199

- "LC_date incorrect")

2200

+ self.assertEqual(time.strftime(self.LT_ins.LC_date, magic_date),

2201

+ strftime_output, "LC_date incorrect")

2202

strftime_output = time.strftime("%X", magic_date)

2203

- self.assertTrue(strftime_output == time.strftime(self.LT_ins.LC_time,

2204

- magic_date),

2205

- "LC_time incorrect")

2206

+ self.assertEqual(time.strftime(self.LT_ins.LC_time, magic_date),

2207

+ strftime_output, "LC_time incorrect")

2208

LT = _strptime.LocaleTime()

2209

LT.am_pm = ('', '')

2210

self.assertTrue(LT.LC_time, "LocaleTime's LC directives cannot handle "

2211

@@ -168,8 +165,8 @@

2212

# Fixes bug #661354

2213

test_locale = _strptime.LocaleTime()

2214

test_locale.timezone = (frozenset(), frozenset())

2215

- self.assertTrue(_strptime.TimeRE(test_locale).pattern("%Z") == '',

2216

- "with timezone == ('',''), TimeRE().pattern('%Z') != ''")

2217

+ self.assertEqual(_strptime.TimeRE(test_locale).pattern("%Z"), '',

2218

+ "with timezone == ('',''), TimeRE().pattern('%Z') != ''")

2219

2220

def test_matching_with_escapes(self):

2221

# Make sure a format that requires escaping of characters works

2222

@@ -195,7 +192,7 @@

2223

# so as to not allow to subpatterns to end up next to each other and

2224

# "steal" characters from each other.

2225

pattern = self.time_re.pattern('%j %H')

2226

- self.assertTrue(not re.match(pattern, "180"))

2227

+ self.assertFalse(re.match(pattern, "180"))

2228

self.assertTrue(re.match(pattern, "18 0"))

2229

2230

2231

diff -r 7085403daf43 Lib/test/test_tools.py

2232

--- /dev/null

2233

+++ b/Lib/test/test_tools.py

2234

@@ -0,0 +1,39 @@

2235

+"""Tests for scripts in the Tools directory.

2236

+

2237

+This file contains regression tests for some of the scripts found in the

2238

+Tools directory of a Python checkout or tarball, such as reindent.py.

2239

+"""

2240

+

2241

+import os

2242

+import unittest

2243

+import sysconfig

2244

+from test import support

2245

+from test.script_helper import assert_python_ok

2246

+

2247

+if not sysconfig.is_python_build():

2248

+ # XXX some installers do contain the tools, should we detect that

2249

+ # and run the tests in that case too?

2250

+ raise unittest.SkipTest('test irrelevant for an installed Python')

2251

+

2252

+srcdir = sysconfig.get_config_var('projectbase')

2253

+basepath = os.path.join(os.getcwd(), srcdir, 'Tools')

2254

+

2255

+

2256

+class ReindentTests(unittest.TestCase):

2257

+ script = os.path.join(basepath, 'scripts', 'reindent.py')

2258

+

2259

+ def test_noargs(self):

2260

+ assert_python_ok(self.script)

2261

+

2262

+ def test_help(self):

2263

+ rc, out, err = assert_python_ok(self.script, '-h')

2264

+ self.assertEqual(out, b'')

2265

+ self.assertGreater(err, b'')

2266

+

2267

+

2268

+def test_main():

2269

+ support.run_unittest(ReindentTests)

2270

+

2271

+

2272

+if __name__ == '__main__':

2273

+ unittest.main()

2274

diff -r 7085403daf43 Lib/test/test_weakref.py

2275

--- a/Lib/test/test_weakref.py

2276

+++ b/Lib/test/test_weakref.py

2277

@@ -812,11 +812,71 @@

2278

def __hash__(self):

2279

return hash(self.arg)

2280

2281

+class RefCycle:

2282

+ def __init__(self):

2283

+ self.cycle = self

2284

+

2285

2286

class MappingTestCase(TestBase):

2287

2288

COUNT = 10

2289

2290

+ def check_len_cycles(self, dict_type, cons):

2291

+ N = 20

2292

+ items = [RefCycle() for i in range(N)]

2293

+ dct = dict_type(cons(o) for o in items)

2294

+ # Keep an iterator alive

2295

+ it = dct.items()

2296

+ try:

2297

+ next(it)

2298

+ except StopIteration:

2299

+ pass

2300

+ del items

2301

+ gc.collect()

2302

+ n1 = len(dct)

2303

+ del it

2304

+ gc.collect()

2305

+ n2 = len(dct)

2306

+ # one item may be kept alive inside the iterator

2307

+ self.assertIn(n1, (0, 1))

2308

+ self.assertEqual(n2, 0)

2309

+

2310

+ def test_weak_keyed_len_cycles(self):

2311

+ self.check_len_cycles(weakref.WeakKeyDictionary, lambda k: (k, 1))

2312

+

2313

+ def test_weak_valued_len_cycles(self):

2314

+ self.check_len_cycles(weakref.WeakValueDictionary, lambda k: (1, k))

2315

+

2316

+ def check_len_race(self, dict_type, cons):

2317

+ # Extended sanity checks for len() in the face of cyclic collection

2318

+ self.addCleanup(gc.set_threshold, *gc.get_threshold())

2319

+ for th in range(1, 100):

2320

+ N = 20

2321

+ gc.collect(0)

2322

+ gc.set_threshold(th, th, th)

2323

+ items = [RefCycle() for i in range(N)]

2324

+ dct = dict_type(cons(o) for o in items)

2325

+ del items

2326

+ # All items will be collected at next garbage collection pass

2327

+ it = dct.items()

2328

+ try:

2329

+ next(it)

2330

+ except StopIteration:

2331

+ pass

2332

+ n1 = len(dct)

2333

+ del it

2334

+ n2 = len(dct)

2335

+ self.assertGreaterEqual(n1, 0)

2336

+ self.assertLessEqual(n1, N)

2337

+ self.assertGreaterEqual(n2, 0)

2338

+ self.assertLessEqual(n2, n1)

2339

+

2340

+ def test_weak_keyed_len_race(self):

2341

+ self.check_len_race(weakref.WeakKeyDictionary, lambda k: (k, 1))

2342

+

2343

+ def test_weak_valued_len_race(self):

2344

+ self.check_len_race(weakref.WeakValueDictionary, lambda k: (1, k))

2345

+

2346

def test_weak_values(self):

2347

#

2348

# This exercises d.copy(), d.items(), d[], del d[], len(d).

2349

diff -r 7085403daf43 Lib/test/test_weakset.py

2350

--- a/Lib/test/test_weakset.py

2351

+++ b/Lib/test/test_weakset.py

2352

@@ -17,6 +17,10 @@

2353

class Foo:

2354

pass

2355

2356

+class RefCycle:

2357

+ def __init__(self):

2358

+ self.cycle = self

2359

+

2360

2361

class TestWeakSet(unittest.TestCase):

2362

2363

@@ -24,6 +28,12 @@

2364

# need to keep references to them

2365

self.items = [ustr(c) for c in ('a', 'b', 'c')]

2366

self.items2 = [ustr(c) for c in ('x', 'y', 'z')]

2367

+ self.ab_items = [ustr(c) for c in 'ab']

2368

+ self.abcde_items = [ustr(c) for c in 'abcde']

2369

+ self.def_items = [ustr(c) for c in 'def']

2370

+ self.ab_weakset = WeakSet(self.ab_items)

2371

+ self.abcde_weakset = WeakSet(self.abcde_items)

2372

+ self.def_weakset = WeakSet(self.def_items)

2373

self.letters = [ustr(c) for c in string.ascii_letters]

2374

self.s = WeakSet(self.items)

2375

self.d = dict.fromkeys(self.items)

2376

@@ -67,6 +77,11 @@

2377

x = WeakSet(self.items + self.items2)

2378

c = C(self.items2)

2379

self.assertEqual(self.s.union(c), x)

2380

+ del c

2381

+ self.assertEqual(len(u), len(self.items) + len(self.items2))

2382

+ self.items2.pop()

2383

+ gc.collect()

2384

+ self.assertEqual(len(u), len(self.items) + len(self.items2))

2385

2386

def test_or(self):

2387

i = self.s.union(self.items2)

2388

@@ -74,14 +89,19 @@

2389

self.assertEqual(self.s | frozenset(self.items2), i)

2390

2391

def test_intersection(self):

2392

- i = self.s.intersection(self.items2)

2393

+ s = WeakSet(self.letters)

2394

+ i = s.intersection(self.items2)

2395

for c in self.letters:

2396

- self.assertEqual(c in i, c in self.d and c in self.items2)

2397

- self.assertEqual(self.s, WeakSet(self.items))

2398

+ self.assertEqual(c in i, c in self.items2 and c in self.letters)

2399

+ self.assertEqual(s, WeakSet(self.letters))

2400

self.assertEqual(type(i), WeakSet)

2401

for C in set, frozenset, dict.fromkeys, list, tuple:

2402

x = WeakSet([])

2403

- self.assertEqual(self.s.intersection(C(self.items2)), x)

2404

+ self.assertEqual(i.intersection(C(self.items)), x)

2405

+ self.assertEqual(len(i), len(self.items2))

2406

+ self.items2.pop()

2407

+ gc.collect()

2408

+ self.assertEqual(len(i), len(self.items2))

2409

2410

def test_isdisjoint(self):

2411

self.assertTrue(self.s.isdisjoint(WeakSet(self.items2)))

2412

@@ -112,6 +132,10 @@

2413

self.assertEqual(self.s, WeakSet(self.items))

2414

self.assertEqual(type(i), WeakSet)

2415

self.assertRaises(TypeError, self.s.symmetric_difference, [[]])

2416

+ self.assertEqual(len(i), len(self.items) + len(self.items2))

2417

+ self.items2.pop()

2418

+ gc.collect()

2419

+ self.assertEqual(len(i), len(self.items) + len(self.items2))

2420

2421

def test_xor(self):

2422

i = self.s.symmetric_difference(self.items2)

2423

@@ -119,22 +143,28 @@

2424

self.assertEqual(self.s ^ frozenset(self.items2), i)

2425

2426

def test_sub_and_super(self):

2427

- pl, ql, rl = map(lambda s: [ustr(c) for c in s], ['ab', 'abcde', 'def'])

2428

- p, q, r = map(WeakSet, (pl, ql, rl))

2429

- self.assertTrue(p < q)

2430

- self.assertTrue(p <= q)

2431

- self.assertTrue(q <= q)

2432

- self.assertTrue(q > p)

2433

- self.assertTrue(q >= p)

2434

- self.assertFalse(q < r)

2435

- self.assertFalse(q <= r)

2436

- self.assertFalse(q > r)

2437

- self.assertFalse(q >= r)

2438

+ self.assertTrue(self.ab_weakset <= self.abcde_weakset)

2439

+ self.assertTrue(self.abcde_weakset <= self.abcde_weakset)

2440

+ self.assertTrue(self.abcde_weakset >= self.ab_weakset)

2441

+ self.assertFalse(self.abcde_weakset <= self.def_weakset)

2442

+ self.assertFalse(self.abcde_weakset >= self.def_weakset)

2443

self.assertTrue(set('a').issubset('abc'))

2444

self.assertTrue(set('abc').issuperset('a'))

2445

self.assertFalse(set('a').issubset('cbs'))

2446

self.assertFalse(set('cbs').issuperset('a'))

2447

2448

+ def test_lt(self):

2449

+ self.assertTrue(self.ab_weakset < self.abcde_weakset)

2450

+ self.assertFalse(self.abcde_weakset < self.def_weakset)

2451

+ self.assertFalse(self.ab_weakset < self.ab_weakset)

2452

+ self.assertFalse(WeakSet() < WeakSet())

2453

+

2454

+ def test_gt(self):

2455

+ self.assertTrue(self.abcde_weakset > self.ab_weakset)

2456

+ self.assertFalse(self.abcde_weakset > self.def_weakset)

2457

+ self.assertFalse(self.ab_weakset > self.ab_weakset)

2458

+ self.assertFalse(WeakSet() > WeakSet())

2459

+

2460

def test_gc(self):

2461

# Create a nest of cycles to exercise overall ref count check

2462

s = WeakSet(Foo() for i in range(1000))

2463

@@ -359,6 +389,49 @@

2464

s.clear()

2465

self.assertEqual(len(s), 0)

2466

2467

+ def test_len_cycles(self):

2468

+ N = 20

2469

+ items = [RefCycle() for i in range(N)]

2470

+ s = WeakSet(items)

2471

+ del items

2472

+ it = iter(s)

2473

+ try:

2474

+ next(it)

2475

+ except StopIteration:

2476

+ pass

2477

+ gc.collect()

2478

+ n1 = len(s)

2479

+ del it

2480

+ gc.collect()

2481

+ n2 = len(s)

2482

+ # one item may be kept alive inside the iterator

2483

+ self.assertIn(n1, (0, 1))

2484

+ self.assertEqual(n2, 0)

2485

+

2486

+ def test_len_race(self):

2487

+ # Extended sanity checks for len() in the face of cyclic collection

2488

+ self.addCleanup(gc.set_threshold, *gc.get_threshold())

2489

+ for th in range(1, 100):

2490

+ N = 20

2491

+ gc.collect(0)

2492

+ gc.set_threshold(th, th, th)

2493

+ items = [RefCycle() for i in range(N)]

2494

+ s = WeakSet(items)

2495

+ del items

2496

+ # All items will be collected at next garbage collection pass

2497

+ it = iter(s)

2498

+ try:

2499

+ next(it)

2500

+ except StopIteration:

2501

+ pass

2502

+ n1 = len(s)

2503

+ del it

2504

+ n2 = len(s)

2505

+ self.assertGreaterEqual(n1, 0)

2506

+ self.assertLessEqual(n1, N)

2507

+ self.assertGreaterEqual(n2, 0)

2508

+ self.assertLessEqual(n2, n1)

2509

+

2510

2511

def test_main(verbose=None):

2512

support.run_unittest(TestWeakSet)

2513

diff -r 7085403daf43 Lib/test/test_zlib.py

2514

--- a/Lib/test/test_zlib.py

2515

+++ b/Lib/test/test_zlib.py

2516

@@ -66,24 +66,11 @@

2517

# Issue #10276 - check that inputs >=4GB are handled correctly.

2518

class ChecksumBigBufferTestCase(unittest.TestCase):

2519

2520

- def setUp(self):

2521

- with open(support.TESTFN, "wb+") as f:

2522

- f.seek(_4G)

2523

- f.write(b"asdf")

2524

- f.flush()

2525

- self.mapping = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)

2526

-

2527

- def tearDown(self):

2528

- self.mapping.close()

2529

- support.unlink(support.TESTFN)

2530

-

2531

- @unittest.skipUnless(mmap, "mmap() is not available.")

2532

- @unittest.skipUnless(sys.maxsize > _4G, "Can't run on a 32-bit system.")

2533

- @unittest.skipUnless(support.is_resource_enabled("largefile"),

2534

- "May use lots of disk space.")

2535

- def test_big_buffer(self):

2536

- self.assertEqual(zlib.crc32(self.mapping), 3058686908)

2537

- self.assertEqual(zlib.adler32(self.mapping), 82837919)

2538

+ @bigmemtest(size=_4G + 4, memuse=1, dry_run=False)

2539

+ def test_big_buffer(self, size):

2540

+ data = b"nyan" * (_1G + 1)

2541

+ self.assertEqual(zlib.crc32(data), 1044521549)

2542

+ self.assertEqual(zlib.adler32(data), 2256789997)

2543

2544

2545

class ExceptionTestCase(unittest.TestCase):

2546

diff -r 7085403daf43 Lib/weakref.py

2547

--- a/Lib/weakref.py

2548

+++ b/Lib/weakref.py

2549

@@ -78,7 +78,7 @@

2550

del self.data[key]

2551

2552

def __len__(self):

2553

- return sum(wr() is not None for wr in self.data.values())

2554

+ return len(self.data) - len(self._pending_removals)

2555

2556

def __contains__(self, key):

2557

try:

2558

@@ -290,7 +290,7 @@

2559

return self.data[ref(key)]

2560

2561

def __len__(self):

2562

- return len(self.data)

2563

+ return len(self.data) - len(self._pending_removals)

2564

2565

def __repr__(self):

2566

return "<WeakKeyDictionary at %s>" % id(self)

2567

diff -r 7085403daf43 Lib/xmlrpc/server.py

2568

--- a/Lib/xmlrpc/server.py

2569

+++ b/Lib/xmlrpc/server.py

2570

@@ -1,4 +1,4 @@

2571

-"""XML-RPC Servers.

2572

+r"""XML-RPC Servers.

2573

2574

This module can be used to create simple XML-RPC servers

2575

by creating a server and either installing functions, a

2576

diff -r 7085403daf43 Misc/ACKS

2577

--- a/Misc/ACKS

2578

+++ b/Misc/ACKS

2579

@@ -341,6 +341,7 @@

2580

Michael Gilfix

2581

Christoph Gohlke

2582

Tim Golden

2583

+Guilherme Gonçalves

2584

Chris Gonnerman

2585

David Goodger

2586

Hans de Graaff

2587

@@ -665,6 +666,7 @@

2588

Tim Northover

2589

Joe Norton

2590

Neal Norwitz

2591

+Mikhail Novikov

2592

Michal Nowikowski

2593

Steffen Daode Nurpmeso

2594

Nigel O'Brian

2595

diff -r 7085403daf43 Misc/NEWS

2596

--- a/Misc/NEWS

2597

+++ b/Misc/NEWS

2598

@@ -2,6 +2,57 @@

2599

Python News

2600

+++++++++++

2601

2602

+What's New in Python 3.2.4

2603

+==========================

2604

+

2605

+*Release date: XX-XX-XXXX*

2606

+

2607

+Core and Builtins

2608

+-----------------

2609

+

2610

+- Issue #1469629: Allow cycles through an object's __dict__ slot to be

2611

+ collected. (For example if ``x.__dict__ is x``).

2612

+

2613

+- Issue #14172: Fix reference leak when marshalling a buffer-like object

2614

+ (other than a bytes object).

2615

+

2616

+- Issue #13521: dict.setdefault() now does only one lookup for the given key,

2617

+ making it "atomic" for many purposes. Patch by Filip Gruszczyński.

2618

+

2619

+Library

2620

+-------

2621

+

2622

+- Issue #14195: An issue that caused weakref.WeakSet instances to incorrectly

2623

+ return True for a WeakSet instance 'a' in both 'a < a' and 'a > a' has been

2624

+ fixed.

2625

+

2626

+- Issue #14177: marshal.loads() now raises TypeError when given an unicode

2627

+ string. Patch by Guilherme Gonçalves.

2628

+

2629

+- Issue #14159: Fix the len() of weak containers (WeakSet, WeakKeyDictionary,

2630

+ WeakValueDictionary) to return a better approximation when some objects

2631

+ are dead or dying. Moreover, the implementation is now O(1) rather than

2632

+ O(n).

2633

+

2634

+- Issue #13125: Silence spurious test_lib2to3 output when in non-verbose mode.

2635

+ Patch by Mikhail Novikov.

2636

+

2637

+- Issue #13447: Add a test file to host regression tests for bugs in the

2638

+ scripts found in the Tools directory.

2639

+

2640

+- Issue #6884: Fix long-standing bugs with MANIFEST.in parsing in distutils

2641

+ on Windows.

2642

+

2643

+- Issue #8033: sqlite3: Fix 64-bit integer handling in user functions

2644

+ on 32-bit architectures. Initial patch by Philippe Devalkeneer.

2645

+

2646

+Extension Modules

2647

+-----------------

2648

+

2649

+- Issue #14212: The re module didn't retain a reference to buffers it was

2650

+ scanning, resulting in segfaults.

2651

+

2652

+

2653

What's New in Python 3.2.3 release candidate 1?

2654

===============================================

2655

2656

@@ -124,9 +175,6 @@

2657

Library

2658

-------

2659

2660

- on 32-bit architectures. Initial patch by Philippe Devalkeneer.

2661

-

2662

- HTMLParser is now able to handle slashes in the start tag.

2663

2664

- Issue #14001: CVE-2012-0845: xmlrpc: Fix an endless loop in

2665

diff -r 7085403daf43 Modules/_io/_iomodule.c

2666

--- a/Modules/_io/_iomodule.c

2667

+++ b/Modules/_io/_iomodule.c

2668

@@ -58,7 +58,7 @@

2669

"\n"

2670

"At the top of the I/O hierarchy is the abstract base class IOBase. It\n"

2671

"defines the basic interface to a stream. Note, however, that there is no\n"

2672

-"seperation between reading and writing to streams; implementations are\n"

2673

+"separation between reading and writing to streams; implementations are\n"

2674

"allowed to throw an IOError if they do not support a given operation.\n"

2675

"\n"

2676

"Extending IOBase is RawIOBase which deals simply with the reading and\n"

2677

diff -r 7085403daf43 Modules/_io/textio.c

2678

--- a/Modules/_io/textio.c

2679

+++ b/Modules/_io/textio.c

2680

@@ -627,7 +627,7 @@

2681

"enabled. With this enabled, on input, the lines endings '\\n', '\\r',\n"

2682

"or '\\r\\n' are translated to '\\n' before being returned to the\n"

2683

"caller. Conversely, on output, '\\n' is translated to the system\n"

2684

- "default line seperator, os.linesep. If newline is any other of its\n"

2685

+ "default line separator, os.linesep. If newline is any other of its\n"

2686

"legal values, that newline becomes the newline when the file is read\n"

2687

"and it is returned untranslated. On output, '\\n' is converted to the\n"

2688

"newline.\n"

2689

diff -r 7085403daf43 Modules/_sre.c

2690

--- a/Modules/_sre.c

2691

+++ b/Modules/_sre.c

2692

@@ -1664,7 +1664,7 @@

2693

}

2694

2695

static void*

2696

-getstring(PyObject* string, Py_ssize_t* p_length, int* p_charsize)

2697

+getstring(PyObject* string, Py_ssize_t* p_length, int* p_charsize, Py_buffer *view)

2698

{

2699

/* given a python object, return a data pointer, a length (in

2700

characters), and a character size. return NULL if the object

2701

@@ -1674,7 +1674,6 @@

2702

Py_ssize_t size, bytes;

2703

int charsize;

2704

void* ptr;

2705

- Py_buffer view;

2706

2707

/* Unicode objects do not support the buffer API. So, get the data

2708

directly instead. */

2709

@@ -1686,26 +1685,21 @@

2710

}

2711

2712

/* get pointer to string buffer */

2713

- view.len = -1;

2714

+ view->len = -1;

2715

buffer = Py_TYPE(string)->tp_as_buffer;

2716

if (!buffer || !buffer->bf_getbuffer ||

2717

- (*buffer->bf_getbuffer)(string, &view, PyBUF_SIMPLE) < 0) {

2718

+ (*buffer->bf_getbuffer)(string, view, PyBUF_SIMPLE) < 0) {

2719

PyErr_SetString(PyExc_TypeError, "expected string or buffer");

2720

return NULL;

2721

}

2722

2723

/* determine buffer size */

2724

- bytes = view.len;

2725

- ptr = view.buf;

2726

-

2727

- /* Release the buffer immediately --- possibly dangerous

2728

- but doing something else would require some re-factoring

2729

- */

2730

- PyBuffer_Release(&view);

2731

+ bytes = view->len;

2732

+ ptr = view->buf;

2733

2734

if (bytes < 0) {

2735

PyErr_SetString(PyExc_TypeError, "buffer has negative size");

2736

- return NULL;

2737

+ goto err;

2738

}

2739

2740

/* determine character size */

2741

@@ -1719,7 +1713,7 @@

2742

#endif

2743

else {

2744

PyErr_SetString(PyExc_TypeError, "buffer size mismatch");

2745

- return NULL;

2746

+ goto err;

2747

}

2748

2749

*p_length = size;

2750

@@ -1728,8 +1722,13 @@

2751

if (ptr == NULL) {

2752

PyErr_SetString(PyExc_ValueError,

2753

"Buffer is NULL");

2754

+ goto err;

2755

}

2756

return ptr;

2757

+ err:

2758

+ PyBuffer_Release(view);

2759

+ view->buf = NULL;

2760

+ return NULL;

2761

}

2762

2763

LOCAL(PyObject*)

2764

@@ -1747,20 +1746,21 @@

2765

state->lastmark = -1;

2766

state->lastindex = -1;

2767

2768

- ptr = getstring(string, &length, &charsize);

2769

+ state->buffer.buf = NULL;

2770

+ ptr = getstring(string, &length, &charsize, &state->buffer);

2771

if (!ptr)

2772

- return NULL;

2773

-

2774

- if (charsize == 1 && pattern->charsize > 1) {

2775

- PyErr_SetString(PyExc_TypeError,

2776

+ goto err;

2777

+

2778

+ if (charsize == 1 && pattern->charsize > 1) {

2779

+ PyErr_SetString(PyExc_TypeError,

2780

"can't use a string pattern on a bytes-like object");

2781

- return NULL;

2782

- }

2783

- if (charsize > 1 && pattern->charsize == 1) {

2784

- PyErr_SetString(PyExc_TypeError,

2785

+ goto err;

2786

+ }

2787

+ if (charsize > 1 && pattern->charsize == 1) {

2788

+ PyErr_SetString(PyExc_TypeError,

2789

"can't use a bytes pattern on a string-like object");

2790

- return NULL;

2791

- }

2792

+ goto err;

2793

+ }

2794

2795

/* adjust boundaries */

2796

if (start < 0)

2797

@@ -1797,11 +1797,17 @@

2798

state->lower = sre_lower;

2799

2800

return string;

2801

+ err:

2802

+ if (state->buffer.buf)

2803

+ PyBuffer_Release(&state->buffer);

2804

+ return NULL;

2805

}

2806

2807

LOCAL(void)

2808

state_fini(SRE_STATE* state)

2809

{

2810

+ if (state->buffer.buf)

2811

+ PyBuffer_Release(&state->buffer);

2812

Py_XDECREF(state->string);

2813

data_stack_dealloc(state);

2814

}

2815

@@ -1863,6 +1869,8 @@

2816

{

2817

if (self->weakreflist != NULL)

2818

PyObject_ClearWeakRefs((PyObject *) self);

2819

+ if (self->view.buf)

2820

+ PyBuffer_Release(&self->view);

2821

Py_XDECREF(self->pattern);

2822

Py_XDECREF(self->groupindex);

2823

Py_XDECREF(self->indexgroup);

2824

@@ -2297,6 +2305,7 @@

2825

Py_ssize_t i, b, e;

2826

int bint;

2827

int filter_is_callable;

2828

+ Py_buffer view;

2829

2830

if (PyCallable_Check(ptemplate)) {

2831

/* sub/subn takes either a function or a template */

2832

@@ -2306,7 +2315,8 @@

2833

} else {

2834

/* if not callable, check if it's a literal string */

2835

int literal;

2836

- ptr = getstring(ptemplate, &n, &bint);

2837

+ view.buf = NULL;

2838

+ ptr = getstring(ptemplate, &n, &bint, &view);

2839

b = bint;

2840

if (ptr) {

2841

if (b == 1) {

2842

@@ -2320,6 +2330,8 @@

2843

PyErr_Clear();

2844

literal = 0;

2845

}

2846

+ if (view.buf)

2847

+ PyBuffer_Release(&view);

2848

if (literal) {

2849

filter = ptemplate;

2850

Py_INCREF(filter);

2851

@@ -2661,6 +2673,7 @@

2852

Py_ssize_t groups = 0;

2853

PyObject* groupindex = NULL;

2854

PyObject* indexgroup = NULL;

2855

+

2856

if (!PyArg_ParseTuple(args, "OiO!|nOO", &pattern, &flags,

2857

&PyList_Type, &code, &groups,

2858

&groupindex, &indexgroup))

2859

@@ -2675,6 +2688,7 @@

2860

self->pattern = NULL;

2861

self->groupindex = NULL;

2862

self->indexgroup = NULL;

2863

+ self->view.buf = NULL;

2864

2865

self->codesize = n;

2866

2867

@@ -2694,15 +2708,15 @@

2868

return NULL;

2869

}

2870

2871

- if (pattern == Py_None)

2872

- self->charsize = -1;

2873

- else {

2874

- Py_ssize_t p_length;

2875

- if (!getstring(pattern, &p_length, &self->charsize)) {

2876

- Py_DECREF(self);

2877

- return NULL;

2878

- }

2879

- }

2880

+ if (pattern == Py_None)

2881

+ self->charsize = -1;

2882

+ else {

2883

+ Py_ssize_t p_length;

2884

+ if (!getstring(pattern, &p_length, &self->charsize, &self->view)) {

2885

+ Py_DECREF(self);

2886

+ return NULL;

2887

+ }

2888

+ }

2889

2890

Py_INCREF(pattern);

2891

self->pattern = pattern;

2892

diff -r 7085403daf43 Modules/sre.h

2893

--- a/Modules/sre.h

2894

+++ b/Modules/sre.h

2895

@@ -31,6 +31,7 @@

2896

int flags; /* flags used when compiling pattern source */

2897

PyObject *weakreflist; /* List of weak references */

2898

int charsize; /* pattern charsize (or -1) */

2899

+ Py_buffer view;

2900

/* pattern code */

2901

Py_ssize_t codesize;

2902

SRE_CODE code[1];

2903

@@ -80,6 +81,7 @@

2904

char* data_stack;

2905

size_t data_stack_size;

2906

size_t data_stack_base;

2907

+ Py_buffer buffer;

2908

/* current repeat context */

2909

SRE_REPEAT *repeat;

2910

/* hooks */

2911

diff -r 7085403daf43 Modules/timemodule.c

2912

--- a/Modules/timemodule.c

2913

+++ b/Modules/timemodule.c

2914

@@ -504,7 +504,7 @@

2915

fmt = PyBytes_AS_STRING(format);

2916

#endif

2917

2918

-#if defined(MS_WINDOWS)

2919

+#if defined(MS_WINDOWS) && !defined(HAVE_WCSFTIME)

2920

/* check that the format string contains only valid directives */

2921

for(outbuf = strchr(fmt, '%');

2922

outbuf != NULL;

2923

@@ -516,7 +516,8 @@

2924

!strchr("aAbBcdHIjmMpSUwWxXyYzZ%", outbuf[1]))

2925

{

2926

PyErr_SetString(PyExc_ValueError, "Invalid format string");

2927

- return 0;

2928

+ Py_DECREF(format);

2929

+ return NULL;

2930

}

2931

}

2932

#endif

2933

diff -r 7085403daf43 Objects/dictobject.c

2934

--- a/Objects/dictobject.c

2935

+++ b/Objects/dictobject.c

2936

@@ -510,27 +510,16 @@

2937

_PyObject_GC_UNTRACK(op);

2938

}

2939

2940

-

2941

/*

2942

-Internal routine to insert a new item into the table.

2943

-Used both by the internal resize routine and by the public insert routine.

2944

-Eats a reference to key and one to value.

2945

-Returns -1 if an error occurred, or 0 on success.

2946

+Internal routine to insert a new item into the table when you have entry object.

2947

+Used by insertdict.

2948

*/

2949

static int

2950

-insertdict(register PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject *value)

2951

+insertdict_by_entry(register PyDictObject *mp, PyObject *key, Py_hash_t hash,

2952

+ PyDictEntry *ep, PyObject *value)

2953

{

2954

PyObject *old_value;

2955

- register PyDictEntry *ep;

2956

- typedef PyDictEntry *(*lookupfunc)(PyDictObject *, PyObject *, Py_hash_t);

2957

2958

- assert(mp->ma_lookup != NULL);

2959

- ep = mp->ma_lookup(mp, key, hash);

2960

- if (ep == NULL) {

2961

- Py_DECREF(key);

2962

- Py_DECREF(value);

2963

- return -1;

2964

- }

2965

MAINTAIN_TRACKING(mp, key, value);

2966

if (ep->me_value != NULL) {

2967

old_value = ep->me_value;

2968

@@ -553,6 +542,28 @@

2969

return 0;

2970

}

2971

2972

+

2973

+/*

2974

+Internal routine to insert a new item into the table.

2975

+Used both by the internal resize routine and by the public insert routine.

2976

+Eats a reference to key and one to value.

2977

+Returns -1 if an error occurred, or 0 on success.

2978

+*/

2979

+static int

2980

+insertdict(register PyDictObject *mp, PyObject *key, Py_hash_t hash, PyObject *value)

2981

+{

2982

+ register PyDictEntry *ep;

2983

+

2984

+ assert(mp->ma_lookup != NULL);

2985

+ ep = mp->ma_lookup(mp, key, hash);

2986

+ if (ep == NULL) {

2987

+ Py_DECREF(key);

2988

+ Py_DECREF(value);

2989

+ return -1;

2990

+ }

2991

+ return insertdict_by_entry(mp, key, hash, ep, value);

2992

+}

2993

+

2994

/*

2995

Internal routine used by dictresize() to insert an item which is

2996

known to be absent from the dict. This routine also assumes that

2997

@@ -776,39 +787,26 @@

2998

return ep->me_value;

2999

}

3000

3001

-/* CAUTION: PyDict_SetItem() must guarantee that it won't resize the

3002

- * dictionary if it's merely replacing the value for an existing key.

3003

- * This means that it's safe to loop over a dictionary with PyDict_Next()

3004

- * and occasionally replace a value -- but you can't insert new keys or

3005

- * remove them.

3006

- */

3007

-int

3008

-PyDict_SetItem(register PyObject *op, PyObject *key, PyObject *value)

3009

+static int

3010

+dict_set_item_by_hash_or_entry(register PyObject *op, PyObject *key,

3011

+ Py_hash_t hash, PyDictEntry *ep, PyObject *value)

3012

{

3013

register PyDictObject *mp;

3014

- register Py_hash_t hash;

3015

register Py_ssize_t n_used;

3016

3017

- if (!PyDict_Check(op)) {

3018

- PyErr_BadInternalCall();

3019

- return -1;

3020

- }

3021

- assert(key);

3022

- assert(value);

3023

mp = (PyDictObject *)op;

3024

- if (!PyUnicode_CheckExact(key) ||

3025

- (hash = ((PyUnicodeObject *) key)->hash) == -1)

3026

- {

3027

- hash = PyObject_Hash(key);

3028

- if (hash == -1)

3029

- return -1;

3030

- }

3031

assert(mp->ma_fill <= mp->ma_mask); /* at least one empty slot */

3032

n_used = mp->ma_used;

3033

Py_INCREF(value);

3034

Py_INCREF(key);

3035

- if (insertdict(mp, key, hash, value) != 0)

3036

- return -1;

3037

+ if (ep == NULL) {

3038

+ if (insertdict(mp, key, hash, value) != 0)

3039

+ return -1;

3040

+ }

3041

+ else {

3042

+ if (insertdict_by_entry(mp, key, hash, ep, value) != 0)

3043

+ return -1;

3044

+ }

3045

/* If we added a key, we can safely resize. Otherwise just return!

3046

* If fill >= 2/3 size, adjust size. Normally, this doubles or

3047

* quaduples the size, but it's also possible for the dict to shrink

3048

@@ -828,6 +826,36 @@

3049

return dictresize(mp, (mp->ma_used > 50000 ? 2 : 4) * mp->ma_used);

3050

}

3051

3052

+/* CAUTION: PyDict_SetItem() must guarantee that it won't resize the

3053

+ * dictionary if it's merely replacing the value for an existing key.

3054

+ * This means that it's safe to loop over a dictionary with PyDict_Next()

3055

+ * and occasionally replace a value -- but you can't insert new keys or

3056

+ * remove them.

3057

+ */

3058

+int

3059

+PyDict_SetItem(register PyObject *op, PyObject *key, PyObject *value)

3060

+{

3061

+ register Py_hash_t hash;

3062

+

3063

+ if (!PyDict_Check(op)) {

3064

+ PyErr_BadInternalCall();

3065

+ return -1;

3066

+ }

3067

+ assert(key);

3068

+ assert(value);

3069

+ if (PyUnicode_CheckExact(key)) {

3070

+ hash = ((PyUnicodeObject *) key)->hash;

3071

+ if (hash == -1)

3072

+ hash = PyObject_Hash(key);

3073

+ }

3074

+ else {

3075

+ hash = PyObject_Hash(key);

3076

+ if (hash == -1)

3077

+ return -1;

3078

+ }

3079

+ return dict_set_item_by_hash_or_entry(op, key, hash, NULL, value);

3080

+}

3081

+

3082

int

3083

PyDict_DelItem(PyObject *op, PyObject *key)

3084

{

3085

@@ -1797,9 +1825,9 @@

3086

return NULL;

3087

val = ep->me_value;

3088

if (val == NULL) {

3089

- val = failobj;

3090

- if (PyDict_SetItem((PyObject*)mp, key, failobj))

3091

- val = NULL;

3092

+ if (dict_set_item_by_hash_or_entry((PyObject*)mp, key, hash, ep,

3093

+ failobj) == 0)

3094

+ val = failobj;

3095

}

3096

Py_XINCREF(val);

3097

return val;

3098

diff -r 7085403daf43 Objects/typeobject.c

3099

--- a/Objects/typeobject.c

3100

+++ b/Objects/typeobject.c

3101

@@ -830,8 +830,13 @@

3102

assert(base);

3103

}

3104

3105

- /* There's no need to clear the instance dict (if any);

3106

- the collector will call its tp_clear handler. */

3107

+ /* Clear the instance dict (if any), to break cycles involving only

3108

+ __dict__ slots (as in the case 'self.__dict__ is self'). */

3109

+ if (type->tp_dictoffset != base->tp_dictoffset) {

3110

+ PyObject **dictptr = _PyObject_GetDictPtr(self);

3111

+ if (dictptr && *dictptr)

3112

+ Py_CLEAR(*dictptr);

3113

+ }

3114

3115

if (baseclear)

3116

return baseclear(self);

3117

diff -r 7085403daf43 Python/marshal.c

3118

--- a/Python/marshal.c

3119

+++ b/Python/marshal.c

3120

@@ -411,11 +411,12 @@

3121

else if (PyObject_CheckBuffer(v)) {

3122

/* Write unknown buffer-style objects as a string */

3123

char *s;

3124

- PyBufferProcs *pb = v->ob_type->tp_as_buffer;

3125

Py_buffer view;

3126

- if ((*pb->bf_getbuffer)(v, &view, PyBUF_SIMPLE) != 0) {

3127

+ if (PyObject_GetBuffer(v, &view, PyBUF_SIMPLE) != 0) {

3128

w_byte(TYPE_UNKNOWN, p);

3129

+ p->depth--;

3130

p->error = WFERR_UNMARSHALLABLE;

3131

+ return;

3132

}

3133

w_byte(TYPE_STRING, p);

3134

n = view.len;

3135

@@ -427,8 +428,7 @@

3136

}

3137

w_long((long)n, p);

3138

w_string(s, (int)n, p);

3139

- if (pb->bf_releasebuffer != NULL)

3140

- (*pb->bf_releasebuffer)(v, &view);

3141

+ PyBuffer_Release(&view);

3142

}

3143

else {

3144

w_byte(TYPE_UNKNOWN, p);

3145

@@ -1383,7 +1383,7 @@

3146

char *s;

3147

Py_ssize_t n;

3148

PyObject* result;

3149

- if (!PyArg_ParseTuple(args, "s*:loads", &p))

3150

+ if (!PyArg_ParseTuple(args, "y*:loads", &p))

3151

return NULL;

3152

s = p.buf;

3153

n = p.len;

3154

@@ -1400,10 +1400,10 @@

3155

}

3156

3157

PyDoc_STRVAR(loads_doc,

3158

-"loads(string)\n\

3159

+"loads(bytes)\n\

3160

\n\

3161

-Convert the string to a value. If no valid value is found, raise\n\

3162

-EOFError, ValueError or TypeError. Extra characters in the string are\n\

3163

+Convert the bytes object to a value. If no valid value is found, raise\n\

3164

+EOFError, ValueError or TypeError. Extra characters in the input are\n\

3165

ignored.");

3166

3167

static PyMethodDef marshal_methods[] = {

3168

diff -r 7085403daf43 Tools/msi/msi.py

3169

--- a/Tools/msi/msi.py

3170

+++ b/Tools/msi/msi.py

3171

@@ -1021,6 +1021,7 @@

3172

lib.add_file("check_soundcard.vbs")

3173

lib.add_file("empty.vbs")

3174

lib.add_file("Sine-1000Hz-300ms.aif")

3175

+ lib.add_file("mime.types")

3176

lib.glob("*.uue")

3177

lib.glob("*.pem")

3178

lib.glob("*.pck")

3179

diff -r 7085403daf43 setup.py

3180

--- a/setup.py

3181

+++ b/setup.py

3182

@@ -197,7 +197,7 @@

3183

3184

# Python header files

3185

headers = [sysconfig.get_config_h_filename()]

3186

- headers += glob(os.path.join(sysconfig.get_path('platinclude'), "*.h"))

3187

+ headers += glob(os.path.join(sysconfig.get_path('include'), "*.h"))

3188

3189

for ext in self.extensions[:]:

3190

ext.sources = [ find_module_file(filename, moddirlist)