~ubuntu-branches/debian/sid/python-decorator/sid : revision 8

1

Metadata-Version: 1.0

2

Name: decorator

3

Version: 3.1.2

4

Summary: Better living through Python with decorators

5

Home-page: http://pypi.python.org/pypi/decorator

6

Author: Michele Simionato

7

Author-email: michele.simionato@gmail.com

8

License: BSD License

9

Description: </pre><?xml version="1.0" encoding="utf-8" ?>

10

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

11

12

<head>

13

14

15

<title>The decorator module</title>

16

17

18

19

.highlight { background: #f8f8f8; }

20

.highlight .c { color: #408080; font-style: italic } /* Comment */

21

.highlight .err { border: 1px solid #FF0000 } /* Error */

22

.highlight .k { color: #008000; font-weight: bold } /* Keyword */

23

.highlight .o { color: #666666 } /* Operator */

24

.highlight .cm { color: #408080; font-style: italic } /* Comment.Multiline */

25

.highlight .cp { color: #BC7A00 } /* Comment.Preproc */

26

.highlight .c1 { color: #408080; font-style: italic } /* Comment.Single */

27

.highlight .cs { color: #408080; font-style: italic } /* Comment.Special */

28

.highlight .gd { color: #A00000 } /* Generic.Deleted */

29

.highlight .ge { font-style: italic } /* Generic.Emph */

30

.highlight .gr { color: #FF0000 } /* Generic.Error */

31

.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */

32

.highlight .gi { color: #00A000 } /* Generic.Inserted */

33

.highlight .go { color: #808080 } /* Generic.Output */

34

.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */

35

.highlight .gs { font-weight: bold } /* Generic.Strong */

36

.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */

37

.highlight .gt { color: #0040D0 } /* Generic.Traceback */

38

.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */

39

.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */

40

.highlight .kp { color: #008000 } /* Keyword.Pseudo */

41

.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */

42

.highlight .kt { color: #008000; font-weight: bold } /* Keyword.Type */

43

.highlight .m { color: #666666 } /* Literal.Number */

44

.highlight .s { color: #BA2121 } /* Literal.String */

45

.highlight .na { color: #7D9029 } /* Name.Attribute */

46

.highlight .nb { color: #008000 } /* Name.Builtin */

47

.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */

48

.highlight .no { color: #880000 } /* Name.Constant */

49

.highlight .nd { color: #AA22FF } /* Name.Decorator */

50

.highlight .ni { color: #999999; font-weight: bold } /* Name.Entity */

51

.highlight .ne { color: #D2413A; font-weight: bold } /* Name.Exception */

52

.highlight .nf { color: #0000FF } /* Name.Function */

53

.highlight .nl { color: #A0A000 } /* Name.Label */

54

.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */

55

.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */

56

.highlight .nv { color: #19177C } /* Name.Variable */

57

.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */

58

.highlight .w { color: #bbbbbb } /* Text.Whitespace */

59

.highlight .mf { color: #666666 } /* Literal.Number.Float */

60

.highlight .mh { color: #666666 } /* Literal.Number.Hex */

61

.highlight .mi { color: #666666 } /* Literal.Number.Integer */

62

.highlight .mo { color: #666666 } /* Literal.Number.Oct */

63

.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */

64

.highlight .sc { color: #BA2121 } /* Literal.String.Char */

65

.highlight .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */

66

.highlight .s2 { color: #BA2121 } /* Literal.String.Double */

67

.highlight .se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */

68

.highlight .sh { color: #BA2121 } /* Literal.String.Heredoc */

69

.highlight .si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */

70

.highlight .sx { color: #008000 } /* Literal.String.Other */

71

.highlight .sr { color: #BB6688 } /* Literal.String.Regex */

72

.highlight .s1 { color: #BA2121 } /* Literal.String.Single */

73

.highlight .ss { color: #19177C } /* Literal.String.Symbol */

74

.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */

75

.highlight .vc { color: #19177C } /* Name.Variable.Class */

76

.highlight .vg { color: #19177C } /* Name.Variable.Global */

77

.highlight .vi { color: #19177C } /* Name.Variable.Instance */

78

.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */

79

80

</style>

81

</head>

82

<body>

83

84

<h1 class="title">The <tt class="docutils literal">decorator</tt> module</h1>

85

86

87

88

89

<tr><th class="docinfo-name">Author:</th>

90

<td>Michele Simionato</td></tr>

91

<tr class="field"><th class="docinfo-name">E-mail:</th><td class="field-body"><a class="reference external" href="mailto:michele.simionato@gmail.com">michele.simionato@gmail.com</a></td>

92

</tr>

93

<tr><th class="docinfo-name">Version:</th>

94

95

<tr class="field"><th class="docinfo-name">Requires:</th><td class="field-body">Python 2.4+</td>

96

</tr>

97

<tr class="field"><th class="docinfo-name">Download page:</th><td class="field-body"><a class="reference external" href="http://pypi.python.org/pypi/decorator/3.1.1">http://pypi.python.org/pypi/decorator/3.1.1</a></td>

98

</tr>

99

<tr class="field"><th class="docinfo-name">Installation:</th><td class="field-body"><tt class="docutils literal">easy_install decorator</tt></td>

100

</tr>

101

<tr class="field"><th class="docinfo-name">License:</th><td class="field-body">BSD license</td>

102

</tr>

103

</tbody>

104

</table>

105

106

Contents

107

108

<li><a class="reference internal" href="#introduction" id="id3">Introduction</a></li>

109

<li><a class="reference internal" href="#definitions" id="id4">Definitions</a></li>

110

<li><a class="reference internal" href="#statement-of-the-problem" id="id5">Statement of the problem</a></li>

111

<li><a class="reference internal" href="#the-solution" id="id6">The solution</a></li>

112

<li><a class="reference internal" href="#a-trace-decorator" id="id7">A <tt class="docutils literal">trace</tt> decorator</a></li>

113

<li><a class="reference internal" href="#decorator-is-a-decorator" id="id8"><tt class="docutils literal">decorator</tt> is a decorator</a></li>

114

<li><a class="reference internal" href="#blocking" id="id9"><tt class="docutils literal">blocking</tt></a></li>

115

<li><a class="reference internal" href="#async" id="id10"><tt class="docutils literal">async</tt></a></li>

116

<li><a class="reference internal" href="#the-functionmaker-class" id="id11">The <tt class="docutils literal">FunctionMaker</tt> class</a></li>

117

<li><a class="reference internal" href="#getting-the-source-code" id="id12">Getting the source code</a></li>

118

<li><a class="reference internal" href="#dealing-with-third-party-decorators" id="id13">Dealing with third party decorators</a></li>

119

<li><a class="reference internal" href="#caveats-and-limitations" id="id14">Caveats and limitations</a></li>

120

<li><a class="reference internal" href="#compatibility-notes" id="id15">Compatibility notes</a></li>

121

<li><a class="reference internal" href="#licence" id="id16">LICENCE</a></li>

122

</ul>

123

</div>

124

125

<h1><a class="toc-backref" href="#id3">Introduction</a></h1>

126

Python decorators are an interesting example of why syntactic sugar

127

matters. In principle, their introduction in Python 2.4 changed

128

nothing, since they do not provide any new functionality which was not

129

already present in the language. In practice, their introduction has

130

significantly changed the way we structure our programs in Python. I

131

believe the change is for the best, and that decorators are a great

132

idea since:

133

134

<li>decorators help reducing boilerplate code;</li>

135

<li>decorators help separation of concerns;</li>

136

<li>decorators enhance readability and maintenability;</li>

137

<li>decorators are explicit.</li>

138

</ul>

139

Still, as of now, writing custom decorators correctly requires

140

some experience and it is not as easy as it could be. For instance,

141

typical implementations of decorators involve nested functions, and

142

we all know that flat is better than nested.

143

The aim of the <tt class="docutils literal">decorator</tt> module it to simplify the usage of

144

decorators for the average programmer, and to popularize decorators by

145

showing various non-trivial examples. Of course, as all techniques,

146

decorators can be abused (I have seen that) and you should not try to

147

solve every problem with a decorator, just because you can.

148

You may find the source code for all the examples

149

discussed here in the <tt class="docutils literal">documentation.py</tt> file, which contains

150

this documentation in the form of doctests.

151

</div>

152

153

<h1><a class="toc-backref" href="#id4">Definitions</a></h1>

154

Technically speaking, any Python object which can be called with one argument

155

can be used as a decorator. However, this definition is somewhat too large

156

to be really useful. It is more convenient to split the generic class of

157

decorators in two subclasses:

158

159

<li>signature-preserving decorators, i.e. callable objects taking a

160

function as input and returning a function with the same

161

signature as output;</li>

162

<li>signature-changing decorators, i.e. decorators that change

163

the signature of their input function, or decorators returning

164

non-callable objects.</li>

165

</ul>

166

Signature-changing decorators have their use: for instance the

167

builtin classes <tt class="docutils literal">staticmethod</tt> and <tt class="docutils literal">classmethod</tt> are in this

168

group, since they take functions and return descriptor objects which

169

are not functions, nor callables.

170

However, signature-preserving decorators are more common and easier to

171

reason about; in particular signature-preserving decorators can be

172

composed together whereas other decorators in general cannot.

173

Writing signature-preserving decorators from scratch is not that

174

obvious, especially if one wants to define proper decorators that

175

can accept functions with any signature. A simple example will clarify

176

the issue.

177

</div>

178

179

<h1><a class="toc-backref" href="#id5">Statement of the problem</a></h1>

180

A very common use case for decorators is the memoization of functions.

181

A <tt class="docutils literal">memoize</tt> decorator works by caching

182

the result of the function call in a dictionary, so that the next time

183

the function is called with the same input parameters the result is retrieved

184

from the cache and not recomputed. There are many implementations of

185

<tt class="docutils literal">memoize</tt> in <a class="reference external" href="http://www.python.org/moin/PythonDecoratorLibrary">http://www.python.org/moin/PythonDecoratorLibrary</a>,

186

but they do not preserve the signature.

187

A simple implementation for Python 2.5 could be the following (notice

188

that in general it is impossible to memoize correctly something

189

that depends on non-hashable arguments):

190

191

<div class="highlight"><pre>def memoize25(func):

192

func.cache = {}

193

def memoize(*args, **kw):

194

if kw: # frozenset is used to ensure hashability

195

key = args, frozenset(kw.iteritems())

196

else:

197

key = args

198

cache = func.cache

199

if key in cache:

200

return cache[key]

201

else:

202

cache[key] = result = func(*args, **kw)

203

return result

204

return functools.update_wrapper(memoize, func)

205

</pre></div>

206

207

</div>

208

Here we used the <a class="reference external" href="http://www.python.org/doc/2.5.2/lib/module-functools.html">functools.update_wrapper</a> utility, which has

209

been added in Python 2.5 expressly to simplify the definition of decorators

210

(in older versions of Python you need to copy the function attributes

211

<tt class="docutils literal">__name__</tt>, <tt class="docutils literal">__doc__</tt>, <tt class="docutils literal">__module__</tt> and <tt class="docutils literal">__dict__</tt>

212

from the original function to the decorated function by hand).

213

The implementation above works in the sense that the decorator

214

can accept functions with generic signatures; unfortunately this

215

implementation does not define a signature-preserving decorator, since in

216

general <tt class="docutils literal">memoize25</tt> returns a function with a

217

different signature from the original function.

218

Consider for instance the following case:

219

220

<div class="highlight"><pre>>>> @memoize25

221

... def f1(x):

222

... time.sleep(1) # simulate some long computation

223

... return x

224

</pre></div>

225

226

</div>

227

Here the original function takes a single argument named <tt class="docutils literal">x</tt>,

228

but the decorated function takes any number of arguments and

229

keyword arguments:

230

231

<div class="highlight"><pre>>>> from inspect import getargspec

232

>>> print getargspec(f1)

233

([], 'args', 'kw', None)

234

</pre></div>

235

236

</div>

237

This means that introspection tools such as pydoc will give

238

wrong informations about the signature of <tt class="docutils literal">f1</tt>. This is pretty bad:

239

pydoc will tell you that the function accepts a generic signature

240

<tt class="docutils literal">*args</tt>, <tt class="docutils literal">**kw</tt>, but when you try to call the function with more than an

241

argument, you will get an error:

242

243

<div class="highlight"><pre>>>> f1(0, 1)

244

Traceback (most recent call last):

245

...

246

TypeError: f1() takes exactly 1 argument (2 given)

247

</pre></div>

248

249

</div>

250

</div>

251

252

<h1><a class="toc-backref" href="#id6">The solution</a></h1>

253

The solution is to provide a generic factory of generators, which

254

hides the complexity of making signature-preserving decorators

255

from the application programmer. The <tt class="docutils literal">decorator</tt> function in

256

the <tt class="docutils literal">decorator</tt> module is such a factory:

257

258

<div class="highlight"><pre>>>> from decorator import decorator

259

</pre></div>

260

261

</div>

262

<tt class="docutils literal">decorator</tt> takes two arguments, a caller function describing the

263

functionality of the decorator and a function to be decorated; it

264

returns the decorated function. The caller function must have

265

signature <tt class="docutils literal">(f, *args, **kw)</tt> and it must call the original function <tt class="docutils literal">f</tt>

266

with arguments <tt class="docutils literal">args</tt> and <tt class="docutils literal">kw</tt>, implementing the wanted capability,

267

i.e. memoization in this case:

268

269

<div class="highlight"><pre>def _memoize(func, *args, **kw):

270

if kw: # frozenset is used to ensure hashability

271

key = args, frozenset(kw.iteritems())

272

else:

273

key = args

274

cache = func.cache # attributed added by memoize

275

if key in cache:

276

return cache[key]

277

else:

278

cache[key] = result = func(*args, **kw)

279

return result

280

</pre></div>

281

282

</div>

283

At this point you can define your decorator as follows:

284

285

<div class="highlight"><pre>def memoize(f):

286

f.cache = {}

287

return decorator(_memoize, f)

288

</pre></div>

289

290

</div>

291

The difference with respect to the Python 2.5 approach, which is based

292

on nested functions, is that the decorator module forces you to lift

293

the inner function at the outer level (flat is better than nested).

294

Moreover, you are forced to pass explicitly the function you want to

295

decorate to the caller function.

296

Here is a test of usage:

297

298

<div class="highlight"><pre>>>> @memoize

299

... def heavy_computation():

300

... time.sleep(2)

301

... return "done"

302

303

>>> print heavy_computation() # the first time it will take 2 seconds

304

done

305

306

>>> print heavy_computation() # the second time it will be instantaneous

307

done

308

</pre></div>

309

310

</div>

311

The signature of <tt class="docutils literal">heavy_computation</tt> is the one you would expect:

312

313

<div class="highlight"><pre>>>> print getargspec(heavy_computation)

314

([], None, None, None)

315

</pre></div>

316

317

</div>

318

</div>

319

320

<h1><a class="toc-backref" href="#id7">A <tt class="docutils literal">trace</tt> decorator</a></h1>

321

As an additional example, here is how you can define a trivial

322

<tt class="docutils literal">trace</tt> decorator, which prints a message everytime the traced

323

function is called:

324

325

<div class="highlight"><pre>def _trace(f, *args, **kw):

326

print "calling %s with args %s, %s" % (f.__name__, args, kw)

327

return f(*args, **kw)

328

</pre></div>

329

330

</div>

331

332

<div class="highlight"><pre>def trace(f):

333

return decorator(_trace, f)

334

</pre></div>

335

336

</div>

337

Here is an example of usage:

338

339

<div class="highlight"><pre>>>> @trace

340

... def f1(x):

341

... pass

342

</pre></div>

343

344

</div>

345

It is immediate to verify that <tt class="docutils literal">f1</tt> works

346

347

<div class="highlight"><pre>>>> f1(0)

348

calling f1 with args (0,), {}

349

</pre></div>

350

351

</div>

352

and it that it has the correct signature:

353

354

<div class="highlight"><pre>>>> print getargspec(f1)

355

(['x'], None, None, None)

356

</pre></div>

357

358

</div>

359

The same decorator works with functions of any signature:

360

361

<div class="highlight"><pre>>>> @trace

362

... def f(x, y=1, z=2, *args, **kw):

363

... pass

364

365

>>> f(0, 3)

366

calling f with args (0, 3, 2), {}

367

368

>>> print getargspec(f)

369

(['x', 'y', 'z'], 'args', 'kw', (1, 2))

370

</pre></div>

371

372

</div>

373

That includes even functions with exotic signatures like the following:

374

375

<div class="highlight"><pre>>>> @trace

376

... def exotic_signature((x, y)=(1,2)): return x+y

377

378

>>> print getargspec(exotic_signature)

379

([['x', 'y']], None, None, ((1, 2),))

380

>>> exotic_signature()

381

calling exotic_signature with args ((1, 2),), {}

382

3

383

</pre></div>

384

385

</div>

386

Notice that the support for exotic signatures has been deprecated

387

in Python 2.6 and removed in Python 3.0.

388

</div>

389

390

<h1><a class="toc-backref" href="#id8"><tt class="docutils literal">decorator</tt> is a decorator</a></h1>

391

It may be annoying to write a caller function (like the <tt class="docutils literal">_trace</tt>

392

function above) and then a trivial wrapper

393

(<tt class="docutils literal">def trace(f): return decorator(_trace, f)</tt>) every time. For this reason,

394

the <tt class="docutils literal">decorator</tt> module provides an easy shortcut to convert

395

the caller function into a signature-preserving decorator:

396

you can just call <tt class="docutils literal">decorator</tt> with a single argument.

397

In our example you can just write <tt class="docutils literal">trace = decorator(_trace)</tt>.

398

The <tt class="docutils literal">decorator</tt> function can also be used as a signature-changing

399

decorator, just as <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt>.

400

However, <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt> return generic

401

objects which are not callable, while <tt class="docutils literal">decorator</tt> returns

402

signature-preserving decorators, i.e. functions of a single argument.

403

For instance, you can write directly

404

405

<div class="highlight"><pre>>>> @decorator

406

... def trace(f, *args, **kw):

407

... print "calling %s with args %s, %s" % (f.func_name, args, kw)

408

... return f(*args, **kw)

409

</pre></div>

410

411

</div>

412

and now <tt class="docutils literal">trace</tt> will be a decorator. Actually <tt class="docutils literal">trace</tt> is a <tt class="docutils literal">partial</tt>

413

object which can be used as a decorator:

414

415

<div class="highlight"><pre>>>> trace

416

417

</pre></div>

418

419

</div>

420

Here is an example of usage:

421

422

<div class="highlight"><pre>>>> @trace

423

... def func(): pass

424

425

>>> func()

426

calling func with args (), {}

427

</pre></div>

428

429

</div>

430

If you are using an old Python version (Python 2.4) the

431

<tt class="docutils literal">decorator</tt> module provides a poor man replacement for

432

<tt class="docutils literal">functools.partial</tt>.

433

There is also an easy way to create one-parameter factories of

434

decorators, based on the following

435

<tt class="docutils literal">decorator_factory</tt> utility:

436

437

<div class="highlight"><pre>def decorator_factory(decfac): # partial is functools.partial

438

"decorator_factory(decfac) returns a one-parameter family of decorators"

439

return partial(lambda df, param: decorator(partial(df, param)), decfac)

440

</pre></div>

441

442

</div>

443

<tt class="docutils literal">decorator_factory</tt> converts a function with signature

444

<tt class="docutils literal">(param, func, *args, **kw)</tt> into a one-parameter family

445

of decorators. Suppose for instance you want to generated different

446

tracing generator, with different tracing messages.

447

Here is how to do it:

448

449

<div class="highlight"><pre>>>> @decorator_factory

450

... def trace_factory(message_template, f, *args, **kw):

451

... name = f.func_name

452

... print message_template % locals()

453

... return f(*args, **kw)

454

</pre></div>

455

456

</div>

457

458

<div class="highlight"><pre>>>> trace_factory

459

<functools.partial object at 0x...>

460

461

>>> trace = trace_factory('Calling %(name)s with args %(args)s '

462

... 'and keywords %(kw)s')

463

</pre></div>

464

465

</div>

466

In this example the parameter (<tt class="docutils literal">message_template</tt>) is

467

just a string, but in general it can be a tuple, a dictionary, or

468

a generic object, so there is no real restriction (for instance,

469

if you want to define a two-parameter family of decorators just

470

use a tuple with two arguments as parameter).

471

Here is an example of usage:

472

473

<div class="highlight"><pre>>>> @trace

474

... def func(): pass

475

476

>>> func()

477

Calling func with args () and keywords {}

478

</pre></div>

479

480

</div>

481

</div>

482

483

<h1><a class="toc-backref" href="#id9"><tt class="docutils literal">blocking</tt></a></h1>

484

Sometimes one has to deal with blocking resources, such as <tt class="docutils literal">stdin</tt>, and

485

sometimes it is best to have back a "busy" message than to block everything.

486

This behavior can be implemented with a suitable family of decorators,

487

where the parameter is the busy message:

488

489

<div class="highlight"><pre>@decorator_factory

490

def blocking(not_avail, f, *args, **kw):

491

if not hasattr(f, "thread"): # no thread running

492

def set_result(): f.result = f(*args, **kw)

493

f.thread = threading.Thread(None, set_result)

494

f.thread.start()

495

return not_avail

496

elif f.thread.isAlive():

497

return not_avail

498

else: # the thread is ended, return the stored result

499

del f.thread

500

return f.result

501

</pre></div>

502

503

</div>

504

Functions decorated with <tt class="docutils literal">blocking</tt> will return a busy message if

505

the resource is unavailable, and the intended result if the resource is

506

available. For instance:

507

508

<div class="highlight"><pre>>>> @blocking("Please wait ...")

509

... def read_data():

510

... time.sleep(3) # simulate a blocking resource

511

... return "some data"

512

513

>>> print read_data() # data is not available yet

514

Please wait ...

515

516

>>> time.sleep(1)

517

>>> print read_data() # data is not available yet

518

Please wait ...

519

520

>>> time.sleep(1)

521

>>> print read_data() # data is not available yet

522

Please wait ...

523

524

>>> time.sleep(1.1) # after 3.1 seconds, data is available

525

>>> print read_data()

526

some data

527

</pre></div>

528

529

</div>

530

</div>

531

532

<h1><a class="toc-backref" href="#id10"><tt class="docutils literal">async</tt></a></h1>

533

We have just seen an examples of a simple decorator factory,

534

implemented as a function returning a decorator.

535

For more complex situations, it is more

536

convenient to implement decorator factories as classes returning

537

callable objects that can be used as signature-preserving

538

decorators. The suggested pattern to do that is to introduce

539

a helper method <tt class="docutils literal">call(self, func, *args, **kw)</tt> and to call

540

it in the <tt class="docutils literal">__call__(self, func)</tt> method.

541

As an example, here I show a decorator

542

which is able to convert a blocking function into an asynchronous

543

function. The function, when called,

544

is executed in a separate thread. Moreover, it is possible to set

545

three callbacks <tt class="docutils literal">on_success</tt>, <tt class="docutils literal">on_failure</tt> and <tt class="docutils literal">on_closing</tt>,

546

to specify how to manage the function call.

547

The implementation is the following:

548

549

<div class="highlight"><pre>def on_success(result): # default implementation

550

"Called on the result of the function"

551

return result

552

</pre></div>

553

554

</div>

555

556

<div class="highlight"><pre>def on_failure(exc_info): # default implementation

557

"Called if the function fails"

558

pass

559

</pre></div>

560

561

</div>

562

563

<div class="highlight"><pre>def on_closing(): # default implementation

564

"Called at the end, both in case of success and failure"

565

pass

566

</pre></div>

567

568

</div>

569

570

<div class="highlight"><pre>class Async(object):

571

"""

572

A decorator converting blocking functions into asynchronous

573

functions, by using threads or processes. Examples:

574

575

async_with_threads = Async(threading.Thread)

576

async_with_processes = Async(multiprocessing.Process)

577

"""

578

579

def __init__(self, threadfactory):

580

self.threadfactory = threadfactory

581

582

def __call__(self, func, on_success=on_success,

583

on_failure=on_failure, on_closing=on_closing):

584

# every decorated function has its own independent thread counter

585

func.counter = itertools.count(1)

586

func.on_success = on_success

587

func.on_failure = on_failure

588

func.on_closing = on_closing

589

return decorator(self.call, func)

590

591

def call(self, func, *args, **kw):

592

def func_wrapper():

593

try:

594

result = func(*args, **kw)

595

except:

596

func.on_failure(sys.exc_info())

597

else:

598

return func.on_success(result)

599

finally:

600

func.on_closing()

601

name = '%s-%s' % (func.__name__, func.counter.next())

602

thread = self.threadfactory(None, func_wrapper, name)

603

thread.start()

604

return thread

605

</pre></div>

606

607

</div>

608

The decorated function returns

609

the current execution thread, which can be stored and checked later, for

610

instance to verify that the thread <tt class="docutils literal">.isAlive()</tt>.

611

Here is an example of usage. Suppose one wants to write some data to

612

an external resource which can be accessed by a single user at once

613

(for instance a printer). Then the access to the writing function must

614

be locked. Here is a minimalistic example:

615

616

<div class="highlight"><pre>>>> async = Async(threading.Thread)

617

618

>>> datalist = [] # for simplicity the written data are stored into a list.

619

620

>>> @async

621

... def write(data):

622

... # append data to the datalist by locking

623

... with threading.Lock():

624

... time.sleep(1) # emulate some long running operation

625

... datalist.append(data)

626

... # other operations not requiring a lock here

627

</pre></div>

628

629

</div>

630

Each call to <tt class="docutils literal">write</tt> will create a new writer thread, but there will

631

be no synchronization problems since <tt class="docutils literal">write</tt> is locked.

632

633

>>> write("data1")

634

<Thread(write-1, started...)>

635

</pre>

636

637

>>> time.sleep(.1) # wait a bit, so we are sure data2 is written after data1

638

</pre>

639

640

>>> write("data2")

641

<Thread(write-2, started...)>

642

</pre>

643

644

>>> time.sleep(2) # wait for the writers to complete

645

</pre>

646

647

>>> print datalist

648

['data1', 'data2']

649

</pre>

650

</div>

651

652

<h1><a class="toc-backref" href="#id11">The <tt class="docutils literal">FunctionMaker</tt> class</a></h1>

653

You may wonder about how the functionality of the <tt class="docutils literal">decorator</tt> module

654

is implemented. The basic building block is

655

a <tt class="docutils literal">FunctionMaker</tt> class which is able to generate on the fly

656

functions with a given name and signature from a function template

657

passed as a string. Generally speaking, you should not need to

658

resort to <tt class="docutils literal">FunctionMaker</tt> when writing ordinary decorators, but

659

it is handy in some circumstances. You will see an example shortly, in

660

the implementation of a cool decorator utility (<tt class="docutils literal">decorator_apply</tt>).

661

<tt class="docutils literal">FunctionMaker</tt> provides a <tt class="docutils literal">.create</tt> classmethod which

662

takes as input the name, signature, and body of the function

663

we want to generate as well as the execution environment

664

were the function is generated by <tt class="docutils literal">exec</tt>. Here is an example:

665

666

<div class="highlight"><pre>>>> def f(*args, **kw): # a function with a generic signature

667

... print args, kw

668

669

>>> f1 = FunctionMaker.create('f1(a, b)', 'f(a, b)', dict(f=f))

670

>>> f1(1,2)

671

(1, 2) {}

672

</pre></div>

673

674

</div>

675

It is important to notice that the function body is interpolated

676

before being executed, so be careful with the <tt class="docutils literal">%</tt> sign!

677

<tt class="docutils literal">FunctionMaker.create</tt> also accepts keyword arguments and such

678

arguments are attached to the resulting function. This is useful

679

if you want to set some function attributes, for instance the

680

docstring <tt class="docutils literal">__doc__</tt>.

681

For debugging/introspection purposes it may be useful to see

682

the source code of the generated function; to do that, just

683

pass the flag <tt class="docutils literal">addsource=True</tt> and a <tt class="docutils literal">__source__</tt> attribute will

684

be added to the generated function:

685

686

<div class="highlight"><pre>>>> f1 = FunctionMaker.create(

687

... 'f1(a, b)', 'f(a, b)', dict(f=f), addsource=True)

688

>>> print f1.__source__

689

def f1(a, b):

690

f(a, b)

691

692

</pre></div>

693

694

</div>

695

<tt class="docutils literal">FunctionMaker.create</tt> can take as first argument a string,

696

as in the examples before, or a function. This is the most common

697

usage, since typically you want to decorate a pre-existing

698

function. A framework author may want to use directly <tt class="docutils literal">FunctionMaker.create</tt>

699

instead of <tt class="docutils literal">decorator</tt>, since it gives you direct access to the body

700

of the generated function. For instance, suppose you want to instrument

701

the <tt class="docutils literal">__init__</tt> methods of a set of classes, by preserving their

702

signature (such use case is not made up; this is done in SQAlchemy

703

and in other frameworks). When the first argument of <tt class="docutils literal">FunctionMaker.create</tt>

704

is a function, a <tt class="docutils literal">FunctionMaker</tt> object is instantiated internally,

705

with attributes <tt class="docutils literal">args</tt>, <tt class="docutils literal">varargs</tt>,

706

<tt class="docutils literal">keywords</tt> and <tt class="docutils literal">defaults</tt> which are the

707

the return values of the standard library function <tt class="docutils literal">inspect.getargspec</tt>.

708

For each argument in the <tt class="docutils literal">args</tt> (which is a list of strings containing

709

the names of the mandatory arguments) an attribute <tt class="docutils literal">arg0</tt>, <tt class="docutils literal">arg1</tt>,

710

..., <tt class="docutils literal">argN</tt> is also generated. Finally, there is a <tt class="docutils literal">signature</tt>

711

attribute, a string with the signature of the original function.

712

Notice that while I do not have plans

713

to change or remove the functionality provided in the

714

<tt class="docutils literal">FunctionMaker</tt> class, I do not guarantee that it will stay

715

unchanged forever. For instance, right now I am using the traditional

716

string interpolation syntax for function templates, but Python 2.6

717

and Python 3.0 provide a newer interpolation syntax and I may use

718

the new syntax in the future.

719

On the other hand, the functionality provided by

720

<tt class="docutils literal">decorator</tt> has been there from version 0.1 and it is guaranteed to

721

stay there forever.

722

</div>

723

724

<h1><a class="toc-backref" href="#id12">Getting the source code</a></h1>

725

Internally <tt class="docutils literal">FunctionMaker.create</tt> uses <tt class="docutils literal">exec</tt> to generate the

726

decorated function. Therefore

727

<tt class="docutils literal">inspect.getsource</tt> will not work for decorated functions. That

728

means that the usual '??' trick in IPython will give you the (right on

729

the spot) message <tt class="docutils literal">Dynamically generated function. No source code

730

available</tt>. In the past I have considered this acceptable, since

731

<tt class="docutils literal">inspect.getsource</tt> does not really work even with regular

732

decorators. In that case <tt class="docutils literal">inspect.getsource</tt> gives you the wrapper

733

source code which is probably not what you want:

734

735

<div class="highlight"><pre>def identity_dec(func):

736

def wrapper(*args, **kw):

737

return func(*args, **kw)

738

return wrapper

739

</pre></div>

740

741

</div>

742

743

<div class="highlight"><pre>@identity_dec

744

def example(): pass

745

746

>>> print inspect.getsource(example)

747

def wrapper(*args, **kw):

748

return func(*args, **kw)

749

750

</pre></div>

751

752

</div>

753

(see bug report <a class="reference external" href="http://bugs.python.org/issue1764286">1764286</a> for an explanation of what is happening).

754

Unfortunately the bug is still there, even in Python 2.6 and 3.0.

755

There is however a workaround. The decorator module adds an

756

attribute <tt class="docutils literal">.undecorated</tt> to the decorated function, containing

757

a reference to the original function. The easy way to get

758

the source code is to call <tt class="docutils literal">inspect.getsource</tt> on the

759

undecorated function:

760

761

<div class="highlight"><pre>>>> print inspect.getsource(factorial.undecorated)

762

@tail_recursive

763

def factorial(n, acc=1):

764

"The good old factorial"

765

if n == 0: return acc

766

return factorial(n-1, n*acc)

767

768

</pre></div>

769

770

</div>

771

</div>

772

773

<h1><a class="toc-backref" href="#id13">Dealing with third party decorators</a></h1>

774

Sometimes you find on the net some cool decorator that you would

775

like to include in your code. However, more often than not the cool

776

decorator is not signature-preserving. Therefore you may want an easy way to

777

upgrade third party decorators to signature-preserving decorators without

778

having to rewrite them in terms of <tt class="docutils literal">decorator</tt>. You can use a

779

<tt class="docutils literal">FunctionMaker</tt> to implement that functionality as follows:

780

781

<div class="highlight"><pre>def decorator_apply(dec, func):

782

"""

783

Decorate a function by preserving the signature even if dec

784

is not a signature-preserving decorator.

785

"""

786

return FunctionMaker.create(

787

func, 'return decorated(%(signature)s)',

788

dict(decorated=dec(func)), undecorated=func)

789

</pre></div>

790

791

</div>

792

<tt class="docutils literal">decorator_apply</tt> sets the attribute <tt class="docutils literal">.undecorated</tt> of the generated

793

function to the original function, so that you can get the right

794

source code.

795

Notice that I am not providing this functionality in the <tt class="docutils literal">decorator</tt>

796

module directly since I think it is best to rewrite the decorator rather

797

than adding an additional level of indirection. However, practicality

798

beats purity, so you can add <tt class="docutils literal">decorator_apply</tt> to your toolbox and

799

use it if you need to.

800

In order to give an example of usage of <tt class="docutils literal">decorator_apply</tt>, I will show a

801

pretty slick decorator that converts a tail-recursive function in an iterative

802

function. I have shamelessly stolen the basic idea from Kay Schluehr's recipe

803

in the Python Cookbook,

804

<a class="reference external" href="http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496691">http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496691</a>.

805

806

<div class="highlight"><pre>class TailRecursive(object):

807

"""

808

tail_recursive decorator based on Kay Schluehr's recipe

809

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496691

810

with improvements by me and George Sakkis.

811

"""

812

813

def __init__(self, func):

814

self.func = func

815

self.firstcall = True

816

self.CONTINUE = object() # sentinel

817

818

def __call__(self, *args, **kwd):

819

CONTINUE = self.CONTINUE

820

if self.firstcall:

821

func = self.func

822

self.firstcall = False

823

try:

824

while True:

825

result = func(*args, **kwd)

826

if result is CONTINUE: # update arguments

827

args, kwd = self.argskwd

828

else: # last call

829

return result

830

finally:

831

self.firstcall = True

832

else: # return the arguments of the tail call

833

self.argskwd = args, kwd

834

return CONTINUE

835

</pre></div>

836

837

</div>

838

Here the decorator is implemented as a class returning callable

839

objects.

840

841

<div class="highlight"><pre>def tail_recursive(func):

842

return decorator_apply(TailRecursive, func)

843

</pre></div>

844

845

</div>

846

Here is how you apply the upgraded decorator to the good old factorial:

847

848

<div class="highlight"><pre>@tail_recursive

849

def factorial(n, acc=1):

850

"The good old factorial"

851

if n == 0: return acc

852

return factorial(n-1, n*acc)

853

</pre></div>

854

855

</div>

856

857

<div class="highlight"><pre>>>> print factorial(4)

858

24

859

</pre></div>

860

861

</div>

862

This decorator is pretty impressive, and should give you some food for

863

your mind ;) Notice that there is no recursion limit now, and you can

864

easily compute <tt class="docutils literal">factorial(1001)</tt> or larger without filling the stack

865

frame. Notice also that the decorator will not work on functions which

866

are not tail recursive, such as the following

867

868

<div class="highlight"><pre>def fact(n): # this is not tail-recursive

869

if n == 0: return 1

870

return n * fact(n-1)

871

</pre></div>

872

873

</div>

874

(reminder: a function is tail recursive if it either returns a value without

875

making a recursive call, or returns directly the result of a recursive

876

call).

877

</div>

878

879

<h1><a class="toc-backref" href="#id14">Caveats and limitations</a></h1>

880

The first thing you should be aware of, it the fact that decorators

881

have a performance penalty.

882

The worse case is shown by the following example:

883

884

$ cat performance.sh

885

python -m timeit -s "

886

from decorator import decorator

887

888

@decorator

889

def do_nothing(func, *args, **kw):

890

return func(*args, **kw)

891

892

@do_nothing

893

def f():

894

pass

895

" "f()"

896

897

python -m timeit -s "

898

def f():

899

pass

900

" "f()"

901

</pre>

902

On my MacBook, using the <tt class="docutils literal">do_nothing</tt> decorator instead of the

903

plain function is more than three times slower:

904

905

$ bash performance.sh

906

1000000 loops, best of 3: 0.995 usec per loop

907

1000000 loops, best of 3: 0.273 usec per loop

908

</pre>

909

It should be noted that a real life function would probably do

910

something more useful than <tt class="docutils literal">f</tt> here, and therefore in real life the

911

performance penalty could be completely negligible. As always, the

912

only way to know if there is

913

a penalty in your specific use case is to measure it.

914

You should be aware that decorators will make your tracebacks

915

longer and more difficult to understand. Consider this example:

916

917

<div class="highlight"><pre>>>> @trace

918

... def f():

919

... 1/0

920

</pre></div>

921

922

</div>

923

Calling <tt class="docutils literal">f()</tt> will give you a <tt class="docutils literal">ZeroDivisionError</tt>, but since the

924

function is decorated the traceback will be longer:

925

926

<div class="highlight"><pre>>>> f()

927

Traceback (most recent call last):

928

...

929

File "<string>", line 2, in f

930

File "<doctest __main__[18]>", line 4, in trace

931

return f(*args, **kw)

932

File "<doctest __main__[47]>", line 3, in f

933

1/0

934

ZeroDivisionError: integer division or modulo by zero

935

</pre></div>

936

937

</div>

938

You see here the inner call to the decorator <tt class="docutils literal">trace</tt>, which calls

939

<tt class="docutils literal">f(*args, **kw)</tt>, and a reference to <tt class="docutils literal">File "<string>", line 2, in f</tt>.

940

This latter reference is due to the fact that internally the decorator

941

module uses <tt class="docutils literal">exec</tt> to generate the decorated function. Notice that

942

<tt class="docutils literal">exec</tt> is not responsibile for the performance penalty, since is the

943

called only once at function decoration time, and not every time

944

the decorated function is called.

945

At present, there is no clean way to avoid <tt class="docutils literal">exec</tt>. A clean solution

946

would require to change the CPython implementation of functions and

947

add an hook to make it possible to change their signature directly.

948

That could happen in future versions of Python (see PEP <a class="reference external" href="http://www.python.org/dev/peps/pep-0362">362</a>) and

949

then the decorator module would become obsolete. However, at present,

950

even in Python 3.1 it is impossible to change the function signature

951

directly, therefore the <tt class="docutils literal">decorator</tt> module is still useful.

952

Actually, this is one of the main reasons why I keep maintaining

953

the module and releasing new versions.

954

In the present implementation, decorators generated by <tt class="docutils literal">decorator</tt>

955

can only be used on user-defined Python functions or methods, not on generic

956

callable objects, nor on built-in functions, due to limitations of the

957

<tt class="docutils literal">inspect</tt> module in the standard library. Moreover, notice

958

that you can decorate a method, but only before if becomes a bound or unbound

959

method, i.e. inside the class.

960

Here is an example of valid decoration:

961

962

<div class="highlight"><pre>>>> class C(object):

963

... @trace

964

... def meth(self):

965

... pass

966

</pre></div>

967

968

</div>

969

Here is an example of invalid decoration, when the decorator in

970

called too late:

971

972

<div class="highlight"><pre>>>> class C(object):

973

... def meth(self):

974

... pass

975

...

976

>>> trace(C.meth)

977

Traceback (most recent call last):

978

...

979

TypeError: You are decorating a non function: <unbound method C.meth>

980

</pre></div>

981

982

</div>

983

The solution is to extract the inner function from the unbound method:

984

985

<div class="highlight"><pre>>>> trace(C.meth.im_func)

986

987

</pre></div>

988

989

</div>

990

There is a restriction on the names of the arguments: for instance,

991

if try to call an argument <tt class="docutils literal">_call_</tt> or <tt class="docutils literal">_func_</tt>

992

you will get a <tt class="docutils literal">NameError</tt>:

993

994

<div class="highlight"><pre>>>> @trace

995

... def f(_func_): print f

996

...

997

Traceback (most recent call last):

998

...

999

NameError: _func_ is overridden in

1000

def f(_func_):

1001

return _call_(_func_, _func_)

1002

</pre></div>

1003

1004

</div>

1005

Finally, the implementation is such that the decorated function contains

1006

a copy of the original function dictionary

1007

(<tt class="docutils literal">vars(decorated_f) is not vars(f)</tt>):

1008

1009

<div class="highlight"><pre>>>> def f(): pass # the original function

1010

>>> f.attr1 = "something" # setting an attribute

1011

>>> f.attr2 = "something else" # setting another attribute

1012

1013

>>> traced_f = trace(f) # the decorated function

1014

1015

>>> traced_f.attr1

1016

'something'

1017

>>> traced_f.attr2 = "something different" # setting attr

1018

>>> f.attr2 # the original attribute did not change

1019

'something else'

1020

</pre></div>

1021

1022

</div>

1023

</div>

1024

1025

<h1><a class="toc-backref" href="#id15">Compatibility notes</a></h1>

1026

Version 3.0 is a complete rewrite of the original implementation.

1027

It is mostly compatible with the past, a part for a few differences.

1028

First of all, the utilites <tt class="docutils literal">get_info</tt> and <tt class="docutils literal">new_wrapper</tt>, available

1029

in the 2.X versions, have been deprecated and they will be removed

1030

in the future. For the moment, using them raises a <tt class="docutils literal">DeprecationWarning</tt>.

1031

Incidentally, the functionality has been implemented through a

1032

decorator which makes a good example for this documentation:

1033

1034

<div class="highlight"><pre>@decorator

1035

def deprecated(func, *args, **kw):

1036

"A decorator for deprecated functions"

1037

warnings.warn(

1038

('Calling the deprecated function %r\n'

1039

'Downgrade to decorator 2.3 if you want to use this functionality')

1040

% func.__name__, DeprecationWarning, stacklevel=3)

1041

return func(*args, **kw)

1042

</pre></div>

1043

1044

</div>

1045

<tt class="docutils literal">get_info</tt> has been removed since it was little used and since it had

1046

to be changed anyway to work with Python 3.0; <tt class="docutils literal">new_wrapper</tt> has been

1047

removed since it was useless: its major use case (converting

1048

signature changing decorators to signature preserving decorators)

1049

has been subsumed by <tt class="docutils literal">decorator_apply</tt>

1050

and the other use case can be managed with the <tt class="docutils literal">FunctionMaker</tt>.

1051

Finally <tt class="docutils literal">decorator</tt> cannot be used as a class decorator and the

1052

<a class="reference external" href="http://www.phyast.pitt.edu/~micheles/python/documentation.html#class-decorators-and-decorator-factories">functionality introduced in version 2.3</a> has been removed. That

1053

means that in order to define decorator factories with classes you

1054

need to define the <tt class="docutils literal">__call__</tt> method explicitly (no magic anymore).

1055

Since there is

1056

an easy way to define decorator factories by using <tt class="docutils literal">decorator_factory</tt>,

1057

there is less need to use classes to implement decorator factories.

1058

All these changes should not cause any trouble, since they were

1059

all rarely used features. Should you have any trouble, you can always

1060

downgrade to the 2.3 version.

1061

The examples shown here have been tested with Python 2.5. Python 2.4

1062

is also supported - of course the examples requiring the <tt class="docutils literal">with</tt>

1063

statement will not work there. Python 2.6 works fine, but if you

1064

run the examples here in the interactive interpreter

1065

you will notice a few differences since

1066

<tt class="docutils literal">getargspec</tt> returns an <tt class="docutils literal">ArgSpec</tt> namedtuple instead of a regular

1067

tuple. That means that running the file

1068

<tt class="docutils literal">documentation.py</tt> under Python 2.5 will a few errors, but

1069

they are not serious. Python 3.0 is kind of supported too.

1070

Simply run the script <tt class="docutils literal">2to3</tt> on the module

1071

<tt class="docutils literal">decorator.py</tt> and you will get a version of the code running

1072

with Python 3.0 (at least, I did some simple checks and it seemed

1073

to work). However there is no support for <a class="reference external" href="http://www.python.org/dev/peps/pep-3107/">function annotations</a> yet

1074

since it seems premature at this moment (most people are

1075

still using Python 2.5).

1076

</div>

1077

1078

<h1><a class="toc-backref" href="#id16">LICENCE</a></h1>

1079

Redistribution and use in source and binary forms, with or without

1080

modification, are permitted provided that the following conditions are

1081

met:

1082

1083

1084

1085

1086

Redistributions of source code must retain the above copyright

1087

notice, this list of conditions and the following disclaimer.

1088

Redistributions in bytecode form must reproduce the above copyright

1089

notice, this list of conditions and the following disclaimer in

1090

the documentation and/or other materials provided with the

1091

distribution.

1092

1093

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS

1094

"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT

1095

LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR

1096

A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT

1097

HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,

1098

INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,

1099

BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS

1100

OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND

1101

ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR

1102

TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE

1103

USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH

1104

DAMAGE.

1105

</pre>

1106

If you use this software and you are happy with it, consider sending me a

1107

note, just to gratify my ego. On the other hand, if you use this software and

1108

you are unhappy with it, send me a patch!

1109

</div>

1110

</div>

1111

1112

1113

<a class="reference external" href="documentation.rst">View document source</a>.

1114

Generated on: 2009-08-25 12:37 UTC.

1115

Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.

1116

1117

</div>

1118

</body>

1119

</html>

1120

<pre>

1121

Keywords: decorators generic utility

1122

Platform: All

1123

Classifier: Development Status :: 5 - Production/Stable

1124

Classifier: Intended Audience :: Developers

1125

Classifier: License :: OSI Approved :: BSD License

1126

Classifier: Natural Language :: English

1127

Classifier: Operating System :: OS Independent

1128

Classifier: Programming Language :: Python

1129

Classifier: Topic :: Software Development :: Libraries

1130

Classifier: Topic :: Utilities