~ubuntu-branches/debian/sid/python-decorator/sid : revision 8

3

4

<head>

5

6

6

7

<title>The decorator module</title>

8

9

73

</head>

74

<body>

75

76

<h1 class="title">The <tt class="docutils literal">decorator</tt> module</h1>

76

<h1 class="title">The <tt class="docutils literal">decorator</tt> module</h1>

77

78

79

83

<tr class="field"><th class="docinfo-name">E-mail:</th><td class="field-body"><a class="reference external" href="mailto:michele.simionato@gmail.com">michele.simionato@gmail.com</a></td>

84

</tr>

85

<tr><th class="docinfo-name">Version:</th>

86

86

87

<tr class="field"><th class="docinfo-name">Requires:</th><td class="field-body">Python 2.4+</td>

88

</tr>

89

<tr class="field"><th class="docinfo-name">Download page:</th><td class="field-body"><a class="reference external" href="http://pypi.python.org/pypi/decorator/3.1.1">http://pypi.python.org/pypi/decorator/3.1.1</a></td>

89

<tr class="field"><th class="docinfo-name">Download page:</th><td class="field-body"><a class="reference external" href="http://pypi.python.org/pypi/decorator/3.2.0">http://pypi.python.org/pypi/decorator/3.2.0</a></td>

90

</tr>

91

<tr class="field"><th class="docinfo-name">Installation:</th><td class="field-body"><tt class="docutils literal">easy_install decorator</tt></td>

91

<tr class="field"><th class="docinfo-name">Installation:</th><td class="field-body"><tt class="docutils literal">easy_install decorator</tt></td>

92

</tr>

93

<tr class="field"><th class="docinfo-name">License:</th><td class="field-body">BSD license</td>

94

</tr>

101

<li><a class="reference internal" href="#definitions" id="id4">Definitions</a></li>

102

<li><a class="reference internal" href="#statement-of-the-problem" id="id5">Statement of the problem</a></li>

103

<li><a class="reference internal" href="#the-solution" id="id6">The solution</a></li>

104

<li><a class="reference internal" href="#a-trace-decorator" id="id7">A <tt class="docutils literal">trace</tt> decorator</a></li>

105

<li><a class="reference internal" href="#decorator-is-a-decorator" id="id8"><tt class="docutils literal">decorator</tt> is a decorator</a></li>

106

<li><a class="reference internal" href="#blocking" id="id9"><tt class="docutils literal">blocking</tt></a></li>

107

<li><a class="reference internal" href="#async" id="id10"><tt class="docutils literal">async</tt></a></li>

108

<li><a class="reference internal" href="#the-functionmaker-class" id="id11">The <tt class="docutils literal">FunctionMaker</tt> class</a></li>

104

<li><a class="reference internal" href="#a-trace-decorator" id="id7">A <tt class="docutils literal">trace</tt> decorator</a></li>

105

<li><a class="reference internal" href="#decorator-is-a-decorator" id="id8"><tt class="docutils literal">decorator</tt> is a decorator</a></li>

106

<li><a class="reference internal" href="#blocking" id="id9"><tt class="docutils literal">blocking</tt></a></li>

107

<li><a class="reference internal" href="#async" id="id10"><tt class="docutils literal">async</tt></a></li>

108

<li><a class="reference internal" href="#the-functionmaker-class" id="id11">The <tt class="docutils literal">FunctionMaker</tt> class</a></li>

109

<li><a class="reference internal" href="#getting-the-source-code" id="id12">Getting the source code</a></li>

110

<li><a class="reference internal" href="#dealing-with-third-party-decorators" id="id13">Dealing with third party decorators</a></li>

111

<li><a class="reference internal" href="#caveats-and-limitations" id="id14">Caveats and limitations</a></li>

132

some experience and it is not as easy as it could be. For instance,

133

typical implementations of decorators involve nested functions, and

134

we all know that flat is better than nested.

135

The aim of the <tt class="docutils literal">decorator</tt> module it to simplify the usage of

135

The aim of the <tt class="docutils literal">decorator</tt> module it to simplify the usage of

136

decorators for the average programmer, and to popularize decorators by

137

showing various non-trivial examples. Of course, as all techniques,

138

decorators can be abused (I have seen that) and you should not try to

139

solve every problem with a decorator, just because you can.

140

You may find the source code for all the examples

141

discussed here in the <tt class="docutils literal">documentation.py</tt> file, which contains

141

discussed here in the <tt class="docutils literal">documentation.py</tt> file, which contains

142

this documentation in the form of doctests.

143

</div>

144

156

non-callable objects.</li>

157

</ul>

158

Signature-changing decorators have their use: for instance the

159

builtin classes <tt class="docutils literal">staticmethod</tt> and <tt class="docutils literal">classmethod</tt> are in this

159

builtin classes <tt class="docutils literal">staticmethod</tt> and <tt class="docutils literal">classmethod</tt> are in this

160

group, since they take functions and return descriptor objects which

161

are not functions, nor callables.

162

However, signature-preserving decorators are more common and easier to

170

171

<h1><a class="toc-backref" href="#id5">Statement of the problem</a></h1>

172

A very common use case for decorators is the memoization of functions.

173

A <tt class="docutils literal">memoize</tt> decorator works by caching

173

A <tt class="docutils literal">memoize</tt> decorator works by caching

174

the result of the function call in a dictionary, so that the next time

175

the function is called with the same input parameters the result is retrieved

176

from the cache and not recomputed. There are many implementations of

177

<tt class="docutils literal">memoize</tt> in <a class="reference external" href="http://www.python.org/moin/PythonDecoratorLibrary">http://www.python.org/moin/PythonDecoratorLibrary</a>,

177

<tt class="docutils literal">memoize</tt> in <a class="reference external" href="http://www.python.org/moin/PythonDecoratorLibrary">http://www.python.org/moin/PythonDecoratorLibrary</a>,

178

but they do not preserve the signature.

179

A simple implementation for Python 2.5 could be the following (notice

179

A simple implementation could be the following (notice

180

that in general it is impossible to memoize correctly something

181

that depends on non-hashable arguments):

182

183

<div class="highlight"><pre>def memoize25(func):

183

<div class="highlight"><pre>def memoize_uw(func):

184

func.cache = {}

185

def memoize(*args, **kw):

186

if kw: # frozenset is used to ensure hashability

200

Here we used the <a class="reference external" href="http://www.python.org/doc/2.5.2/lib/module-functools.html">functools.update_wrapper</a> utility, which has

201

been added in Python 2.5 expressly to simplify the definition of decorators

202

(in older versions of Python you need to copy the function attributes

203

<tt class="docutils literal">__name__</tt>, <tt class="docutils literal">__doc__</tt>, <tt class="docutils literal">__module__</tt> and <tt class="docutils literal">__dict__</tt>

203

<tt class="docutils literal">__name__</tt>, <tt class="docutils literal">__doc__</tt>, <tt class="docutils literal">__module__</tt> and <tt class="docutils literal">__dict__</tt>

204

from the original function to the decorated function by hand).

205

The implementation above works in the sense that the decorator

206

can accept functions with generic signatures; unfortunately this

207

implementation does not define a signature-preserving decorator, since in

208

general <tt class="docutils literal">memoize25</tt> returns a function with a

208

general <tt class="docutils literal">memoize_uw</tt> returns a function with a

209

different signature from the original function.

210

Consider for instance the following case:

211

212

<div class="highlight"><pre>>>> @memoize25

212

<div class="highlight"><pre>>>> @memoize_uw

213

... def f1(x):

214

... time.sleep(1) # simulate some long computation

215

... return x

216

</pre></div>

217

218

</div>

219

Here the original function takes a single argument named <tt class="docutils literal">x</tt>,

219

Here the original function takes a single argument named <tt class="docutils literal">x</tt>,

220

but the decorated function takes any number of arguments and

221

keyword arguments:

222

223

<div class="highlight"><pre>>>> from inspect import getargspec

224

>>> print getargspec(f1)

225

([], 'args', 'kw', None)

224

>>> print getargspec(f1) # I am using Python 2.6+ here

225

ArgSpec(args=[], varargs='args', keywords='kw', defaults=None)

226

</pre></div>

227

228

</div>

229

This means that introspection tools such as pydoc will give

230

wrong informations about the signature of <tt class="docutils literal">f1</tt>. This is pretty bad:

230

wrong informations about the signature of <tt class="docutils literal">f1</tt>. This is pretty bad:

231

pydoc will tell you that the function accepts a generic signature

232

<tt class="docutils literal">*args</tt>, <tt class="docutils literal">**kw</tt>, but when you try to call the function with more than an

232

<tt class="docutils literal">*args</tt>, <tt class="docutils literal">**kw</tt>, but when you try to call the function with more than an

233

argument, you will get an error:

234

235

<div class="highlight"><pre>>>> f1(0, 1)

244

<h1><a class="toc-backref" href="#id6">The solution</a></h1>

245

The solution is to provide a generic factory of generators, which

246

hides the complexity of making signature-preserving decorators

247

from the application programmer. The <tt class="docutils literal">decorator</tt> function in

248

the <tt class="docutils literal">decorator</tt> module is such a factory:

247

from the application programmer. The <tt class="docutils literal">decorator</tt> function in

248

the <tt class="docutils literal">decorator</tt> module is such a factory:

249

250

<div class="highlight"><pre>>>> from decorator import decorator

251

</pre></div>

252

253

</div>

254

<tt class="docutils literal">decorator</tt> takes two arguments, a caller function describing the

254

<tt class="docutils literal">decorator</tt> takes two arguments, a caller function describing the

255

functionality of the decorator and a function to be decorated; it

256

returns the decorated function. The caller function must have

257

signature <tt class="docutils literal">(f, *args, **kw)</tt> and it must call the original function <tt class="docutils literal">f</tt>

258

with arguments <tt class="docutils literal">args</tt> and <tt class="docutils literal">kw</tt>, implementing the wanted capability,

257

signature <tt class="docutils literal">(f, *args, **kw)</tt> and it must call the original function <tt class="docutils literal">f</tt>

258

with arguments <tt class="docutils literal">args</tt> and <tt class="docutils literal">kw</tt>, implementing the wanted capability,

259

i.e. memoization in this case:

260

261

<div class="highlight"><pre>def _memoize(func, *args, **kw):

280

</pre></div>

281

282

</div>

283

The difference with respect to the Python 2.5 approach, which is based

283

The difference with respect to the <tt class="docutils literal">memoize_uw</tt> approach, which is based

284

on nested functions, is that the decorator module forces you to lift

285

the inner function at the outer level (flat is better than nested).

286

Moreover, you are forced to pass explicitly the function you want to

300

</pre></div>

301

302

</div>

303

The signature of <tt class="docutils literal">heavy_computation</tt> is the one you would expect:

303

The signature of <tt class="docutils literal">heavy_computation</tt> is the one you would expect:

304

305

<div class="highlight"><pre>>>> print getargspec(heavy_computation)

306

([], None, None, None)

306

ArgSpec(args=[], varargs=None, keywords=None, defaults=None)

307

</pre></div>

308

309

</div>

310

</div>

311

312

<h1><a class="toc-backref" href="#id7">A <tt class="docutils literal">trace</tt> decorator</a></h1>

312

<h1><a class="toc-backref" href="#id7">A <tt class="docutils literal">trace</tt> decorator</a></h1>

313

As an additional example, here is how you can define a trivial

314

<tt class="docutils literal">trace</tt> decorator, which prints a message everytime the traced

314

<tt class="docutils literal">trace</tt> decorator, which prints a message everytime the traced

315

function is called:

316

317

<div class="highlight"><pre>def _trace(f, *args, **kw):

334

</pre></div>

335

336

</div>

337

It is immediate to verify that <tt class="docutils literal">f1</tt> works

337

It is immediate to verify that <tt class="docutils literal">f1</tt> works

338

339

<div class="highlight"><pre>>>> f1(0)

340

calling f1 with args (0,), {}

344

and it that it has the correct signature:

345

346

<div class="highlight"><pre>>>> print getargspec(f1)

347

(['x'], None, None, None)

347

ArgSpec(args=['x'], varargs=None, keywords=None, defaults=None)

348

</pre></div>

349

350

</div>

358

calling f with args (0, 3, 2), {}

359

360

>>> print getargspec(f)

361

(['x', 'y', 'z'], 'args', 'kw', (1, 2))

361

ArgSpec(args=['x', 'y', 'z'], varargs='args', keywords='kw', defaults=(1, 2))

362

</pre></div>

363

364

</div>

368

... def exotic_signature((x, y)=(1,2)): return x+y

369

370

>>> print getargspec(exotic_signature)

371

([['x', 'y']], None, None, ((1, 2),))

371

ArgSpec(args=[['x', 'y']], varargs=None, keywords=None, defaults=((1, 2),))

372

>>> exotic_signature()

373

calling exotic_signature with args ((1, 2),), {}

374

3

379

in Python 2.6 and removed in Python 3.0.

380

</div>

381

382

<h1><a class="toc-backref" href="#id8"><tt class="docutils literal">decorator</tt> is a decorator</a></h1>

383

It may be annoying to write a caller function (like the <tt class="docutils literal">_trace</tt>

382

<h1><a class="toc-backref" href="#id8"><tt class="docutils literal">decorator</tt> is a decorator</a></h1>

383

It may be annoying to write a caller function (like the <tt class="docutils literal">_trace</tt>

384

function above) and then a trivial wrapper

385

(<tt class="docutils literal">def trace(f): return decorator(_trace, f)</tt>) every time. For this reason,

386

the <tt class="docutils literal">decorator</tt> module provides an easy shortcut to convert

385

(<tt class="docutils literal">def trace(f): return decorator(_trace, f)</tt>) every time. For this reason,

386

the <tt class="docutils literal">decorator</tt> module provides an easy shortcut to convert

387

the caller function into a signature-preserving decorator:

388

you can just call <tt class="docutils literal">decorator</tt> with a single argument.

389

In our example you can just write <tt class="docutils literal">trace = decorator(_trace)</tt>.

390

The <tt class="docutils literal">decorator</tt> function can also be used as a signature-changing

391

decorator, just as <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt>.

392

However, <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt> return generic

393

objects which are not callable, while <tt class="docutils literal">decorator</tt> returns

388

you can just call <tt class="docutils literal">decorator</tt> with a single argument.

389

In our example you can just write <tt class="docutils literal">trace = decorator(_trace)</tt>.

390

The <tt class="docutils literal">decorator</tt> function can also be used as a signature-changing

391

decorator, just as <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt>.

392

However, <tt class="docutils literal">classmethod</tt> and <tt class="docutils literal">staticmethod</tt> return generic

393

objects which are not callable, while <tt class="docutils literal">decorator</tt> returns

394

signature-preserving decorators, i.e. functions of a single argument.

395

For instance, you can write directly

396

401

</pre></div>

402

403

</div>

404

and now <tt class="docutils literal">trace</tt> will be a decorator. Actually <tt class="docutils literal">trace</tt> is a <tt class="docutils literal">partial</tt>

404

and now <tt class="docutils literal">trace</tt> will be a decorator. Actually <tt class="docutils literal">trace</tt> is a <tt class="docutils literal">partial</tt>

405

object which can be used as a decorator:

406

407

<div class="highlight"><pre>>>> trace

420

421

</div>

422

If you are using an old Python version (Python 2.4) the

423

<tt class="docutils literal">decorator</tt> module provides a poor man replacement for

424

<tt class="docutils literal">functools.partial</tt>.

425

There is also an easy way to create one-parameter factories of

426

decorators, based on the following

427

<tt class="docutils literal">decorator_factory</tt> utility:

428

429

<div class="highlight"><pre>def decorator_factory(decfac): # partial is functools.partial

430

"decorator_factory(decfac) returns a one-parameter family of decorators"

431

return partial(lambda df, param: decorator(partial(df, param)), decfac)

432

</pre></div>

433

434

</div>

435

<tt class="docutils literal">decorator_factory</tt> converts a function with signature

436

<tt class="docutils literal">(param, func, *args, **kw)</tt> into a one-parameter family

437

of decorators. Suppose for instance you want to generated different

438

tracing generator, with different tracing messages.

439

Here is how to do it:

440

441

<div class="highlight"><pre>>>> @decorator_factory

442

... def trace_factory(message_template, f, *args, **kw):

443

... name = f.func_name

444

... print message_template % locals()

445

... return f(*args, **kw)

446

</pre></div>

447

448

</div>

449

450

<div class="highlight"><pre>>>> trace_factory

451

<functools.partial object at 0x...>

452

453

>>> trace = trace_factory('Calling %(name)s with args %(args)s '

454

... 'and keywords %(kw)s')

455

</pre></div>

456

457

</div>

458

In this example the parameter (<tt class="docutils literal">message_template</tt>) is

459

just a string, but in general it can be a tuple, a dictionary, or

460

a generic object, so there is no real restriction (for instance,

461

if you want to define a two-parameter family of decorators just

462

use a tuple with two arguments as parameter).

463

Here is an example of usage:

464

465

<div class="highlight"><pre>>>> @trace

466

... def func(): pass

467

468

>>> func()

469

Calling func with args () and keywords {}

470

</pre></div>

471

472

</div>

423

<tt class="docutils literal">decorator</tt> module provides a poor man replacement for

424

<tt class="docutils literal">functools.partial</tt>.

473

425

</div>

474

426

475

<h1><a class="toc-backref" href="#id9"><tt class="docutils literal">blocking</tt></a></h1>

476

Sometimes one has to deal with blocking resources, such as <tt class="docutils literal">stdin</tt>, and

427

<h1><a class="toc-backref" href="#id9"><tt class="docutils literal">blocking</tt></a></h1>

428

Sometimes one has to deal with blocking resources, such as <tt class="docutils literal">stdin</tt>, and

477

429

sometimes it is best to have back a "busy" message than to block everything.

478

430

This behavior can be implemented with a suitable family of decorators,

479

431

where the parameter is the busy message:

480

432

481

<div class="highlight"><pre>@decorator_factory

482

def blocking(not_avail, f, *args, **kw):

483

if not hasattr(f, "thread"): # no thread running

484

def set_result(): f.result = f(*args, **kw)

485

f.thread = threading.Thread(None, set_result)

486

f.thread.start()

487

return not_avail

488

elif f.thread.isAlive():

489

return not_avail

490

else: # the thread is ended, return the stored result

491

del f.thread

492

return f.result

433

<div class="highlight"><pre>def blocking(not_avail):

434

def blocking(f, *args, **kw):

435

if not hasattr(f, "thread"): # no thread running

436

def set_result(): f.result = f(*args, **kw)

437

f.thread = threading.Thread(None, set_result)

438

f.thread.start()

439

return not_avail

440

elif f.thread.isAlive():

441

return not_avail

442

else: # the thread is ended, return the stored result

443

del f.thread

444

return f.result

445

return decorator(blocking)

493

446

</pre></div>

494

447

495

448

</div>

496

Functions decorated with <tt class="docutils literal">blocking</tt> will return a busy message if

449

Functions decorated with <tt class="docutils literal">blocking</tt> will return a busy message if

497

450

the resource is unavailable, and the intended result if the resource is

498

451

available. For instance:

499

452

521

474

</div>

522

475

</div>

523

476

524

<h1><a class="toc-backref" href="#id10"><tt class="docutils literal">async</tt></a></h1>

477

<h1><a class="toc-backref" href="#id10"><tt class="docutils literal">async</tt></a></h1>

525

478

We have just seen an examples of a simple decorator factory,

526

479

implemented as a function returning a decorator.

527

480

For more complex situations, it is more

528

481

convenient to implement decorator factories as classes returning

529

482

callable objects that can be used as signature-preserving

530

483

decorators. The suggested pattern to do that is to introduce

531

a helper method <tt class="docutils literal">call(self, func, *args, **kw)</tt> and to call

532

it in the <tt class="docutils literal">__call__(self, func)</tt> method.

484

a helper method <tt class="docutils literal">call(self, func, *args, **kw)</tt> and to call

485

it in the <tt class="docutils literal">__call__(self, func)</tt> method.

533

486

As an example, here I show a decorator

534

487

which is able to convert a blocking function into an asynchronous

535

488

function. The function, when called,

536

489

is executed in a separate thread. Moreover, it is possible to set

537

three callbacks <tt class="docutils literal">on_success</tt>, <tt class="docutils literal">on_failure</tt> and <tt class="docutils literal">on_closing</tt>,

490

three callbacks <tt class="docutils literal">on_success</tt>, <tt class="docutils literal">on_failure</tt> and <tt class="docutils literal">on_closing</tt>,

538

491

to specify how to manage the function call.

539

492

The implementation is the following:

540

493

599

552

</div>

600

553

The decorated function returns

601

554

the current execution thread, which can be stored and checked later, for

602

instance to verify that the thread <tt class="docutils literal">.isAlive()</tt>.

555

instance to verify that the thread <tt class="docutils literal">.isAlive()</tt>.

603

556

Here is an example of usage. Suppose one wants to write some data to

604

557

an external resource which can be accessed by a single user at once

605

558

(for instance a printer). Then the access to the writing function must

619

572

</pre></div>

620

573

621

574

</div>

622

Each call to <tt class="docutils literal">write</tt> will create a new writer thread, but there will

623

be no synchronization problems since <tt class="docutils literal">write</tt> is locked.

624

625

>>> write("data1")

626

<Thread(write-1, started...)>

627

</pre>

628

629

>>> time.sleep(.1) # wait a bit, so we are sure data2 is written after data1

630

</pre>

631

632

>>> write("data2")

633

<Thread(write-2, started...)>

634

</pre>

635

636

>>> time.sleep(2) # wait for the writers to complete

637

</pre>

638

639

>>> print datalist

640

['data1', 'data2']

641

</pre>

575

Each call to <tt class="docutils literal">write</tt> will create a new writer thread, but there will

576

be no synchronization problems since <tt class="docutils literal">write</tt> is locked.

577

578

<div class="highlight"><pre>>>> write("data1")

579

<Thread(write-1, started...)>

580

581

>>> time.sleep(.1) # wait a bit, so we are sure data2 is written after data1

582

583

>>> write("data2")

584

<Thread(write-2, started...)>

585

586

>>> time.sleep(2) # wait for the writers to complete

587

588

>>> print datalist

589

['data1', 'data2']

590

</pre></div>

591

592

</div>

642

593

</div>

643

594

644

<h1><a class="toc-backref" href="#id11">The <tt class="docutils literal">FunctionMaker</tt> class</a></h1>

645

You may wonder about how the functionality of the <tt class="docutils literal">decorator</tt> module

595

<h1><a class="toc-backref" href="#id11">The <tt class="docutils literal">FunctionMaker</tt> class</a></h1>

596

You may wonder about how the functionality of the <tt class="docutils literal">decorator</tt> module

646

597

is implemented. The basic building block is

647

a <tt class="docutils literal">FunctionMaker</tt> class which is able to generate on the fly

598

a <tt class="docutils literal">FunctionMaker</tt> class which is able to generate on the fly

648

599

functions with a given name and signature from a function template

649

600

passed as a string. Generally speaking, you should not need to

650

resort to <tt class="docutils literal">FunctionMaker</tt> when writing ordinary decorators, but

601

resort to <tt class="docutils literal">FunctionMaker</tt> when writing ordinary decorators, but

651

602

it is handy in some circumstances. You will see an example shortly, in

652

the implementation of a cool decorator utility (<tt class="docutils literal">decorator_apply</tt>).

653

<tt class="docutils literal">FunctionMaker</tt> provides a <tt class="docutils literal">.create</tt> classmethod which

603

the implementation of a cool decorator utility (<tt class="docutils literal">decorator_apply</tt>).

604

<tt class="docutils literal">FunctionMaker</tt> provides a <tt class="docutils literal">.create</tt> classmethod which

654

605

takes as input the name, signature, and body of the function

655

606

we want to generate as well as the execution environment

656

were the function is generated by <tt class="docutils literal">exec</tt>. Here is an example:

607

were the function is generated by <tt class="docutils literal">exec</tt>. Here is an example:

657

608

658

609

<div class="highlight"><pre>>>> def f(*args, **kw): # a function with a generic signature

659

610

... print args, kw

665

616

666

617

</div>

667

618

It is important to notice that the function body is interpolated

668

before being executed, so be careful with the <tt class="docutils literal">%</tt> sign!

669

<tt class="docutils literal">FunctionMaker.create</tt> also accepts keyword arguments and such

619

before being executed, so be careful with the <tt class="docutils literal">%</tt> sign!

620

<tt class="docutils literal">FunctionMaker.create</tt> also accepts keyword arguments and such

670

621

arguments are attached to the resulting function. This is useful

671

622

if you want to set some function attributes, for instance the

672

docstring <tt class="docutils literal">__doc__</tt>.

623

docstring <tt class="docutils literal">__doc__</tt>.

673

624

For debugging/introspection purposes it may be useful to see

674

625

the source code of the generated function; to do that, just

675

pass the flag <tt class="docutils literal">addsource=True</tt> and a <tt class="docutils literal">__source__</tt> attribute will

626

pass the flag <tt class="docutils literal">addsource=True</tt> and a <tt class="docutils literal">__source__</tt> attribute will

676

627

be added to the generated function:

677

628

678

629

<div class="highlight"><pre>>>> f1 = FunctionMaker.create(

684

635

</pre></div>

685

636

686

637

</div>

687

<tt class="docutils literal">FunctionMaker.create</tt> can take as first argument a string,

638

<tt class="docutils literal">FunctionMaker.create</tt> can take as first argument a string,

688

639

as in the examples before, or a function. This is the most common

689

640

usage, since typically you want to decorate a pre-existing

690

function. A framework author may want to use directly <tt class="docutils literal">FunctionMaker.create</tt>

691

instead of <tt class="docutils literal">decorator</tt>, since it gives you direct access to the body

641

function. A framework author may want to use directly <tt class="docutils literal">FunctionMaker.create</tt>

642

instead of <tt class="docutils literal">decorator</tt>, since it gives you direct access to the body

692

643

of the generated function. For instance, suppose you want to instrument

693

the <tt class="docutils literal">__init__</tt> methods of a set of classes, by preserving their

644

the <tt class="docutils literal">__init__</tt> methods of a set of classes, by preserving their

694

645

signature (such use case is not made up; this is done in SQAlchemy

695

and in other frameworks). When the first argument of <tt class="docutils literal">FunctionMaker.create</tt>

696

is a function, a <tt class="docutils literal">FunctionMaker</tt> object is instantiated internally,

697

with attributes <tt class="docutils literal">args</tt>, <tt class="docutils literal">varargs</tt>,

698

<tt class="docutils literal">keywords</tt> and <tt class="docutils literal">defaults</tt> which are the

699

the return values of the standard library function <tt class="docutils literal">inspect.getargspec</tt>.

700

For each argument in the <tt class="docutils literal">args</tt> (which is a list of strings containing

701

the names of the mandatory arguments) an attribute <tt class="docutils literal">arg0</tt>, <tt class="docutils literal">arg1</tt>,

702

..., <tt class="docutils literal">argN</tt> is also generated. Finally, there is a <tt class="docutils literal">signature</tt>

646

and in other frameworks). When the first argument of <tt class="docutils literal">FunctionMaker.create</tt>

647

is a function, a <tt class="docutils literal">FunctionMaker</tt> object is instantiated internally,

648

with attributes <tt class="docutils literal">args</tt>, <tt class="docutils literal">varargs</tt>,

649

<tt class="docutils literal">keywords</tt> and <tt class="docutils literal">defaults</tt> which are the

650

the return values of the standard library function <tt class="docutils literal">inspect.getargspec</tt>.

651

For each argument in the <tt class="docutils literal">args</tt> (which is a list of strings containing

652

the names of the mandatory arguments) an attribute <tt class="docutils literal">arg0</tt>, <tt class="docutils literal">arg1</tt>,

653

..., <tt class="docutils literal">argN</tt> is also generated. Finally, there is a <tt class="docutils literal">signature</tt>

703

654

attribute, a string with the signature of the original function.

704

655

Notice that while I do not have plans

705

656

to change or remove the functionality provided in the

706

<tt class="docutils literal">FunctionMaker</tt> class, I do not guarantee that it will stay

657

<tt class="docutils literal">FunctionMaker</tt> class, I do not guarantee that it will stay

707

658

unchanged forever. For instance, right now I am using the traditional

708

659

string interpolation syntax for function templates, but Python 2.6

709

660

and Python 3.0 provide a newer interpolation syntax and I may use

710

661

the new syntax in the future.

711

662

On the other hand, the functionality provided by

712

<tt class="docutils literal">decorator</tt> has been there from version 0.1 and it is guaranteed to

663

<tt class="docutils literal">decorator</tt> has been there from version 0.1 and it is guaranteed to

713

664

stay there forever.

714

665

</div>

715

666

716

667

<h1><a class="toc-backref" href="#id12">Getting the source code</a></h1>

717

Internally <tt class="docutils literal">FunctionMaker.create</tt> uses <tt class="docutils literal">exec</tt> to generate the

668

Internally <tt class="docutils literal">FunctionMaker.create</tt> uses <tt class="docutils literal">exec</tt> to generate the

718

669

decorated function. Therefore

719

<tt class="docutils literal">inspect.getsource</tt> will not work for decorated functions. That

670

<tt class="docutils literal">inspect.getsource</tt> will not work for decorated functions. That

720

671

means that the usual '??' trick in IPython will give you the (right on

721

the spot) message <tt class="docutils literal">Dynamically generated function. No source code

722

available</tt>. In the past I have considered this acceptable, since

723

<tt class="docutils literal">inspect.getsource</tt> does not really work even with regular

724

decorators. In that case <tt class="docutils literal">inspect.getsource</tt> gives you the wrapper

672

the spot) message <tt class="docutils literal">Dynamically generated function. No source code

673

available</tt>. In the past I have considered this acceptable, since

674

<tt class="docutils literal">inspect.getsource</tt> does not really work even with regular

675

decorators. In that case <tt class="docutils literal">inspect.getsource</tt> gives you the wrapper

725

676

source code which is probably not what you want:

726

677

727

678

<div class="highlight"><pre>def identity_dec(func):

745

696

(see bug report <a class="reference external" href="http://bugs.python.org/issue1764286">1764286</a> for an explanation of what is happening).

746

697

Unfortunately the bug is still there, even in Python 2.6 and 3.0.

747

698

There is however a workaround. The decorator module adds an

748

attribute <tt class="docutils literal">.undecorated</tt> to the decorated function, containing

699

attribute <tt class="docutils literal">.undecorated</tt> to the decorated function, containing

749

700

a reference to the original function. The easy way to get

750

the source code is to call <tt class="docutils literal">inspect.getsource</tt> on the

701

the source code is to call <tt class="docutils literal">inspect.getsource</tt> on the

751

702

undecorated function:

752

703

753

704

<div class="highlight"><pre>>>> print inspect.getsource(factorial.undecorated)

767

718

like to include in your code. However, more often than not the cool

768

719

decorator is not signature-preserving. Therefore you may want an easy way to

769

720

upgrade third party decorators to signature-preserving decorators without

770

having to rewrite them in terms of <tt class="docutils literal">decorator</tt>. You can use a

771

<tt class="docutils literal">FunctionMaker</tt> to implement that functionality as follows:

721

having to rewrite them in terms of <tt class="docutils literal">decorator</tt>. You can use a

722

<tt class="docutils literal">FunctionMaker</tt> to implement that functionality as follows:

772

723

773

724

<div class="highlight"><pre>def decorator_apply(dec, func):

774

725

"""

781

732

</pre></div>

782

733

783

734

</div>

784

<tt class="docutils literal">decorator_apply</tt> sets the attribute <tt class="docutils literal">.undecorated</tt> of the generated

735

<tt class="docutils literal">decorator_apply</tt> sets the attribute <tt class="docutils literal">.undecorated</tt> of the generated

785

736

function to the original function, so that you can get the right

786

737

source code.

787

Notice that I am not providing this functionality in the <tt class="docutils literal">decorator</tt>

738

Notice that I am not providing this functionality in the <tt class="docutils literal">decorator</tt>

788

739

module directly since I think it is best to rewrite the decorator rather

789

740

than adding an additional level of indirection. However, practicality

790

beats purity, so you can add <tt class="docutils literal">decorator_apply</tt> to your toolbox and

741

beats purity, so you can add <tt class="docutils literal">decorator_apply</tt> to your toolbox and

791

742

use it if you need to.

792

In order to give an example of usage of <tt class="docutils literal">decorator_apply</tt>, I will show a

743

In order to give an example of usage of <tt class="docutils literal">decorator_apply</tt>, I will show a

793

744

pretty slick decorator that converts a tail-recursive function in an iterative

794

745

function. I have shamelessly stolen the basic idea from Kay Schluehr's recipe

795

746

in the Python Cookbook,

853

804

</div>

854

805

This decorator is pretty impressive, and should give you some food for

855

806

your mind ;) Notice that there is no recursion limit now, and you can

856

easily compute <tt class="docutils literal">factorial(1001)</tt> or larger without filling the stack

807

easily compute <tt class="docutils literal">factorial(1001)</tt> or larger without filling the stack

857

808

frame. Notice also that the decorator will not work on functions which

858

809

are not tail recursive, such as the following

859

810

891

842

pass

892

843

" "f()"

893

844

</pre>

894

On my MacBook, using the <tt class="docutils literal">do_nothing</tt> decorator instead of the

845

On my MacBook, using the <tt class="docutils literal">do_nothing</tt> decorator instead of the

895

846

plain function is more than three times slower:

896

847

897

848

$ bash performance.sh

899

850

1000000 loops, best of 3: 0.273 usec per loop

900

851

</pre>

901

852

It should be noted that a real life function would probably do

902

something more useful than <tt class="docutils literal">f</tt> here, and therefore in real life the

853

something more useful than <tt class="docutils literal">f</tt> here, and therefore in real life the

903

854

performance penalty could be completely negligible. As always, the

904

855

only way to know if there is

905

856

a penalty in your specific use case is to measure it.

912

863

</pre></div>

913

864

914

865

</div>

915

Calling <tt class="docutils literal">f()</tt> will give you a <tt class="docutils literal">ZeroDivisionError</tt>, but since the

866

Calling <tt class="docutils literal">f()</tt> will give you a <tt class="docutils literal">ZeroDivisionError</tt>, but since the

916

867

function is decorated the traceback will be longer:

917

868

918

869

<div class="highlight"><pre>>>> f()

927

878

</pre></div>

928

879

929

880

</div>

930

You see here the inner call to the decorator <tt class="docutils literal">trace</tt>, which calls

931

<tt class="docutils literal">f(*args, **kw)</tt>, and a reference to <tt class="docutils literal">File "<string>", line 2, in f</tt>.

881

You see here the inner call to the decorator <tt class="docutils literal">trace</tt>, which calls

882

<tt class="docutils literal">f(*args, **kw)</tt>, and a reference to <tt class="docutils literal">File "<string>", line 2, in f</tt>.

932

883

This latter reference is due to the fact that internally the decorator

933

module uses <tt class="docutils literal">exec</tt> to generate the decorated function. Notice that

934

<tt class="docutils literal">exec</tt> is not responsibile for the performance penalty, since is the

884

module uses <tt class="docutils literal">exec</tt> to generate the decorated function. Notice that

885

<tt class="docutils literal">exec</tt> is not responsibile for the performance penalty, since is the

935

886

called only once at function decoration time, and not every time

936

887

the decorated function is called.

937

At present, there is no clean way to avoid <tt class="docutils literal">exec</tt>. A clean solution

888

At present, there is no clean way to avoid <tt class="docutils literal">exec</tt>. A clean solution

938

889

would require to change the CPython implementation of functions and

939

890

add an hook to make it possible to change their signature directly.

940

891

That could happen in future versions of Python (see PEP <a class="reference external" href="http://www.python.org/dev/peps/pep-0362">362</a>) and

941

892

then the decorator module would become obsolete. However, at present,

942

893

even in Python 3.1 it is impossible to change the function signature

943

directly, therefore the <tt class="docutils literal">decorator</tt> module is still useful.

894

directly, therefore the <tt class="docutils literal">decorator</tt> module is still useful.

944

895

Actually, this is one of the main reasons why I keep maintaining

945

896

the module and releasing new versions.

946

In the present implementation, decorators generated by <tt class="docutils literal">decorator</tt>

897

In the present implementation, decorators generated by <tt class="docutils literal">decorator</tt>

947

898

can only be used on user-defined Python functions or methods, not on generic

948

899

callable objects, nor on built-in functions, due to limitations of the

949

<tt class="docutils literal">inspect</tt> module in the standard library. Moreover, notice

900

<tt class="docutils literal">inspect</tt> module in the standard library. Moreover, notice

950

901

that you can decorate a method, but only before if becomes a bound or unbound

951

902

method, i.e. inside the class.

952

903

Here is an example of valid decoration:

980

931

981

932

</div>

982

933

There is a restriction on the names of the arguments: for instance,

983

if try to call an argument <tt class="docutils literal">_call_</tt> or <tt class="docutils literal">_func_</tt>

984

you will get a <tt class="docutils literal">NameError</tt>:

934

if try to call an argument <tt class="docutils literal">_call_</tt> or <tt class="docutils literal">_func_</tt>

935

you will get a <tt class="docutils literal">NameError</tt>:

985

936

986

937

<div class="highlight"><pre>>>> @trace

987

938

... def f(_func_): print f

996

947

</div>

997

948

Finally, the implementation is such that the decorated function contains

998

949

a copy of the original function dictionary

999

(<tt class="docutils literal">vars(decorated_f) is not vars(f)</tt>):

950

(<tt class="docutils literal">vars(decorated_f) is not vars(f)</tt>):

1000

951

1001

952

<div class="highlight"><pre>>>> def f(): pass # the original function

1002

953

>>> f.attr1 = "something" # setting an attribute

1015

966

</div>

1016

967

1017

968

<h1><a class="toc-backref" href="#id15">Compatibility notes</a></h1>

1018

Version 3.0 is a complete rewrite of the original implementation.

1019

It is mostly compatible with the past, a part for a few differences.

1020

First of all, the utilites <tt class="docutils literal">get_info</tt> and <tt class="docutils literal">new_wrapper</tt>, available

1021

in the 2.X versions, have been deprecated and they will be removed

1022

in the future. For the moment, using them raises a <tt class="docutils literal">DeprecationWarning</tt>.

1023

Incidentally, the functionality has been implemented through a

1024

decorator which makes a good example for this documentation:

1025

1026

<div class="highlight"><pre>@decorator

1027

def deprecated(func, *args, **kw):

1028

"A decorator for deprecated functions"

1029

warnings.warn(

1030

('Calling the deprecated function %r\n'

1031

'Downgrade to decorator 2.3 if you want to use this functionality')

1032

% func.__name__, DeprecationWarning, stacklevel=3)

1033

return func(*args, **kw)

1034

</pre></div>

1035

1036

</div>

1037

<tt class="docutils literal">get_info</tt> has been removed since it was little used and since it had

1038

to be changed anyway to work with Python 3.0; <tt class="docutils literal">new_wrapper</tt> has been

1039

removed since it was useless: its major use case (converting

1040

signature changing decorators to signature preserving decorators)

1041

has been subsumed by <tt class="docutils literal">decorator_apply</tt>

1042

and the other use case can be managed with the <tt class="docutils literal">FunctionMaker</tt>.

1043

Finally <tt class="docutils literal">decorator</tt> cannot be used as a class decorator and the

969

Version 3.2 is the first version of the <tt class="docutils literal">decorator</tt> module to officially

970

support Python 3. Actually, the module has supported Python 3 from

971

the beginning, via the <tt class="docutils literal">2to3</tt> conversion tool, but this step has

972

been now integrated in the build process, thanks to the <a class="reference external" href="http://packages.python.org/distribute/">distribute</a>

973

project, the Python 3-compatible replacement of easy_install.

974

The hard work (for me) has been converting the documentation and the

975

doctests. This has been possible only now that <a class="reference external" href="http://docutils.sourceforge.net/">docutils</a> and <a class="reference external" href="http://pygments.org/">pygments</a>

976

have been ported to Python 3.

977

The <tt class="docutils literal">decorator</tt> module per se does not contain any change, apart

978

from the removal of the functions <tt class="docutils literal">get_info</tt> and <tt class="docutils literal">new_wrapper</tt>,

979

which have been deprecated for years. <tt class="docutils literal">get_info</tt> has been removed

980

since it was little used and since it had to be changed anyway to work

981

with Python 3.0; <tt class="docutils literal">new_wrapper</tt> has been removed since it was

982

useless: its major use case (converting signature changing decorators

983

to signature preserving decorators) has been subsumed by

984

<tt class="docutils literal">decorator_apply</tt> and the other use case can be managed with the

985

<tt class="docutils literal">FunctionMaker</tt>.

986

There are a few changes in the documentation: I removed the

987

<tt class="docutils literal">decorator_factory</tt> example, which was confusing some of my users,

988

and I removed the part about exotic signatures in the Python 3

989

documentation, since Python 3 does not support them.

990

Notice that there is no support for Python 3 <a class="reference external" href="http://www.python.org/dev/peps/pep-3107/">function annotations</a>

991

since it seems premature at the moment, when most people are

992

still using Python 2.X.

993

Finally <tt class="docutils literal">decorator</tt> cannot be used as a class decorator and the

1044

994

<a class="reference external" href="http://www.phyast.pitt.edu/~micheles/python/documentation.html#class-decorators-and-decorator-factories">functionality introduced in version 2.3</a> has been removed. That

1045

995

means that in order to define decorator factories with classes you

1046

need to define the <tt class="docutils literal">__call__</tt> method explicitly (no magic anymore).

1047

Since there is

1048

an easy way to define decorator factories by using <tt class="docutils literal">decorator_factory</tt>,

1049

there is less need to use classes to implement decorator factories.

1050

All these changes should not cause any trouble, since they were

996

need to define the <tt class="docutils literal">__call__</tt> method explicitly (no magic anymore).

997

All these changes should not cause any trouble, since they were

1051

998

all rarely used features. Should you have any trouble, you can always

1052

999

downgrade to the 2.3 version.

1053

The examples shown here have been tested with Python 2.5. Python 2.4

1054

is also supported - of course the examples requiring the <tt class="docutils literal">with</tt>

1055

statement will not work there. Python 2.6 works fine, but if you

1056

run the examples here in the interactive interpreter

1000

The examples shown here have been tested with Python 2.6. Python 2.4

1001

is also supported - of course the examples requiring the <tt class="docutils literal">with</tt>

1002

statement will not work there. Python 2.5 works fine, but if you

1003

run the examples in the interactive interpreter

1057

1004

you will notice a few differences since

1058

<tt class="docutils literal">getargspec</tt> returns an <tt class="docutils literal">ArgSpec</tt> namedtuple instead of a regular

1005

<tt class="docutils literal">getargspec</tt> returns an <tt class="docutils literal">ArgSpec</tt> namedtuple instead of a regular

1059

1006

tuple. That means that running the file

1060

<tt class="docutils literal">documentation.py</tt> under Python 2.5 will a few errors, but

1061

they are not serious. Python 3.0 is kind of supported too.

1062

Simply run the script <tt class="docutils literal">2to3</tt> on the module

1063

<tt class="docutils literal">decorator.py</tt> and you will get a version of the code running

1064

with Python 3.0 (at least, I did some simple checks and it seemed

1065

to work). However there is no support for <a class="reference external" href="http://www.python.org/dev/peps/pep-3107/">function annotations</a> yet

1066

since it seems premature at this moment (most people are

1067

still using Python 2.5).

1007

<tt class="docutils literal">documentation.py</tt> under Python 2.5 will print a few errors, but

1008

they are not serious.

1068

1009

</div>

1069

1010

1070

1011

<h1><a class="toc-backref" href="#id16">LICENCE</a></h1>

1100

1041

you are unhappy with it, send me a patch!

1101

1042

</div>

1102

1043

</div>

1103

1104

1105

<a class="reference external" href="documentation.rst">View document source</a>.

1106

Generated on: 2009-08-25 12:37 UTC.

1107

Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.

1108

1109

</div>

1110

1044

</body>

1111

1045

</html>