~justin-fathomdb/nova/justinsb-openstack-api-volumes

<?xml version="1.0" encoding="utf-8"?><!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html lang="en" xmlns="http://www.w3.org/1999/xhtml">

<head>

<title>Twisted Documentation: Deferreds are beautiful! (A Tutorial)</title>

</head>

<h1 class="title">Deferreds are beautiful! (A Tutorial)</h1>

<div class="toc"><ol><li><a href="#auto0">Introduction</a></li><li><a href="#auto1">A simple example</a></li><li><a href="#auto2">Errbacks</a></li><ul><li><a href="#auto3">Failure in requested operation</a></li><li><a href="#auto4">Exceptions raised in callbacks</a></li><li><a href="#auto5">Exceptions will only be handled by errbacks</a></li><li><a href="#auto6">Handling an exception and continuing on</a></li></ul><li><a href="#auto7">addBoth: the deferred version of finally</a></li><li><a href="#auto8">addCallbacks: decision making based on previous success or failure</a></li><li><a href="#auto9">Hints, tips, common mistakes, and miscellaney</a></li><ul><li><a href="#auto10">The deferred callback chain is stateful</a></li><li><a href="#auto11">Don't call .callback() on deferreds you didn't create!</a></li><li><a href="#auto12">Callbacks can return deferreds</a></li></ul><li><a href="#auto13">Conclusion</a></li></ol></div>

<h2>Introduction<a name="auto0"/></h2>

Deferreds are quite possibly the single most confusing topic that a

newcomer to Twisted has to deal with. I am going to forgo the normal talk

about what deferreds are, what they aren't, and why they're used in Twisted.

Instead, I'm going show you the logic behind what they

do.

A deferred allows you to encapsulate the logic that you'd normally use to

make a series of function calls after receiving a result into a single object.

In the examples that follow, I'll first show you what's going to go on behind

the scenes in the deferred chain, then show you the deferred API calls that set

up that chain. All of these examples are runnable code, so feel free to play

around with them.

<h2>A simple example<a name="auto1"/></h2>

First, a simple example so that we have something to talk about:

<div class="py-listing"><pre> 1

#!/usr/bin/env python

# See LICENSE for details.

from twisted.internet import defer

100

from twisted.python import failure, util

101

102

"""

103

Here we have the simplest case, a single callback and a single errback.

104

"""

105

106

num = 0

107

108

def handleFailure(f):

109

print "errback"

110

print "we got an exception: %s" % (f.getTraceback(),)

111

f.trap(RuntimeError)

112

113

def handleResult(result):

114

global num; num += 1

115

print "callback %s" % (num,)

116

print "\tgot result: %s" % (result,)

117

return "yay! handleResult was successful!"

118

119

120

def behindTheScenes(result):

121

# equivalent to d.callback(result)

122

123

if not isinstance(result, failure.Failure): # ---- callback

124

try:

125

result = handleResult(result)

126

except:

127

result = failure.Failure()

128

else: # ---- errback

129

pass

130

131

132

133

pass

134

else: # ---- errback

135

try:

136

result = handleFailure(result)

137

except:

138

result = failure.Failure()

139

140

141

def deferredExample():

142

d = defer.Deferred()

143

d.addCallback(handleResult)

144

d.addErrback(handleFailure)

145

146

d.callback("success")

147

148

149

if __name__ == '__main__':

150

behindTheScenes("success")

151

print "\n-------------------------------------------------\n"

152

global num; num = 0

153

deferredExample()

154

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex.py">listings/deferred/deferred_ex.py</a></div></div>

155

156

And the output: (since both methods in the example produce the same output,

157

it will only be shown once.)

158

159

160

callback 1

161

got result: success

162

</pre>

163

164

Here we have the simplest case. A deferred with a single callback and a

165

single errback. Normally, a function would create a deferred and hand it back

166

to you when you request an operation that needs to wait for an event for

167

completion. The object you called then does <code>d.callback(result)</code>

168

when the results are in.

169

170

171

The thing to notice is that there is only one result that is passed from

172

method to method, and that the result returned from a method is the argument

173

to the next method in the chain. In case of an exception, result is set to an

174

instance of <code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.python.failure.Failure.html" title="twisted.python.failure.Failure">Failure</a></code>

175

that describes the exception.

176

177

<h2>Errbacks<a name="auto2"/></h2>

178

<h3>Failure in requested operation<a name="auto3"/></h3>

179

Things don't always go as planned, and sometimes the function that

180

returned the deferred needs to alert the callback chain that an error

181

has occurred.

182

183

<div class="py-listing"><pre> 1

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

#!/usr/bin/env python

257

258

259

# See LICENSE for details.

260

261

262

263

264

"""

265

This example is analogous to a function calling .errback(failure)

266

"""

267

268

269

class Counter(object):

270

num = 0

271

272

def handleFailure(f):

273

print "errback"

274

print "we got an exception: %s" % (f.getTraceback(),)

275

f.trap(RuntimeError)

276

277

def handleResult(result):

278

Counter.num += 1

279

print "callback %s" % (Counter.num,)

280

print "\tgot result: %s" % (result,)

281

return "yay! handleResult was successful!"

282

283

def failAtHandlingResult(result):

284

Counter.num += 1

285

print "callback %s" % (Counter.num,)

286

print "\tgot result: %s" % (result,)

287

print "\tabout to raise exception"

288

raise RuntimeError, "whoops! we encountered an error"

289

290

291

def behindTheScenes(result):

292

293

try:

294

result = handleResult(result)

295

except:

296

result = failure.Failure()

297

else: # ---- errback

298

pass

299

300

301

302

pass

303

else: # ---- errback

304

try:

305

result = handleFailure(result)

306

except:

307

result = failure.Failure()

308

309

310

def deferredExample(result):

311

d = defer.Deferred()

312

d.addCallback(handleResult)

313

d.addCallback(failAtHandlingResult)

314

d.addErrback(handleFailure)

315

316

d.errback(result)

317

318

319

if __name__ == '__main__':

320

result = None

321

try:

322

raise RuntimeError, "*doh*! failure!"

323

except:

324

result = failure.Failure()

325

behindTheScenes(result)

326

print "\n-------------------------------------------------\n"

327

Counter.num = 0

328

deferredExample(result)

329

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex1a.py">listings/deferred/deferred_ex1a.py</a></div></div>

330

331

332

errback

333

we got an exception: Traceback (most recent call last):

334

--- exception caught here ---

335

File "deferred_ex1a.py", line 73, in ?

336

raise RuntimeError, "*doh*! failure!"

337

exceptions.RuntimeError: *doh*! failure!

338

</pre>

339

340

The important thing to note (as it will come up again in later examples)

341

is that the callback isn't touched, the failure goes right to the errback.

342

Also note that the errback trap()s the expected exception type. If you don't

343

trap the exception, an error will be logged when the deferred is

344

garbage-collected.

345

346

347

348

<h3>Exceptions raised in callbacks<a name="auto4"/></h3>

349

350

Now let's see what happens when our callback raises an

351

exception

352

353

<div class="py-listing"><pre> 1

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

#!/usr/bin/env python

433

434

435

# See LICENSE for details.

436

437

438

439

440

"""

441

Here we have a slightly more involved case. The deferred is called back with a

442

result. the first callback returns a value, the second callback, however

443

raises an exception, which is handled by the errback.

444

"""

445

446

447

class Counter(object):

448

num = 0

449

450

def handleFailure(f):

451

print "errback"

452

print "we got an exception: %s" % (f.getTraceback(),)

453

f.trap(RuntimeError)

454

455

def handleResult(result):

456

Counter.num += 1

457

print "callback %s" % (Counter.num,)

458

print "\tgot result: %s" % (result,)

459

return "yay! handleResult was successful!"

460

461

def failAtHandlingResult(result):

462

Counter.num += 1

463

print "callback %s" % (Counter.num,)

464

print "\tgot result: %s" % (result,)

465

print "\tabout to raise exception"

466

raise RuntimeError, "whoops! we encountered an error"

467

468

469

def behindTheScenes(result):

470

471

try:

472

result = handleResult(result)

473

except:

474

result = failure.Failure()

475

else: # ---- errback

476

pass

477

478

479

480

try:

481

result = failAtHandlingResult(result)

482

except:

483

result = failure.Failure()

484

else: # ---- errback

485

pass

486

487

488

489

pass

490

else: # ---- errback

491

try:

492

result = handleFailure(result)

493

except:

494

result = failure.Failure()

495

496

497

def deferredExample():

498

d = defer.Deferred()

499

d.addCallback(handleResult)

500

d.addCallback(failAtHandlingResult)

501

d.addErrback(handleFailure)

502

503

d.callback("success")

504

505

506

if __name__ == '__main__':

507

behindTheScenes("success")

508

print "\n-------------------------------------------------\n"

509

Counter.num = 0

510

deferredExample()

511

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex1b.py">listings/deferred/deferred_ex1b.py</a></div></div>

512

513

And the output: (note, tracebacks will be edited slightly to conserve

514

space)

515

516

517

callback 1

518

got result: success

519

callback 2

520

got result: yay! handleResult was successful!

521

about to raise exception

522

errback

523

we got an exception: Traceback (most recent call last):

524

--- <exception caught here> ---

525

File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line

526

326, in _runCallbacks

527

self.result = callback(self.result, *args, **kw)

528

File "./deferred_ex1.py", line 32, in failAtHandlingResult

529

raise RuntimeError, "whoops! we encountered an error"

530

exceptions.RuntimeError: whoops! we encountered an error

531

</pre>

532

533

If your callback raises an exception, the next method to be called will be

534

the next errback in your chain.

535

536

537

<h3>Exceptions will only be handled by errbacks<a name="auto5"/></h3>

538

539

If a callback raises an exception the next method to be called will be next

540

errback in the chain. If the chain is started off with a failure, the first

541

method to be called will be the first errback.

542

543

<div class="py-listing"><pre> 1

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

#!/usr/bin/env python

635

636

637

# See LICENSE for details.

638

639

640

641

642

"""

643

This example shows an important concept that many deferred newbies

644

(myself included) have trouble understanding.

645

646

when an error occurs in a callback, the first errback after the error

647

occurs will be the next method called. (in the next example we'll

648

see what happens in the 'chain' after an errback).

649

"""

650

651

class Counter(object):

652

num = 0

653

654

def handleFailure(f):

655

print "errback"

656

print "we got an exception: %s" % (f.getTraceback(),)

657

f.trap(RuntimeError)

658

659

def handleResult(result):

660

Counter.num += 1

661

print "callback %s" % (Counter.num,)

662

print "\tgot result: %s" % (result,)

663

return "yay! handleResult was successful!"

664

665

def failAtHandlingResult(result):

666

Counter.num += 1

667

print "callback %s" % (Counter.num,)

668

print "\tgot result: %s" % (result,)

669

print "\tabout to raise exception"

670

raise RuntimeError, "whoops! we encountered an error"

671

672

673

674

def behindTheScenes(result):

675

# equivalent to d.callback(result)

676

677

# now, let's make the error happen in the first callback

678

679

680

try:

681

result = failAtHandlingResult(result)

682

except:

683

result = failure.Failure()

684

else: # ---- errback

685

pass

686

687

688

# note: this callback will be skipped because

689

# result is a failure

690

691

692

try:

693

result = handleResult(result)

694

except:

695

result = failure.Failure()

696

else: # ---- errback

697

pass

698

699

700

701

pass

702

else: # ---- errback

703

try:

704

result = handleFailure(result)

705

except:

706

result = failure.Failure()

707

708

709

710

def deferredExample():

711

d = defer.Deferred()

712

d.addCallback(failAtHandlingResult)

713

d.addCallback(handleResult)

714

d.addErrback(handleFailure)

715

716

d.callback("success")

717

718

719

if __name__ == '__main__':

720

behindTheScenes("success")

721

print "\n-------------------------------------------------\n"

722

Counter.num = 0

723

deferredExample()

724

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex2.py">listings/deferred/deferred_ex2.py</a></div></div>

725

726

727

callback 1

728

got result: success

729

about to raise exception

730

errback

731

we got an exception: Traceback (most recent call last):

732

File "./deferred_ex2.py", line 85, in ?

733

nonDeferredExample("success")

734

--- <exception caught here> ---

735

File "./deferred_ex2.py", line 46, in nonDeferredExample

736

result = failAtHandlingResult(result)

737

File "./deferred_ex2.py", line 35, in failAtHandlingResult

738

raise RuntimeError, "whoops! we encountered an error"

739

exceptions.RuntimeError: whoops! we encountered an error

740

</pre>

741

742

You can see that our second callback, handleResult was not called because

743

failAtHandlingResult raised an exception

744

745

<h3>Handling an exception and continuing on<a name="auto6"/></h3>

746

747

In this example, we see an errback handle an exception raised in the

748

preceeding callback. Take note that it could just as easily been an exception

749

from any other preceeding method. You'll see that after the

750

exception is handled in the errback (i.e. the errback does not return a

751

failure or raise an exception) the chain continues on with the next

752

callback.

753

754

<div class="py-listing"><pre> 1

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

770

771

772

773

774

775

776

777

778

779

780

781

782

783

784

785

786

787

788

789

790

791

792

793

794

795

796

797

798

799

800

801

802

803

804

805

806

807

808

809

810

811

812

813

814

815

816

817

818

819

820

821

822

823

824

825

826

827

828

829

830

831

832

833

834

835

836

837

838

839

840

841

842

843

844

845

846

847

848

849

850

851

852

853

100

854

#!/usr/bin/env python

855

856

857

# See LICENSE for details.

858

859

860

861

862

"""

863

Now we see how an errback can handle errors. if an errback

864

does not raise an exception, the next callback in the chain

865

will be called.

866

"""

867

868

class Counter(object):

869

num = 0

870

871

872

def handleFailure(f):

873

print "errback"

874

print "we got an exception: %s" % (f.getTraceback(),)

875

f.trap(RuntimeError)

876

return "okay, continue on"

877

878

def handleResult(result):

879

Counter.num += 1

880

print "callback %s" % (Counter.num,)

881

print "\tgot result: %s" % (result,)

882

return "yay! handleResult was successful!"

883

884

def failAtHandlingResult(result):

885

Counter.num += 1

886

print "callback %s" % (Counter.num,)

887

print "\tgot result: %s" % (result,)

888

print "\tabout to raise exception"

889

raise RuntimeError, "whoops! we encountered an error"

890

891

def callbackAfterErrback(result):

892

Counter.num += 1

893

print "callback %s" % (Counter.num,)

894

print "\tgot result: %s" % (result,)

895

896

897

898

def behindTheScenes(result):

899

# equivalent to d.callback(result)

900

901

902

try:

903

result = handleResult(result)

904

except:

905

result = failure.Failure()

906

else: # ---- errback

907

pass

908

909

910

911

try:

912

result = failAtHandlingResult(result)

913

except:

914

result = failure.Failure()

915

else: # ---- errback

916

pass

917

918

919

920

pass

921

else: # ---- errback

922

try:

923

result = handleFailure(result)

924

except:

925

result = failure.Failure()

926

927

928

929

try:

930

result = callbackAfterErrback(result)

931

except:

932

result = failure.Failure()

933

else: # ---- errback

934

pass

935

936

937

938

def deferredExample():

939

d = defer.Deferred()

940

d.addCallback(handleResult)

941

d.addCallback(failAtHandlingResult)

942

d.addErrback(handleFailure)

943

d.addCallback(callbackAfterErrback)

944

945

d.callback("success")

946

947

948

if __name__ == '__main__':

949

behindTheScenes("success")

950

print "\n-------------------------------------------------\n"

951

Counter.num = 0

952

deferredExample()

953

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex3.py">listings/deferred/deferred_ex3.py</a></div></div>

954

955

956

callback 1

957

got result: success

958

callback 2

959

got result: yay! handleResult was successful!

960

about to raise exception

961

errback

962

we got an exception: Traceback (most recent call last):

963

File "./deferred_ex3.py", line 97, in <module>

964

deferredExample()

965

File "./deferred_ex3.py", line 90, in deferredExample

966

d.callback("success")

967

File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line 243, in callback

968

self._startRunCallbacks(result)

969

File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line 312, in _startRunCallbacks

970

self._runCallbacks()

971

--- <exception caught here> ---

972

File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line 328, in _runCallbacks

973

self.result = callback(self.result, *args, **kw)

974

File "./deferred_ex3.py", line 34, in failAtHandlingResult

975

raise RuntimeError, "whoops! we encountered an error"

976

exceptions.RuntimeError: whoops! we encountered an error

977

978

callback 3

979

got result: okay, continue on

980

</pre>

981

982

<h2>addBoth: the deferred version of finally<a name="auto7"/></h2>

983

984

Now we see how deferreds do finally, with .addBoth. The

985

callback that gets added as addBoth will be called if the result is a failure

986

or non-failure. We'll also see in this example, that our doThisNoMatterWhat()

987

method follows a common idiom in deferred callbacks by acting as a passthru,

988

returning the value that it received to allow processing the chain to

989

continue, but appearing transparent in terms of the result.

990

991

<div class="py-listing"><pre> 1

992

993

994

995

996

997

998

999

1000

1001

1002

1003

1004

1005

1006

1007

1008

1009

1010

1011

1012

1013

1014

1015

1016

1017

1018

1019

1020

1021

1022

1023

1024

1025

1026

1027

1028

1029

1030

1031

1032

1033

1034

1035

1036

1037

1038

1039

1040

1041

1042

1043

1044

1045

1046

1047

1048

1049

1050

1051

1052

1053

1054

1055

1056

1057

1058

1059

1060

1061

1062

1063

1064

1065

1066

1067

1068

1069

1070

1071

1072

1073

1074

1075

1076

1077

1078

1079

1080

1081

1082

1083

1084

1085

1086

1087

1088

1089

1090

100

1091

101

1092

102

1093

103

1094

104

1095

#!/usr/bin/env python

1096

1097

1098

# See LICENSE for details.

1099

1100

1101

1102

1103

"""

1104

Now we'll see what happens when you use 'addBoth'.

1105

"""

1106

1107

class Counter(object):

1108

num = 0

1109

1110

1111

def handleFailure(f):

1112

print "errback"

1113

print "we got an exception: %s" % (f.getTraceback(),)

1114

f.trap(RuntimeError)

1115

return "okay, continue on"

1116

1117

def handleResult(result):

1118

Counter.num += 1

1119

print "callback %s" % (Counter.num,)

1120

print "\tgot result: %s" % (result,)

1121

return "yay! handleResult was successful!"

1122

1123

def failAtHandlingResult(result):

1124

Counter.num += 1

1125

print "callback %s" % (Counter.num,)

1126

print "\tgot result: %s" % (result,)

1127

print "\tabout to raise exception"

1128

raise RuntimeError, "whoops! we encountered an error"

1129

1130

def doThisNoMatterWhat(arg):

1131

Counter.num += 1

1132

print "both %s" % (Counter.num,)

1133

print "\tgot argument %r" % (arg,)

1134

print "\tdoing something very important"

1135

# we pass the argument we received to the next phase here

1136

return arg

1137

1138

1139

1140

def behindTheScenes(result):

1141

# equivalent to d.callback(result)

1142

1143

1144

try:

1145

result = handleResult(result)

1146

except:

1147

result = failure.Failure()

1148

else: # ---- errback

1149

pass

1150

1151

1152

1153

try:

1154

result = failAtHandlingResult(result)

1155

except:

1156

result = failure.Failure()

1157

else: # ---- errback

1158

pass

1159

1160

1161

# ---- this is equivalent to addBoth(doThisNoMatterWhat)

1162

1163

1164

try:

1165

result = doThisNoMatterWhat(result)

1166

except:

1167

result = failure.Failure()

1168

else:

1169

try:

1170

result = doThisNoMatterWhat(result)

1171

except:

1172

result = failure.Failure()

1173

1174

1175

1176

pass

1177

else: # ---- errback

1178

try:

1179

result = handleFailure(result)

1180

except:

1181

result = failure.Failure()

1182

1183

1184

def deferredExample():

1185

d = defer.Deferred()

1186

d.addCallback(handleResult)

1187

d.addCallback(failAtHandlingResult)

1188

d.addBoth(doThisNoMatterWhat)

1189

d.addErrback(handleFailure)

1190

1191

d.callback("success")

1192

1193

1194

if __name__ == '__main__':

1195

behindTheScenes("success")

1196

print "\n-------------------------------------------------\n"

1197

Counter.num = 0

1198

deferredExample()

1199

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex4.py">listings/deferred/deferred_ex4.py</a></div></div>

1200

1201

1202

callback 1

1203

got result: success

1204

callback 2

1205

got result: yay! handleResult was successful!

1206

about to raise exception

1207

both 3

1208

got argument <twisted.python.failure.Failure exceptions.RuntimeError>

1209

doing something very important

1210

errback

1211

we got an exception: Traceback (most recent call last):

1212

--- <exception caught here> ---

1213

File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line

1214

326, in _runCallbacks

1215

self.result = callback(self.result, *args, **kw)

1216

File "./deferred_ex4.py", line 32, in failAtHandlingResult

1217

raise RuntimeError, "whoops! we encountered an error"

1218

exceptions.RuntimeError: whoops! we encountered an error

1219

</pre>

1220

1221

You can see that the errback is called, (and consequently, the failure is

1222

trapped). This is because doThisNoMatterWhat method returned the value it

1223

received, a failure.

1224

1225

<h2>addCallbacks: decision making based on previous success or failure<a name="auto8"/></h2>

1226

1227

As we've been seeing in the examples, the callback is a pair of

1228

callback/errback. Using addCallback or addErrback is actually a special case

1229

where one of the pair is a pass statement. If you want to make a decision

1230

based on whether or not the previous result in the chain was a failure or not

1231

(which is very rare, but included here for completeness), you use

1232

addCallbacks. Note that this is not the same thing as an

1233

addCallback followed by an addErrback.

1234

1235

1236

<div class="py-listing"><pre> 1

1237

1238

1239

1240

1241

1242

1243

1244

1245

1246

1247

1248

1249

1250

1251

1252

1253

1254

1255

1256

1257

1258

1259

1260

1261

1262

1263

1264

1265

1266

1267

1268

1269

1270

1271

1272

1273

1274

1275

1276

1277

1278

1279

1280

1281

1282

1283

1284

1285

1286

1287

1288

1289

1290

1291

1292

1293

1294

1295

1296

1297

1298

1299

1300

1301

1302

1303

1304

1305

1306

1307

1308

1309

1310

1311

1312

1313

1314

1315

1316

1317

1318

1319

1320

1321

1322

1323

1324

1325

1326

1327

1328

1329

1330

1331

1332

1333

1334

1335

100

1336

101

1337

102

1338

103

1339

104

1340

105

1341

106

1342

107

1343

108

1344

109

1345

110

1346

111

1347

112

1348

113

1349

114

1350

115

1351

116

1352

117

1353

118

1354

119

1355

120

1356

121

1357

122

1358

123

1359

124

1360

125

1361

126

1362

127

1363

128

1364

129

1365

130

1366

131

1367

132

1368

133

1369

134

1370

135

1371

136

1372

#!/usr/bin/env python

1373

1374

1375

# See LICENSE for details.

1376

1377

1378

1379

1380

"""

1381

Now comes the more nuanced addCallbacks, which allows us to make a

1382

yes/no (branching) decision based on whether the result at a given point is

1383

a failure or not.

1384

"""

1385

1386

class Counter(object):

1387

num = 0

1388

1389

1390

def handleFailure(f):

1391

print "errback"

1392

print "we got an exception: %s" % (f.getTraceback(),)

1393

f.trap(RuntimeError)

1394

return "okay, continue on"

1395

1396

def handleResult(result):

1397

Counter.num += 1

1398

print "callback %s" % (Counter.num,)

1399

print "\tgot result: %s" % (result,)

1400

return "yay! handleResult was successful!"

1401

1402

def failAtHandlingResult(result):

1403

Counter.num += 1

1404

print "callback %s" % (Counter.num,)

1405

print "\tgot result: %s" % (result,)

1406

print "\tabout to raise exception"

1407

raise RuntimeError, "whoops! we encountered an error"

1408

1409

def yesDecision(result):

1410

Counter.num += 1

1411

print "yes decision %s" % (Counter.num,)

1412

print "\twasn't a failure, so we can plow ahead"

1413

return "go ahead!"

1414

1415

def noDecision(result):

1416

Counter.num += 1

1417

result.trap(RuntimeError)

1418

print "no decision %s" % (Counter.num,)

1419

print "\t*doh*! a failure! quick! damage control!"

1420

return "damage control successful!"

1421

1422

1423

1424

def behindTheScenes(result):

1425

1426

1427

try:

1428

result = failAtHandlingResult(result)

1429

except:

1430

result = failure.Failure()

1431

else: # ---- errback

1432

pass

1433

1434

1435

# this is equivalent to addCallbacks(yesDecision, noDecision)

1436

1437

1438

try:

1439

result = yesDecision(result)

1440

except:

1441

result = failure.Failure()

1442

else: # ---- errback

1443

try:

1444

result = noDecision(result)

1445

except:

1446

result = failure.Failure()

1447

1448

1449

1450

try:

1451

result = handleResult(result)

1452

except:

1453

result = failure.Failure()

1454

else: # ---- errback

1455

pass

1456

1457

1458

# this is equivalent to addCallbacks(yesDecision, noDecision)

1459

1460

1461

try:

1462

result = yesDecision(result)

1463

except:

1464

result = failure.Failure()

1465

else: # ---- errback

1466

try:

1467

result = noDecision(result)

1468

except:

1469

result = failure.Failure()

1470

1471

1472

1473

try:

1474

result = handleResult(result)

1475

except:

1476

result = failure.Failure()

1477

else: # ---- errback

1478

pass

1479

1480

1481

1482

pass

1483

else: # ---- errback

1484

try:

1485

result = handleFailure(result)

1486

except:

1487

result = failure.Failure()

1488

1489

1490

def deferredExample():

1491

d = defer.Deferred()

1492

d.addCallback(failAtHandlingResult)

1493

d.addCallbacks(yesDecision, noDecision) # noDecision will be called

1494

d.addCallback(handleResult) # - A -

1495

1496

d.addCallback(handleResult)

1497

d.addErrback(handleFailure)

1498

1499

d.callback("success")

1500

1501

1502

if __name__ == '__main__':

1503

behindTheScenes("success")

1504

print "\n-------------------------------------------------\n"

1505

Counter.num = 0

1506

deferredExample()

1507

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex5.py">listings/deferred/deferred_ex5.py</a></div></div>

1508

1509

1510

callback 1

1511

got result: success

1512

about to raise exception

1513

no decision 2

1514

*doh*! a failure! quick! damage control!

1515

callback 3

1516

got result: damage control successful!

1517

yes decision 4

1518

wasn't a failure, so we can plow ahead

1519

callback 5

1520

got result: go ahead!

1521

</pre>

1522

1523

Notice that our errback is never called. The noDecision method returns a

1524

non-failure so processing continues with the next callback. If we wanted to

1525

skip the callback at "- A -" because of the error, but do some kind of

1526

processing in response to the error, we would have used a passthru, and

1527

returned the failure we received, as we see in this next example:

1528

1529

<div class="py-listing"><pre> 1

1530

1531

1532

1533

1534

1535

1536

1537

1538

1539

1540

1541

1542

1543

1544

1545

1546

1547

1548

1549

1550

1551

1552

1553

1554

1555

1556

1557

1558

1559

1560

1561

1562

1563

1564

1565

1566

1567

1568

1569

1570

1571

1572

1573

1574

1575

1576

1577

1578

1579

1580

1581

1582

1583

1584

1585

1586

1587

1588

1589

1590

1591

1592

1593

1594

1595

1596

1597

1598

1599

1600

1601

1602

1603

1604

1605

1606

1607

1608

1609

1610

1611

1612

1613

1614

1615

1616

1617

1618

1619

1620

1621

1622

1623

1624

1625

1626

1627

1628

100

1629

101

1630

102

1631

103

1632

104

1633

105

1634

106

1635

107

1636

108

1637

109

1638

110

1639

111

1640

112

1641

113

1642

114

1643

115

1644

116

1645

117

1646

118

1647

119

1648

120

1649

121

1650

122

1651

123

1652

124

1653

125

1654

126

1655

127

1656

128

1657

129

1658

130

1659

131

1660

132

1661

133

1662

134

1663

135

1664

136

1665

137

1666

138

1667

139

1668

140

1669

141

1670

142

1671

143

1672

144

1673

145

1674

146

1675

147

1676

148

1677

#!/usr/bin/env python

1678

1679

1680

# See LICENSE for details.

1681

1682

1683

1684

1685

"""

1686

Now comes the more nuanced addCallbacks, which allows us to make a

1687

yes/no (branching) decision based on whether the result at a given point is

1688

a failure or not.

1689

1690

here, we return the failure from noDecisionPassthru, the errback argument to

1691

the first addCallbacks method invocation, and see what happens.

1692

"""

1693

1694

class Counter(object):

1695

num = 0

1696

1697

1698

def handleFailure(f):

1699

print "errback"

1700

print "we got an exception: %s" % (f.getTraceback(),)

1701

f.trap(RuntimeError)

1702

return "okay, continue on"

1703

1704

def handleResult(result):

1705

Counter.num += 1

1706

print "callback %s" % (Counter.num,)

1707

print "\tgot result: %s" % (result,)

1708

return "yay! handleResult was successful!"

1709

1710

def failAtHandlingResult(result):

1711

Counter.num += 1

1712

print "callback %s" % (Counter.num,)

1713

print "\tgot result: %s" % (result,)

1714

print "\tabout to raise exception"

1715

raise RuntimeError, "whoops! we encountered an error"

1716

1717

def yesDecision(result):

1718

Counter.num += 1

1719

print "yes decision %s" % (Counter.num,)

1720

print "\twasn't a failure, so we can plow ahead"

1721

return "go ahead!"

1722

1723

def noDecision(result):

1724

Counter.num += 1

1725

result.trap(RuntimeError)

1726

print "no decision %s" % (Counter.num,)

1727

print "\t*doh*! a failure! quick! damage control!"

1728

return "damage control successful!"

1729

1730

def noDecisionPassthru(result):

1731

Counter.num += 1

1732

print "no decision %s" % (Counter.num,)

1733

print "\t*doh*! a failure! don't know what to do, returning failure!"

1734

return result

1735

1736

1737

def behindTheScenes(result):

1738

1739

1740

try:

1741

result = failAtHandlingResult(result)

1742

except:

1743

result = failure.Failure()

1744

else: # ---- errback

1745

pass

1746

1747

1748

# this is equivalent to addCallbacks(yesDecision, noDecision)

1749

1750

1751

try:

1752

result = yesDecision(result)

1753

except:

1754

result = failure.Failure()

1755

else: # ---- errback

1756

try:

1757

result = noDecisionPassthru(result)

1758

except:

1759

result = failure.Failure()

1760

1761

1762

1763

try:

1764

result = handleResult(result)

1765

except:

1766

result = failure.Failure()

1767

else: # ---- errback

1768

pass

1769

1770

1771

# this is equivalent to addCallbacks(yesDecision, noDecision)

1772

1773

1774

try:

1775

result = yesDecision(result)

1776

except:

1777

result = failure.Failure()

1778

else: # ---- errback

1779

try:

1780

result = noDecision(result)

1781

except:

1782

result = failure.Failure()

1783

1784

1785

1786

try:

1787

result = handleResult(result)

1788

except:

1789

result = failure.Failure()

1790

else: # ---- errback

1791

pass

1792

1793

1794

1795

pass

1796

else: # ---- errback

1797

try:

1798

result = handleFailure(result)

1799

except:

1800

result = failure.Failure()

1801

1802

1803

def deferredExample():

1804

d = defer.Deferred()

1805

d.addCallback(failAtHandlingResult)

1806

1807

# noDecisionPassthru will be called

1808

d.addCallbacks(yesDecision, noDecisionPassthru)

1809

d.addCallback(handleResult) # - A -

1810

1811

# noDecision will be called

1812

d.addCallbacks(yesDecision, noDecision)

1813

d.addCallback(handleResult) # - B -

1814

d.addErrback(handleFailure)

1815

1816

d.callback("success")

1817

1818

1819

if __name__ == '__main__':

1820

behindTheScenes("success")

1821

print "\n-------------------------------------------------\n"

1822

Counter.num = 0

1823

deferredExample()

1824

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex6.py">listings/deferred/deferred_ex6.py</a></div></div>

1825

1826

1827

callback 1

1828

got result: success

1829

about to raise exception

1830

no decision 2

1831

*doh*! a failure! don't know what to do, returning failure!

1832

no decision 3

1833

*doh*! a failure! quick! damage control!

1834

callback 4

1835

got result: damage control successful!

1836

</pre>

1837

1838

Two things to note here. First, "- A -" was skipped, like we wanted it to,

1839

and the second thing is that after "- A -", noDecision is called, because

1840

it is the next errback that exists in the chain. It returns a

1841

non-failure, so processing continues with the next callback at "- B -", and

1842

the errback at the end of the chain is never called

1843

1844

<h2>Hints, tips, common mistakes, and miscellaney<a name="auto9"/></h2>

1845

1846

<h3>The deferred callback chain is stateful<a name="auto10"/></h3>

1847

1848

A deferred that has been called back will call its addCallback and

1849

addErrback methods as appropriate in the order they are added, when they are

1850

added. So we see in the following example, deferredExample1 and

1851

deferredExample2 are equivalent. The first sets up the processing chain

1852

beforehand and then executes it, the other executes the chain as it is being

1853

constructed. This is because deferreds are stateful.

1854

1855

<div class="py-listing"><pre> 1

1856

1857

1858

1859

1860

1861

1862

1863

1864

1865

1866

1867

1868

1869

1870

1871

1872

1873

1874

1875

1876

1877

1878

1879

1880

1881

1882

1883

1884

1885

1886

1887

1888

1889

1890

1891

1892

1893

1894

1895

1896

1897

1898

1899

1900

1901

1902

1903

1904

1905

1906

1907

1908

1909

1910

1911

1912

1913

1914

1915

1916

#!/usr/bin/env python

1917

1918

1919

# See LICENSE for details.

1920

1921

1922

1923

1924

"""

1925

The deferred callback chain is stateful, and can be executed before

1926

or after all callbacks have been added to the chain

1927

"""

1928

1929

class Counter(object):

1930

num = 0

1931

1932

def handleFailure(f):

1933

print "errback"

1934

print "we got an exception: %s" % (f.getTraceback(),)

1935

f.trap(RuntimeError)

1936

1937

def handleResult(result):

1938

Counter.num += 1

1939

print "callback %s" % (Counter.num,)

1940

print "\tgot result: %s" % (result,)

1941

return "yay! handleResult was successful!"

1942

1943

def failAtHandlingResult(result):

1944

Counter.num += 1

1945

print "callback %s" % (Counter.num,)

1946

print "\tgot result: %s" % (result,)

1947

print "\tabout to raise exception"

1948

raise RuntimeError, "whoops! we encountered an error"

1949

1950

def deferredExample1():

1951

# this is another common idiom, since all add* methods

1952

# return the deferred instance, you can just chain your

1953

# calls to addCallback and addErrback

1954

1955

d = defer.Deferred().addCallback(failAtHandlingResult

1956

).addCallback(handleResult

1957

).addErrback(handleFailure)

1958

1959

d.callback("success")

1960

1961

def deferredExample2():

1962

d = defer.Deferred()

1963

1964

d.callback("success")

1965

1966

d.addCallback(failAtHandlingResult)

1967

d.addCallback(handleResult)

1968

d.addErrback(handleFailure)

1969

1970

1971

if __name__ == '__main__':

1972

deferredExample1()

1973

print "\n-------------------------------------------------\n"

1974

Counter.num = 0

1975

deferredExample2()

1976

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex7.py">listings/deferred/deferred_ex7.py</a></div></div>

1977

1978

1979

callback 1

1980

got result: success

1981

about to raise exception

1982

errback

1983

we got an exception: Traceback (most recent call last):

1984

--- <exception caught here> ---

1985

File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line

1986

326, in _runCallbacks

1987

self.result = callback(self.result, *args, **kw)

1988

File "./deferred_ex7.py", line 35, in failAtHandlingResult

1989

raise RuntimeError, "whoops! we encountered an error"

1990

exceptions.RuntimeError: whoops! we encountered an error

1991

1992

1993

-------------------------------------------------

1994

1995

callback 1

1996

got result: success

1997

about to raise exception

1998

errback

1999

we got an exception: Traceback (most recent call last):

2000

--- <exception caught here> ---

2001

File "/home/slyphon/Projects/Twisted/trunk/twisted/internet/defer.py", line

2002

326, in _runCallbacks

2003

self.result = callback(self.result, *args, **kw)

2004

File "./deferred_ex7.py", line 35, in failAtHandlingResult

2005

raise RuntimeError, "whoops! we encountered an error"

2006

exceptions.RuntimeError: whoops! we encountered an error

2007

</pre>

2008

2009

This example also shows you the common idiom of chaining calls to

2010

addCallback and addErrback.

2011

2012

2013

<h3>Don't call .callback() on deferreds you didn't create!<a name="auto11"/></h3>

2014

2015

It is an error to reinvoke deferreds callback or errback method, therefore

2016

if you didn't create a deferred, do not under any

2017

circumstances call its callback or errback. doing so will raise

2018

an exception

2019

2020

<h3>Callbacks can return deferreds<a name="auto12"/></h3>

2021

2022

If you need to call a method that returns a deferred within your callback

2023

chain, just return that deferred, and the result of the secondary deferred's

2024

processing chain will become the result that gets passed to the next callback

2025

of the primary deferreds processing chain

2026

2027

<div class="py-listing"><pre> 1

2028

2029

2030

2031

2032

2033

2034

2035

2036

2037

2038

2039

2040

2041

2042

2043

2044

2045

2046

2047

2048

2049

2050

2051

2052

2053

2054

2055

2056

2057

2058

2059

2060

2061

2062

2063

2064

2065

2066

2067

2068

2069

2070

2071

2072

2073

2074

2075

2076

2077

2078

2079

2080

2081

2082

2083

2084

2085

2086

2087

2088

2089

2090

2091

2092

2093

#!/usr/bin/env python

2094

2095

2096

# See LICENSE for details.

2097

2098

2099

2100

2101

2102

class Counter(object):

2103

num = 0

2104

let = 'a'

2105

2106

def incrLet(cls):

2107

cls.let = chr(ord(cls.let) + 1)

2108

incrLet = classmethod(incrLet)

2109

2110

2111

def handleFailure(f):

2112

print "errback"

2113

print "we got an exception: %s" % (f.getTraceback(),)

2114

return f

2115

2116

def subCb_B(result):

2117

print "sub-callback %s" % (Counter.let,)

2118

Counter.incrLet()

2119

s = " beautiful!"

2120

print "\tadding %r to result" % (s,)

2121

result += s

2122

return result

2123

2124

def subCb_A(result):

2125

print "sub-callback %s" % (Counter.let,)

2126

Counter.incrLet()

2127

s = " are "

2128

print "\tadding %r to result" % (s,)

2129

result += s

2130

return result

2131

2132

def mainCb_1(result):

2133

Counter.num += 1

2134

print "callback %s" % (Counter.num,)

2135

print "\tgot result: %s" % (result,)

2136

result += " Deferreds "

2137

2138

2139

).addCallback(subCb_B)

2140

d.callback(result)

2141

return d

2142

2143

def mainCb_2(result):

2144

Counter.num += 1

2145

print "callback %s" % (Counter.num,)

2146

print "\tgot result: %s" % (result,)

2147

2148

2149

def deferredExample():

2150

2151

).addCallback(mainCb_2)

2152

2153

d.callback("I hope you'll agree: ")

2154

2155

2156

if __name__ == '__main__':

2157

deferredExample()

2158

</pre><div class="caption">Source listing - <a href="listings/deferred/deferred_ex8.py">listings/deferred/deferred_ex8.py</a></div></div>

2159

2160

2161

callback 1

2162

got result: I hope you'll agree:

2163

sub-callback a

2164

adding ' are ' to result

2165

sub-callback b

2166

adding ' beautiful!' to result

2167

callback 2

2168

got result: I hope you'll agree: Deferreds are beautiful!

2169

</pre>

2170

2171

<h2>Conclusion<a name="auto13"/></h2>

2172

2173

Deferreds can be confusing, but only because they're so elegant and simple.

2174

There is a lot of logical power that can expressed with a deferred's

2175

processing chain, and once you see what's going on behind the curtain, it's a

2176

lot easier to understand how to make use of what deferreds have to offer.

2177

2178

</div>

2179

2180

<a href="index.html">Index</a>

2181

Version: 10.0.0

2182

</body>

2183

</html>

b'\\ No newline at end of file'

Older »