~soren/nova/iptables-security-groups

<?xml version="1.0" encoding="utf-8"?><!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'><html lang="en" xmlns="http://www.w3.org/1999/xhtml">

<head>

<title>Twisted Documentation: Generating Deferreds</title>

</head>

<h1 class="title">Generating Deferreds</h1>

<div class="toc"><ol><li><a href="#auto0">Class overview</a></li><ul><li><a href="#auto1">Basic Callback Functions</a></li></ul><li><a href="#auto2">What Deferreds don't do: make your code asynchronous</a></li><li><a href="#auto3">Advanced Processing Chain Control</a></li><li><a href="#auto4">Returning Deferreds from synchronous functions</a></li><li><a href="#auto5">Integrating blocking code with Twisted</a></li><li><a href="#auto6">Possible sources of error</a></li><ul><li><a href="#auto7">Firing Deferreds more than once is impossible</a></li><li><a href="#auto8">Synchronous callback execution</a></li></ul></ol></div>

<code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.defer.Deferred.html" title="twisted.internet.defer.Deferred">Deferred</a></code> objects are

signals that a function you have called does not yet have the data you want

available. When a function returns a Deferred object, your calling function

attaches callbacks to it to handle the data when available.

This document addresses the other half of the question: writing functions

that return Deferreds, that is, constructing Deferred objects, arranging for

them to be returned immediately without blocking until data is available, and

firing their callbacks when the data is available.

This document assumes that you are familiar with the asynchronous model used

by Twisted, and with <a href="defer.html" shape="rect">using deferreds returned by functions</a>

.

<h2>Class overview<a name="auto0"/></h2>

This is an overview API reference for Deferred from the point of creating a

Deferred and firing its callbacks and errbacks. It is not meant to be a

substitute for the docstrings in the Deferred class, but can provide

guidelines for its use.

There is a parallel overview of functions used by calling function which

the Deferred is returned to at <a href="defer.html#class" shape="rect">Using Deferreds</a>.

<h3>Basic Callback Functions<a name="auto1"/></h3>

<ul>

<li>

<code class="py-prototype">callback(result)</code>

Run success callbacks with the given result. This

can only be run once. Later calls to this or

<code>errback</code> will raise <code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.defer.AlreadyCalledError.html" title="twisted.internet.defer.AlreadyCalledError">twisted.internet.defer.AlreadyCalledError</a></code>.

If further callbacks or errbacks are added after this

point, addCallbacks will run the callbacks immediately.

</li>

<li>

<code class="py-prototype">errback(failure)</code>

Run error callbacks with the given failure. This can

only be run once. Later calls to this or

<code>callback</code> will raise <code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.defer.AlreadyCalledError.html" title="twisted.internet.defer.AlreadyCalledError">twisted.internet.defer.AlreadyCalledError</a></code>.

If further callbacks or errbacks are added after this

point, addCallbacks will run the callbacks immediately.

</li>

</ul>

<h2>What Deferreds don't do: make your code asynchronous<a name="auto2"/></h2>

Deferreds do not make the code magically not block.

Let's take this function as an example:

<pre class="python"> 1

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

from twisted.internet import defer

117

118

TARGET = 10000

119

120

def largeFibonnaciNumber():

121

# create a Deferred object to return:

122

d = defer.Deferred()

123

124

# calculate the ten thousandth Fibonnaci number

125

126

first = 0

127

second = 1

128

129

for i in xrange(TARGET - 1):

130

new = first + second

131

first = second

132

second = new

133

if i % 100 == 0:

134

print "Progress: calculating the %dth Fibonnaci number" % i

135

136

# give the Deferred the answer to pass to the callbacks:

137

d.callback(second)

138

139

# return the Deferred with the answer:

140

return d

141

142

import time

143

144

timeBefore = time.time()

145

146

# call the function and get our Deferred

147

d = largeFibonnaciNumber()

148

149

timeAfter = time.time()

150

151

print "Total time taken for largeFibonnaciNumber call: %0.3f seconds" %

152

(timeAfter - timeBefore)

153

154

# add a callback to it to print the number

155

156

def printNumber(number):

157

print "The %dth Fibonacci number is %d" % (TARGET, number)

158

159

print "Adding the callback now."

160

161

d.addCallback(printNumber)

162

</pre>

163

164

You will notice that despite creating a Deferred in the

165

<code>largeFibonnaciNumber</code> function, these things happened:

166

<ul>

167

<li>the "Total time taken for largeFibonnaciNumber call" output

168

shows that the function did not return immediately as asynchronous functions

169

are expected to do; and</li>

170

<li>rather than the callback being added before the result was available and

171

called after the result is available, it isn't even added until after the

172

calculation has been completed.</li>

173

</ul>

174

175

The function completed its calculation before returning, blocking the

176

process until it had finished, which is exactly what asynchronous functions

177

are not meant to do. Deferreds are not a non-blocking talisman: they are a

178

signal for asynchronous functions to use to pass results onto

179

callbacks, but using them does not guarantee that you have an asynchronous

180

function.

181

182

183

<h2>Advanced Processing Chain Control<a name="auto3"/></h2>

184

185

<ul>

186

<li>

187

<code class="py-prototype">pause()</code>

188

189

Cease calling any methods as they are added, and do not

190

respond to <code>callback</code>, until

191

<code>self.unpause()</code> is called.

192

</li>

193

194

<li>

195

<code class="py-prototype">unpause()</code>

196

197

If <code>callback</code> has been called on this

198

Deferred already, call all the callbacks that have been

199

added to this Deferred since <code>pause</code> was

200

called.

201

202

Whether it was called or not, this will put this

203

Deferred in a state where further calls to

204

<code>addCallbacks</code> or <code>callback</code> will

205

work as normal.

206

</li>

207

</ul>

208

209

<h2>Returning Deferreds from synchronous functions<a name="auto4"/></h2>

210

211

Sometimes you might wish to return a Deferred from a synchronous function.

212

There are several reasons why, the major two are maintaining API compatibility

213

with another version of your function which returns a Deferred, or allowing

214

for the possiblity that in the future your function might need to be

215

asynchronous.

216

217

In the <a href="defer.html" shape="rect">Using Deferreds</a> reference, we gave the

218

following example of a synchronous function:

219

220

<div class="py-listing"><pre>1

221

222

223

224

225

def synchronousIsValidUser(user):

226

'''

227

Return true if user is a valid user, false otherwise

228

'''

229

return user in ["Alice", "Angus", "Agnes"]

230

</pre><div class="caption">Source listing - <a href="listings/deferred/synch-validation.py">listings/deferred/synch-validation.py</a></div></div>

231

232

While we can require that callers of our function wrap our synchronous

233

result in a Deferred using <code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.defer.maybeDeferred.html" title="twisted.internet.defer.maybeDeferred">maybeDeferred</a></code>, for the sake of API

234

compatibility it is better to return a Deferred ourself using <code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.defer.succeed.html" title="twisted.internet.defer.succeed">defer.succeed</a></code>:

235

236

<pre class="python"> 1

237

238

239

240

241

242

243

244

245

246

247

248

249

250

def immediateIsValidUser(user):

251

'''

252

Returns a Deferred resulting in true if user is a valid user, false

253

otherwise

254

'''

255

256

result = user in ["Alice", "Angus", "Agnes"]

257

258

# return a Deferred object already called back with the value of result

259

return defer.succeed(result)

260

</pre>

261

262

There is an equivalent <code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.defer.fail.html" title="twisted.internet.defer.fail">defer.fail</a></code> method to return a Deferred with the

263

errback chain already fired.

264

265

<h2>Integrating blocking code with Twisted<a name="auto5"/></h2>

266

267

At some point, you are likely to need to call a blocking function: many

268

functions in third party libraries will have long running blocking functions.

269

There is no way to 'force' a function to be asynchronous: it must be written

270

that way specifically. When using Twisted, your own code should be

271

asynchronous, but there is no way to make third party functions asynchronous

272

other than rewriting them.

273

274

In this case, Twisted provides the ability to run the blocking code in a

275

separate thread rather than letting it block your application. The <code class="API"><a href="http://twistedmatrix.com/documents/10.0.0/api/twisted.internet.threads.deferToThread.html" title="twisted.internet.threads.deferToThread">twisted.internet.threads.deferToThread</a></code> function will set up

276

a thread to run your blocking function, return a Deferred and later fire that

277

Deferred when the thread completes.

278

279

Let's assume our <code class="python">largeFibonnaciNumber</code> function

280

from above is in a third party library (returning the result of the

281

calculation, not a Deferred) and is not easily modifiable to be finished in

282

discrete blocks. This example shows it being called in a thread, unlike in the

283

earlier section we'll see that the operation does not block our entire

284

program:

285

286

<pre class="python"> 1

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

def largeFibonnaciNumber():

332

"""

333

Represent a long running blocking function by calculating

334

the TARGETth Fibonnaci number

335

"""

336

TARGET = 10000

337

338

first = 0

339

second = 1

340

341

342

new = first + second

343

first = second

344

second = new

345

346

return second

347

348

from twisted.internet import threads, reactor

349

350

def fibonacciCallback(result):

351

"""

352

Callback which manages the largeFibonnaciNumber result by

353

printing it out

354

"""

355

print "largeFibonnaciNumber result =", result

356

# make sure the reactor stops after the callback chain finishes,

357

# just so that this example terminates

358

reactor.stop()

359

360

def run():

361

"""

362

Run a series of operations, deferring the largeFibonnaciNumber

363

operation to a thread and performing some other operations after

364

adding the callback

365

"""

366

# get our Deferred which will be called with the largeFibonnaciNumber result

367

d = threads.deferToThread(largeFibonnaciNumber)

368

# add our callback to print it out

369

d.addCallback(fibonacciCallback)

370

print "1st line after the addition of the callback"

371

print "2nd line after the addition of the callback"

372

373

if __name__ == '__main__':

374

run()

375

reactor.run()

376

</pre>

377

378

<h2>Possible sources of error<a name="auto6"/></h2>

379

380

Deferreds greatly simplify the process of writing asynchronous code by

381

providing a standard for registering callbacks, but there are some subtle and

382

sometimes confusing rules that you need to follow if you are going to use

383

them. This mostly applies to people who are writing new systems that use

384

Deferreds internally, and not writers of applications that just add callbacks

385

to Deferreds produced and processed by other systems. Nevertheless, it is good

386

to know.

387

388

<h3>Firing Deferreds more than once is impossible<a name="auto7"/></h3>

389

390

Deferreds are one-shot. You can only call <code>Deferred.callback</code> or

391

<code>Deferred.errback</code> once. The processing chain continues each time

392

you add new callbacks to an already-called-back-to Deferred.

393

394

<h3>Synchronous callback execution<a name="auto8"/></h3>

395

396

If a Deferred already has a result available, addCallback

397

may call the callback synchronously: that is, immediately

398

after it's been added. In situations where callbacks modify state, it is

399

might be desirable for the chain of processing to halt until all callbacks are

400

added. For this, it is possible to <code>pause</code> and <code>unpause</code>

401

a Deferred's processing chain while you are adding lots of callbacks.

402

403

Be careful when you use these methods! If you <code>pause</code> a

404

Deferred, it is your responsibility to make sure that you unpause it.

405

The function adding the callbacks must unpause a paused Deferred, it should

406

never be the responsibility of the code that actually fires the

407

callback chain by calling <code>callback</code> or <code>errback</code> as

408

this would negate its usefulness!

409

410

</div>

411

412

<a href="index.html">Index</a>

413

Version: 10.0.0

414

</body>

415

</html>

b'\\ No newline at end of file'

Older »