1
EventMachine (EM) adds two different formalisms for lightweight concurrency
2
to the Ruby programmer's toolbox: spawned processes and deferrables. This
3
note will show you how to use deferrables. For more information, see the
4
separate document LIGHTWEIGHT_CONCURRENCY.
6
=== What are Deferrables?
8
EventMachine's Deferrable borrows heavily from the "deferred" object in
9
Python's "Twisted" event-handling framework. Here's a minimal example that
10
illustrates Deferrable:
12
require 'eventmachine'
15
include EM::Deferrable
18
puts "MyClass instance received #{x}"
30
df.set_deferred_status :succeeded, 100
35
This program will spin for two seconds, print out the string "MyClass
36
instance received 100" and then exit. The Deferrable pattern relies on
37
an unusual metaphor that may be unfamiliar to you, unless you've used
38
Python's Twisted. You may need to read the following material through
39
more than once before you get the idea.
41
EventMachine::Deferrable is simply a Ruby Module that you can include
42
in your own classes. (There also is a class named
43
EventMachine::DefaultDeferrable for when you want to create one without
44
including it in code of your own.)
46
An object that includes EventMachine::Deferrable is like any other Ruby
47
object: it can be created whenever you want, returned from your functions,
48
or passed as an argument to other functions.
50
The Deferrable pattern allows you to specify any number of Ruby code
51
blocks (callbacks or errbacks) that will be executed at some future time
52
when the status of the Deferrable object changes.
54
How might that be useful? Well, imagine that you're implementing an HTTP
55
server, but you need to make a call to some other server in order to fulfill
58
When you receive a request from one of your clients, you can create and
59
return a Deferrable object. Some other section of your program can add a
60
callback to the Deferrable that will cause the client's request to be
61
fulfilled. Simultaneously, you initiate an event-driven or threaded client
62
request to some different server. And then your EM program will continue to
63
process other events and service other client requests.
65
When your client request to the other server completes some time later, you
66
will call the #set_deferred_status method on the Deferrable object, passing
67
either a success or failure status, and an arbitrary number of parameters
68
(which might include the data you received from the other server).
70
At that point, the status of the Deferrable object becomes known, and its
71
callback or errback methods are immediately executed. Callbacks and errbacks
72
are code blocks that are attached to Deferrable objects at any time through
73
the methods #callback and #errback.
75
The deep beauty of this pattern is that it decouples the disposition of one
76
operation (such as a client request to an outboard server) from the
77
subsequent operations that depend on that disposition (which may include
78
responding to a different client or any other operation).
80
The code which invokes the deferred operation (that will eventually result
81
in a success or failure status together with associated data) is completely
82
separate from the code which depends on that status and data. This achieves
83
one of the primary goals for which threading is typically used in
84
sophisticated applications, with none of the nondeterminacy or debugging
85
difficulties of threads.
87
As soon as the deferred status of a Deferrable becomes known by way of a call
88
to #set_deferred_status, the Deferrable will IMMEDIATELY execute all of its
89
callbacks or errbacks in the order in which they were added to the Deferrable.
91
Callbacks and errbacks can be added to a Deferrable object at any time, not
92
just when the object is created. They can even be added after the status of
93
the object has been determined! (In this case, they will be executed
94
immediately when they are added.)
96
A call to Deferrable#set_deferred_status takes :succeeded or :failed as its
97
first argument. (This determines whether the object will call its callbacks
98
or its errbacks.) #set_deferred_status also takes zero or more additional
99
parameters, that will in turn be passed as parameters to the callbacks or
102
In general, you can only call #set_deferred_status ONCE on a Deferrable
103
object. A call to #set_deferred_status will not return until all of the
104
associated callbacks or errbacks have been called. If you add callbacks or
105
errbacks AFTER making a call to #set_deferred_status, those additional
106
callbacks or errbacks will execute IMMEDIATELY. Any given callback or
107
errback will be executed AT MOST once.
109
It's possible to call #set_deferred_status AGAIN, during the execution a
110
callback or errback. This makes it possible to change the parameters which
111
will be sent to the callbacks or errbacks farther down the chain, enabling
112
some extremely elegant use-cases. You can transform the data returned from
113
a deferred operation in arbitrary ways as needed by subsequent users, without
114
changing any of the code that generated the original data.
116
A call to #set_deferred_status will not return until all of the associated
117
callbacks or errbacks have been called. If you add callbacks or errbacks
118
AFTER making a call to #set_deferred_status, those additional callbacks or
119
errbacks will execute IMMEDIATELY.
121
Let's look at some more sample code. It turns out that many of the internal
122
protocol implementations in the EventMachine package rely on Deferrable. One
123
of these is EM::Protocols::HttpClient.
125
To make an evented HTTP request, use the module function
126
EM::Protocols::HttpClient#request, which returns a Deferrable object.
129
require 'eventmachine'
132
df = EM::Protocols::HttpClient.request( :host=>"www.example.com",
133
:request=>"/index.html" )
135
df.callback {|response|
136
puts "Succeeded: #{response[:content]}"
140
df.errback {|response|
141
puts "ERROR: #{response[:status]}"
146
(See the documentation of EventMachine::Protocols::HttpClient for information
147
on the object returned by #request.)
149
In this code, we make a call to HttpClient#request, which immediately returns
150
a Deferrable object. In the background, an HTTP client request is being made
151
to www.example.com, although your code will continue to run concurrently.
153
At some future point, the HTTP client request will complete, and the code in
154
EM::Protocols::HttpClient will process either a valid HTTP response (including
155
returned content), or an error.
157
At that point, EM::Protocols::HttpClient will call
158
EM::Deferrable#set_deferred_status on the Deferrable object that was returned
159
to your program, as the return value from EM::Protocols::HttpClient.request.
160
You don't have to do anything to make this happen. All you have to do is tell
161
the Deferrable what to do in case of either success, failure, or both.
163
In our code sample, we set one callback and one errback. The former will be
164
called if the HTTP call succeeds, and the latter if it fails. (For
165
simplicity, we have both of them calling EM#stop to end the program, although
166
real programs would be very unlikely to do this.)
168
Setting callbacks and errbacks is optional. They are handlers to defined
169
events in the lifecycle of the Deferrable event. It's not an error if you
170
fail to set either a callback, an errback, or both. But of course your
171
program will then fail to receive those notifications.
173
If through some bug it turns out that #set_deferred_status is never called
174
on a Deferrable object, then that object's callbacks or errbacks will NEVER
175
be called. It's also possible to set a timeout on a Deferrable. If the
176
timeout elapses before any other call to #set_deferred_status, the Deferrable
177
object will behave as is you had called set_deferred_status(:failed) on it.
180
Now let's modify the example to illustrate some additional points:
182
require 'eventmachine'
185
df = EM::Protocols::HttpClient.request( :host=>"www.example.com",
186
:request=>"/index.html" )
188
df.callback {|response|
189
df.set_deferred_status :succeeded, response[:content]
192
df.callback {|string|
193
puts "Succeeded: #{string}"
197
df.errback {|response|
198
puts "ERROR: #{response[:status]}"
204
Just for the sake of illustration, we've now set two callbacks instead of
205
one. If the deferrable operation (the HTTP client-request) succeeds, then
206
both of the callbacks will be executed in order.
208
But notice that we've also made our own call to #set_deferred_status in the
209
first callback. This isn't required, because the HttpClient implementation
210
already made a call to #set_deferred_status. (Otherwise, of course, the
211
callback would not be executing.)
213
But we used #set_deferred_status in the first callback in order to change the
214
parameters that will be sent to subsequent callbacks in the chain. In this
215
way, you can construct powerful sequences of layered functionality. If you
216
want, you can even change the status of the Deferrable from :succeeded to
217
:failed, which would abort the chain of callback calls, and invoke the chain
220
Now of course it's somewhat trivial to define two callbacks in the same
221
method, even with the parameter-changing effect we just described. It would
222
be much more interesting to pass the Deferrable to some other function (for
223
example, a function defined in another module or a different gem), that would
224
in turn add callbacks and/or errbacks of its own. That would illustrate the
225
true power of the Deferrable pattern: to isolate the HTTP client-request
226
from other functions that use the data that it returns without caring where
227
those data came from.
229
Remember that you can add a callback or an errback to a Deferrable at any
230
point in time, regardless of whether the status of the deferred operation is
231
known (more precisely, regardless of when #set_deferred_status is called on
232
the object). Even hours or days later.
234
When you add a callback or errback to a Deferrable object on which
235
#set_deferred_status has not yet been called, the callback/errback is queued
236
up for future execution, inside the Deferrable object. When you add a
237
callback or errback to a Deferrable on which #set_deferred_status has
238
already been called, the callback/errback will be executed immediately.
239
Your code doesn't have to worry about the ordering, and there are no timing
240
issues, as there would be with a threaded approach.
242
For more information on Deferrables and their typical usage patterns, look
243
in the EM unit tests. There are also quite a few sugarings (including
244
EM::Deferrable#future) that make typical Deferrable usages syntactically
added
removed
docs/DEFERRABLES
