1
{1 Advanced features of Netplex}
3
Some information about advanced techniques.
7
- {!Netplex_advanced.timers}
8
- {!Netplex_advanced.contvars}
9
- {!Netplex_advanced.contsocks}
10
- {!Netplex_advanced.initonce}
11
- {!Netplex_advanced.sharedvars}
12
- {!Netplex_advanced.passdown}
13
- {!Netplex_advanced.levers}
15
{2:timers Running timers in containers}
17
With {!Netplex_cenv.create_timer} one can start a timer that runs directly
18
in the container event loop. This event loop is normally used for accepting
19
new connections, and for exchanging control messages with the master
20
process. If the processor supports it (like the RPC processor), the
21
event loop is also used by the processor itself for
22
protocol interpretation. Running a timer in this loop means that the
23
expiration of the timer is first detected when the control flow of the
24
container returns to the event loop. In the worst case, this happens only
25
when the current connection is finished, and it is waited for the next
28
So, this is not the kind of high-precision timer one would use for the
29
exact control of latencies. However, these timers are still useful for
30
things that run only infrequently, like
32
- processing statistical information
33
- checking whether configuration updates have arrived
34
- checking whether resources have "timed out" and can be released
35
(e.g. whether a connection to a database system can be closed)
37
Timers can be cancelled by called {!Netplex_cenv.cancel_timer}. Timers
38
are automatically cancelled at container shutdown time.
40
Example: Start a timer at container startup: We have to do this in the
41
[post_start_hook] of the processor. It depends on the kind of
42
processor how the hooks are set. For example, the processor factories
43
{!Rpc_netplex.rpc_factory} and {!Nethttpd_plex.nethttpd_factory} have
44
an argument [hooks], and one can create it like:
49
inherit Netplex_kit.empty_processor_hooks()
50
method post_start_hook cont =
52
Netplex_cenv.create_timer
61
{2:contvars Container variables}
63
If multi-processing is used, one can simply store per-container values
64
in global variables. This works because for every new container the
65
whole program is forked, and thus a new instance of the variable is
68
For multi-threaded programs this is a lot more difficult. For this
69
reason there is built-in support for per-container variables.
71
Example: We want to implement a statistics how often the functions
72
[foo] and [bar] are called, per-container. We define a record
76
{ mutable foo_count : int;
77
mutable bar_count : int
81
Furthermore, we need an access module that looks for the current value
82
of the variable (get), or overwrites the value (set). We can simply
83
create this module by using the functor {!Netplex_cenv.Make_var_type}:
87
Netplex_cenv.Make_var_type(struct type t = stats end)
90
Now, one can get the value of a [stats]-typed variable "count" by
98
(which will raise {!Netplex_cenv.Container_variable_not_found} if the
99
value of "count" never has been set before), and one can set the value
103
Stats_var.set "count" stats
106
As mentioned, the variable "count" exists once per container. One
107
can access it only from the scope of a Netplex container (e.g. from a
108
callback function that is invoked by a Netplex processor). It is a
109
good idea to initialize "count" in the [post_start_hook] of the
110
processor (see the timer example above).
112
See also below on "Storing global state" for another kind of variable that
113
can be accessed from all containers.
116
{2:contsocks Sending messages to individual containers}
118
Sometimes it is useful when a container can directly communicate with
119
another container, and the latter can be addressed by a unique name
120
within the Netplex system. A normal Netplex socket is not useful here
121
because Netplex determines which container will accept new connections
122
on the socket, i.e. from the perspective of the message sender it is
123
random which container receives the message.
125
In Ocamlnet 3, a special kind of socket, called "container socket" has
126
been added to solve this problem. This type of socket is not created by
127
the master process, but by the container process (hence the name). The
128
socket is a Unix Domain socket for Unix, and a named pipe for Win32.
129
It has a unique name, and if the message sender knows the name, it can
130
send the message to a specific container.
132
One creates such sockets by adding an [address] section to the config
141
If this [address] section is simply added to an existing [protocol]
142
section, the network protocol of the container socket is the same as
143
that of the main socket of the container. If a different network protocol
144
is going to be used for the container socket, one can also add a second
145
[protocol] section. For example, here is a main HTTP service, and a
146
separate service [control] that is run over the container sockets:
166
http { ... webserver config ... }
167
control { ... rpc config ... }
172
One can now employ {!Netplex_kit.protocol_switch_factory} to route
173
incoming TCP connections arriving at "http" sockets to web server
174
code, and to route incoming TCP connections arriving at "control"
175
sockets to a e.g. an RPC server:
178
let compound_factory =
179
new Netplex_kit.protocol_switch_factory
181
[ "http", Nethttpd_plex.nethttpd_factory ...;
182
"control", Rpc_netplex.rpc_factory ...;
186
The implementation of "control" would be a normal RPC server.
188
The remaining question is now how to get the unique names of the
189
container sockets. There is the function
190
{!Netplex_cenv.lookup_container_sockets} helping here. The function
191
is called with the service name and the protocol name as arguments:
195
Netplex_cenv.lookup_container_sockets "sample" "control"
198
It returns an array of Unix Domain paths, each corresponding to the
199
container socket of one container. It is recommended to use
200
{!Netplex_sockserv.any_file_client_connector} for creating RPC
207
let connector = Netplex_sockserv.any_file_client_connector cs_path in
208
create_client ... connector ...
213
There is no way to get more information about the [cs_paths], e.g.
214
in order to find a special container. (Of course, except by calling RPC
215
functions and asking the containers directly.)
217
A container can also find out the address of its own container socket.
218
Use the method [owned_container_sockets] to get a list of pairs
219
[(protocol_name, path)], e.g.
222
let cont = Netplex_cenv.self_cont() in
223
let path = List.assoc "control" cont#owned_container_sockets
228
{2:initonce One-time initialization code}
230
It is sometimes necessary to run some initialization code only once
231
for all containers of a certain service. Of course, there is always
232
the option of doing this at program startup. However, this might be
233
too early, e.g. because some information is not yet known.
235
Another option is to do such initialization in the [pre_start_hook] of
236
the container. The [pre_start_hook] is run before the container process
237
is forked off, and executes in the master process. Because of this it is
238
easy to have a global variable that checks whether [pre_start_hook] is
239
called the first time:
242
let first_time = ref true
244
let pre_start_hook _ _ _ =
245
if !first_time then (* do initialization *) ... ;
250
inherit Netplex_kit.empty_processor_hooks()
251
method pre_start_hook socksrv ctrl cid =
252
pre_start_hook socksrv ctrl cid
257
Last but not least there is also the possibility to run such
258
initialization code in the [post_start_hook]. This is different as
259
this hook is called from the container, i.e. from the forked-off child
260
process. This might be convenient if the initialization routine is
261
written for container context.
263
There is some additional complexity, though. One can no longer simply
264
use a global variable to catch the first time [post_start_hook] is
265
called. Instead, one has to use a storage medium that is shared by all
266
containers, and that is accessible from all containers. There are
267
plenty of possibilities, e.g. a file. In this example, however, we use
273
inherit Netplex_kit.empty_processor_hooks()
275
method post_add_hook socksrv ctrl =
276
ctrl # add_plugin Netplex_semaphore.plugin
278
method post_start_hook cont =
280
Netplex_semaphore.create "myinit" 0L in
281
if first_time then (* do initialization *) ... ;
286
The semaphore is visible in the whole Netplex system. We use here the
287
fact that {!Netplex_semaphore.create} returns [true] when the semaphore
288
is created at the first call of [create]. The semaphore is then never
289
increased or decreased.
292
{2:sharedvars Storing global state}
294
Sometimes global state is unavoidable. We mean here state variables
295
that are accessed by all processes of the Netplex system.
297
Since Ocamlnet 3 there is {!Netplex_sharedvar}. This modules provides
298
Netplex-global string variables that are identified by a user-chosen
301
For example, to make a variable of [type stats] globally accessible
305
{ mutable foo_count : int;
306
mutable bar_count : int
310
(see also above, "Container variables"), we can accomplish this as
315
Netplex_sharedvar.Make_var_type(struct type t = stats end)
318
Now, this defines functions [Stats_var.get] and [Stats_var.set] to
319
get and set the value, respectively. Note that this is type-safe
320
although {!Netplex_sharedvar.Make_var_type} uses the [Marshal] module
321
internally. If a get/set function is applied to a variable of the
322
wrong type we will get the exception
323
{!Netplex_sharedvar.Sharedvar_type_mismatch}.
325
Before one can get/set values, one has to create the variable with
329
Netplex_sharedvar.create ~enc:true name
332
The parameter [enc:true] is required for variables accessed via
333
{!Netplex_sharedvar.Make_var_type}.
335
In order to use {!Netplex_sharedvar} we have to add this plugin:
340
inherit Netplex_kit.empty_processor_hooks()
342
method post_add_hook socksrv ctrl =
343
ctrl # add_plugin Netplex_sharedvar.plugin
348
Now, imagine that we want to increase the counters in a [stats]
349
variable. As we have now truly parallel accesses, we have to
350
ensure that these accesses do not overlap. We use a Netplex
351
mutex to ensure this like in:
354
let mutex = Netplex_mutex.access "mymutex" in
355
Netplex_mutex.lock mutex;
357
let v = Stats_var.get "mystats" in
358
v.foo_count <- v.foo_count + foo_delta;
359
v.bar_count <- v.bar_count + bar_delta;
360
Stats_var.set "mystats" v;
361
Netplex_mutex.unlock mutex;
363
error -> Netplex_mutex.unlock mutex; raise error
366
As Netplex mutexes are also plugins, we have to add them in the
367
[post_add_hook], too. Also see {!Netplex_mutex} for more information.
369
Generally, shared variables should not be used to store large
370
quantities of data. A few megabytes are probably ok. The reason is
371
that these variables exist in the Netplex master process, and each
372
time a child is forked off the variables are also copied although this
373
is not necessary. (It is possible and likely that a future version of
374
Ocamnet improves this.)
376
For bigger amounts of data, it is advised to store them in an external
377
file, a shared memory segment ({!Netshm} might help here), or even in
378
a database system. Shared variables should then only be used to
379
pass around the name of this file/segment/database.
382
{2:passdown Hooks, and how to pass values down}
384
Usually, the user configures processor factories by creating hook
385
objects. We have shown this already several times in previous
386
sections of this chapter. Sometimes the question arises how to pass
387
values from one hook to another.
389
The hooks are called in a certain order. Unfortunately, there is
390
no easy way to pass values from one hook to another. As workaround,
391
it is suggested to store the values in the hooks object.
393
For example, consider we need to allocate a database ID for each
394
container. We do this in the [pre_start_hook], so we know the ID
395
early. Of course, the code started from the [post_start_hook] also
396
needs the ID, and in the [post_finish_hook] we would like to delete
397
everything in the database referenced by this ID.
399
This could be done in a hook object like
404
inherit Netplex_kit.empty_processor_hooks()
406
val db_id_tbl = Hashtbl.create 11
408
method pre_start_hook _ _ cid =
409
let db_id = allocate_db_id() in (* create db ID *)
410
Hashtbl.add db_id_tbl cid db_id (* remember it for later *)
412
method post_start_hook cont =
413
let cid = cont # container_id in (* the container ID *)
414
let db_id = Hashtbl.find db_id_tbl cid in (* look up the db ID *)
417
method post_finish_hook _ _ cid =
418
let db_id = Hashtbl.find db_id_tbl cid in (* look up the db ID *)
419
delete_db_id db_id; (* clean up db *)
420
Hashtbl.remove db_id_tbl cid
425
We use here the container ID to identify the container. This works in
426
all used hooks - either the container ID is passed directly, or we can
427
get it from the container object itself.
429
Normally there is only one controller per program. It is imaginable that
430
a multi-threaded program has several controllers, though. In this case
431
one has to be careful with this technique, because it should be avoided
432
that values from the Netplex system driven by one controller are visible
433
in the system driven by the other controller. Often, this can be easily
434
achieved by creating separate hook objects, one per controller.
437
{2:levers Levers - calling controller functions from containers}
439
In a multi-process setup, the controller runs in the master process,
440
and the containers run in child processes. Because of this, container
441
code cannot directly invoke functions of the controller.
443
For multi-threaded programs, this is quite easy to solve. With the
444
function {!Netplex_cenv.run_in_controller_context} it can be
445
temporarily switched to the controller thread to run code there.
447
For example, to start a helper container one can do
450
Netplex_cenv.run_in_controller_context ctrl
452
Netplex_kit.add_helper_service ctrl "helper1" hooks
456
which starts a new container with an empty processor that only consists
457
of the [hooks] object. The [post_start_hook] can be considered as the
458
"body" of the new thread. The advantage of this is (compared to
459
[Thread.start]) that this thread counts as a regular container, and
460
can e.g. use logging functions.
462
There is no such easy way in the multi-processing case. As a
463
workaround, a special mechanism has been added to Netplex, the
464
so-called {b levers}. Levers are registered functions that are known
465
to the controller and which can be invoked from container context.
466
Levers have an argument and can deliver a result. The types of
467
argument and result can be arbitrary (but must be monomorphic, and
468
must not contain functions). (The name, lever, was chosen because
469
it reminds of additional operating handles, as we add such handles
472
Levers are usually registered in the [post_add] hook of the processor.
473
For example, let us define a lever that can start a helper container.
474
As arguments we pass a tuple of a string and an int [(s,i)]. The
475
arguments do not have any meaning here, we only do this to demonstrate
476
how to pass arguments. As result, we pass a boolean value back that
477
says whether the helper container was started successfully.
479
First we need to create a type module:
483
type s = string * int (* argument type *)
484
type r = bool (* result type *)
488
As second step, we need to create the lever module. This means only to
489
apply the functor {!Netplex_cenv.Make_lever}:
492
module L = Netplex_cenv.Make_lever(T)
495
What happens behind the scene is that a function [L.register] is
496
created that can marshal the argument and result values from the
497
container process to the master process and back. This is invisible
498
to the user, and type-safe.
500
Now, we have to call [L.register] from the [post_add_hook]. The result
501
of [L.register] is another function that represents the lever. By
502
calling it, the lever is activated:
507
inherit Netplex_kit.empty_processor_hooks()
509
method post_add_hook socksrv ctrl =
514
Netplex_kit.add_helper_service ctrl "helper1" ...;
515
true (* successful *)
517
false (* not successful *)
524
So, when we call [lever ("X",42)] from the container, the lever
525
mechanism routes this call to the controller process, and calls there
526
the function [(fun (s,i) -> ...)] that is the argument of
529
Finally, the question is how can we make the function [lever] known to
530
containers. The hackish way to do this is to store [lever] in a global
531
variable. The clean way is to store [lever] in a container variable,
536
module LV = Netplex_cenv.Make_var_type(L)
537
(* This works because L.t is the type of the lever *)
541
inherit Netplex_kit.empty_processor_hooks()
543
val mutable helper1_lever = (fun _ -> assert false)
545
method post_add_hook socksrv ctrl =
550
Netplex_kit.add_helper_service ctrl "helper1" ...;
551
true (* successful *)
553
false (* not successful *)
555
helper1_lever <- lever
557
method post_start_hook cont =
558
LV.set "helper1_lever" helper1_lever
563
and later in container code:
566
let helper1_lever = LV.get "helper1_lever" in
567
let success = helper1_lever ("X",42) in
569
print_endline "OK, started the new helper"
571
print_endline "There was an error"
added
removed
src/netplex/netplex_advanced.txt
