1
<?xml version="1.0" encoding="iso-8859-1" ?>
2
<!DOCTYPE chapter SYSTEM "chapter.dtd">
9
<holder>Ericsson AB, All Rights Reserved</holder>
12
The contents of this file are subject to the Erlang Public License,
13
Version 1.1, (the "License"); you may not use this file except in
14
compliance with the License. You should have received a copy of the
15
Erlang Public License along with this software. If not, it can be
16
retrieved online at http://www.erlang.org/.
18
Software distributed under the License is distributed on an "AS IS"
19
basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
20
the License for the specific language governing rights and limitations
23
The Initial Developer of the Original Code is Ericsson AB.
26
<title>Distribution Protocol</title>
29
<date>2007-09-21</date>
34
The description here is far from complete and will therefore be further
35
refined in upcoming releases.
37
The protocols both from Erlang nodes towards
38
EPMD (Erlang Port Mapper Daemon) and between Erlang nodes, however, are
39
stable since many years.
42
<p>The distribution protocol can be divided into four (4) parts:</p>
46
1. Low level socket connection.
50
2. Handshake, interchange node name and authenticate.
53
3. Authentication (done by net_kernel).
60
A node fetches the Port number of another node through the EPMD (at the
61
other host) in order to initiate a connection request.
64
For each host where a distributed Erlang node is running there should also
65
be an EPMD running. The EPMD can be started explicitly or automatically
66
as a result of the Erlang node startup.
69
By default EPMD listens on port 4369.
72
3 and 4 are performed at the same level but the net_kernel disconnects the
73
other node if it communicates using an invalid cookie (after one (1) second).
76
<p>The integers in all multi-byte fields are in big-endian order.</p>
79
<title>EPMD Protocol</title>
81
The requests served by the EPMD (Erlang Port Mapper Daemon) are
82
summarized in the figure below.
85
<image file="erl_ext_fig">
87
Summary of EPMD requests.
91
Each request <c>*_REQ</c> is preceeded by a two-byte length field.
92
Thus, the overall request format is:
97
<cell align="center">2</cell>
98
<cell align="center">n</cell>
101
<cell align="center">Length</cell>
102
<cell align="center">Request</cell>
104
<tcaption></tcaption></table>
107
<title>Register a node in the EPMD</title>
109
When a distributed node is started it registers itself in EPMD.
110
The message ALIVE2_REQ described below is sent from the node towards
111
EPMD. The response from EPMD is ALIVE2_RESP.
115
<cell align="center">1</cell>
116
<cell align="center">2</cell>
117
<cell align="center">1</cell>
118
<cell align="center">1</cell>
119
<cell align="center">2</cell>
120
<cell align="center">2</cell>
121
<cell align="center">2</cell>
122
<cell align="center">Nlen</cell>
123
<cell align="center">2</cell>
124
<cell align="center">Elen</cell>
127
<cell align="center">120</cell>
128
<cell align="center">PortNo</cell>
129
<cell align="center">NodeType</cell>
130
<cell align="center">Protocol</cell>
131
<cell align="center">LowestVersion</cell>
132
<cell align="center">HighestVersion</cell>
133
<cell align="center">Nlen</cell>
134
<cell align="center">NodeName</cell>
135
<cell align="center">Elen</cell>
136
<cell align="center">Extra</cell>
138
<tcaption>ALIVE2_REQ (120)</tcaption></table>
140
<tag><c>PortNo</c></tag>
142
The port number on which the node accept connection requests.
144
<tag><c>NodeType</c></tag>
146
77 = normal Erlang node, 72 = hidden node (C-node),...
148
<tag><c>Protocol</c></tag>
152
<tag><c>LowestVersion</c></tag>
154
The lowest distribution version that this node can handle.
155
See the next field for possible values.
157
<tag><c>HighestVersion</c></tag>
159
The highest distribution version that this node can handle.
160
The value in R6B and later is 5.
162
<tag><c>Nlen</c></tag>
164
The length of the <c>NodeName</c>.
166
<tag><c>NodeName</c></tag>
168
The NodeName as a string of length <c>Nlen</c>.
170
<tag><c>Elen</c></tag>
172
The length of the <c>Extra</c> field.
174
<tag><c>Extra</c></tag>
176
Extra field of <c>Elen</c> bytes.
180
The connection created to the EPMD must be kept as long as the
181
node is a distributed node. When the connection is closed
182
the node is automatically unregistered from the EPMD.
185
The response message ALIVE2_RESP is described below.
190
<cell align="center">1</cell>
191
<cell align="center">1</cell>
192
<cell align="center">2</cell>
195
<cell align="center">121</cell>
196
<cell align="center">Result</cell>
197
<cell align="center">Creation</cell>
199
<tcaption>ALIVE2_RESP (121)</tcaption></table>
201
Result = 0 -> ok, Result > 0 -> error
206
<title>Unregister a node from the EPMD</title>
208
A node unregister itself from the EPMD by simply closing the
209
TCP connection towards EPMD established when the node was registered.
214
<title>Get the distribution port of another node</title>
216
When one node wants to connect to another node it starts with
217
a PORT_PLEASE2_REQ request towards EPMD on the host where the
218
node resides in order to get the distribution port that the node
224
<cell align="center">1</cell>
225
<cell align="center">N</cell>
228
<cell align="center">122</cell>
229
<cell align="center">NodeName</cell>
231
<tcaption>PORT_PLEASE2_REQ (122)</tcaption></table>
240
<cell align="center">1</cell>
241
<cell align="center">1</cell>
244
<cell align="center">119</cell>
245
<cell align="center">Result</cell>
248
PORT2_RESP (119) response indicating error, Result > 0.
254
<cell align="center">1</cell>
255
<cell align="center">1</cell>
256
<cell align="center">2</cell>
257
<cell align="center">1</cell>
258
<cell align="center">1</cell>
259
<cell align="center">2</cell>
260
<cell align="center">2</cell>
261
<cell align="center">2</cell>
262
<cell align="center">Nlen</cell>
263
<cell align="center">2</cell>
264
<cell align="center">Elen</cell>
267
<cell align="center">119</cell>
268
<cell align="center">Result</cell>
269
<cell align="center">PortNo</cell>
270
<cell align="center">NodeType</cell>
271
<cell align="center">Protocol</cell>
272
<cell align="center">HighestVersion</cell>
273
<cell align="center">LowestVersion</cell>
274
<cell align="center">Nlen</cell>
275
<cell align="center">NodeName</cell>
276
<cell align="center">Elen</cell>
277
<cell align="center">Extra</cell>
279
<tcaption>PORT2_RESP when Result = 0.</tcaption></table>
281
If Result > 0, the packet only consists of [119, Result].
284
<p>EPMD will close the socket as soon as it has sent the information.</p>
288
<title>Get all registered names from EPMD</title>
290
This request is used via the Erlang function
291
<c>net_adm:names/1,2</c>. A TCP connection is opened
292
towards EPMD and this request is sent.
296
<cell align="center">1</cell>
299
<cell align="center">110</cell>
301
<tcaption>NAMES_REQ (110)</tcaption></table>
304
<p>The response for a <c>NAMES_REQ</c> looks like this:</p>
307
<cell align="center">4</cell>
308
<cell align="center"> </cell>
311
<cell align="center">EPMDPortNo</cell>
312
<cell align="center">NodeInfo*</cell>
314
<tcaption>NAMES_RESP</tcaption></table>
316
NodeInfo is a string written for each active node.
317
When all NodeInfo has been written the connection is
321
NodeInfo is, as expressed in Erlang:
324
io:format("name ~s at port ~p~n", [NodeName, Port]).
330
<title>Dump all data from EPMD</title>
332
This request is not really used, it should be regarded as a debug
337
<cell align="center">1</cell>
340
<cell align="center">100</cell>
342
<tcaption>DUMP_REQ</tcaption></table>
344
<p>The response for a <c>DUMP_REQ</c> looks like this:</p>
347
<cell align="center">4</cell>
348
<cell align="center"> </cell>
351
<cell align="center">EPMDPortNo</cell>
352
<cell align="center">NodeInfo*</cell>
354
<tcaption>DUMP_RESP</tcaption></table>
356
NodeInfo is a string written for each node kept in EPMD.
357
When all NodeInfo has been written the connection is
361
NodeInfo is, as expressed in Erlang:
364
io:format("active name ~s at port ~p, fd = ~p ~n",
365
[NodeName, Port, Fd]).
371
io:format("old/unused name ~s at port ~p, fd = ~p~n",
372
[NodeName, Port, Fd]).
378
<title>Kill the EPMD</title>
380
This request will kill the running EPMD. It is almost never used.
384
<cell align="center">1</cell>
387
<cell align="center">107</cell>
389
<tcaption>KILL_REQ</tcaption></table>
391
<p>The response fo a <c>KILL_REQ</c> looks like this:</p>
394
<cell align="center">2</cell>
397
<cell align="center">OKString</cell>
399
<tcaption>KILL_RESP</tcaption></table>
401
where <c>OKString</c> is "OK".
406
<title>STOP_REQ (Not Used)</title>
410
<cell align="center">1</cell>
411
<cell align="center">n</cell>
414
<cell align="center">115</cell>
415
<cell align="center">NodeName</cell>
417
<tcaption>STOP_REQ</tcaption></table>
422
The current implementation of Erlang does not care if the connection
423
to the EPMD is broken.
425
<p>The response for a <c>STOP_REQ</c> looks like this.</p>
428
<cell align="center">7</cell>
431
<cell align="center">OKString</cell>
433
<tcaption>STOP_RESP</tcaption></table>
435
where OKString is "STOPPED".
437
<p>A negative response can look like this.</p>
440
<cell align="center">7</cell>
443
<cell align="center">NOKString</cell>
445
<tcaption>STOP_NOTOK_RESP</tcaption></table>
447
where NOKString is "NOEXIST".
452
<title>ALIVE_REQ (97)</title>
457
<cell align="center">1</cell>
458
<cell align="center">2</cell>
459
<cell align="center">n</cell>
462
<cell align="center">97</cell>
463
<cell align="center">PortNo</cell>
464
<cell align="center">NodeName</cell>
466
<tcaption></tcaption></table>
472
The connection created to the EPMD must be kept until the node is
473
not a distributed node any longer.
478
<title>ALIVE_OK_RESP (89)</title>
482
<cell align="center">1</cell>
483
<cell align="center">2</cell>
486
<cell align="center">89</cell>
487
<cell align="center">Creation</cell>
489
<tcaption></tcaption></table>
494
<title>ALIVE_NOTOK_RESP ()</title>
496
EPMD closed the connection.
501
<title>PORT_PLEASE_REQ (112)</title>
505
<cell align="center">1</cell>
506
<cell align="center">n</cell>
509
<cell align="center">112</cell>
510
<cell align="center">NodeName</cell>
512
<tcaption></tcaption></table>
519
<title>PORT_OK_RESP ()</title>
523
<cell align="center">2</cell>
526
<cell align="center">PortNo</cell>
528
<tcaption></tcaption></table>
533
<title>PORT_NOTOK_RESP ()</title>
535
EPMD closed the connection.
541
<title>PORT_NOTOK_RESP ()</title>
543
EPMD closed the connection.
551
<title>Handshake</title>
553
The handshake is discussed in detail in the internal documentation for
554
the kernel (Erlang) application.
559
<title>Protocol between connected nodes</title>
562
<cell align="center">4</cell>
563
<cell align="center">1</cell>
564
<cell align="center">n</cell>
565
<cell align="center">m</cell>
568
<cell align="center">Length</cell>
569
<cell align="center">Type</cell>
570
<cell align="center">ControlMsg</cell>
571
<cell align="center">Message</cell>
573
<tcaption></tcaption></table>
578
Length is equal to 1 + n + m
580
Type is: 112 - pass through
582
ControlMsg is a tuple passed using the external format of
585
Message is the message sent to another node using the '!'
586
(in external format).
587
But, Message is only passed in combination with a ControlMsg
588
encoding a send ('!').
592
The control message is a tuple, where the first element indicates
593
which distributed operation it encodes.
596
<tag><c>LINK</c></tag>
612
<note><p>Message is sent as well.</p></note>
618
{3, FromPid, ToPid, Reason}
639
{6, FromPid, Cookie, ToName}
643
<note><p>Message is sent as well.</p></note>
646
<tag>GROUP_LEADER</tag>
656
{8, FromPid, ToPid, Reason}
664
<title>New Ctrlmessages for distrvsn = 1 (OTP R4)</title>
666
<title>SEND_TT</title>
668
{12, Cookie, ToPid, TraceToken}
671
<note><p>Message is sent as well.</p></note>
675
<title>EXIT_TT</title>
677
{13, FromPid, ToPid, TraceToken, Reason}
683
<title>REG_SEND_TT</title>
685
{16, FromPid, Cookie, ToName, TraceToken}
688
<note><p>Message is sent as well.</p></note>
692
<title>EXIT2_TT</title>
694
{18, FromPid, ToPid, TraceToken, Reason}
700
<title>New Ctrlmessages for distrvsn = 2</title>
702
distrvsn 2 was never used.
707
<title>New Ctrlmessages for distrvsn = 3 (OTP R5C)</title>
709
None, but the version number was increased anyway.
714
<title>New Ctrlmessages for distrvsn = 4 (OTP R6)</title>
716
These are only recognized by Erlang nodes, not by hidden nodes.
719
<title>MONITOR_P</title>
721
{19, FromPid, ToProc, Ref}
723
FromPid = monitoring process
724
ToProc = monitored process pid or name (atom)
729
<title>DEMONITOR_P</title>
731
{20, FromPid, ToProc, Ref}
732
We include the FromPid just in case we want to trace this.
734
FromPid = monitoring process
735
ToProc = monitored process pid or name (atom)
740
<title>MONITOR_P_EXIT</title>
742
{21, FromProc, ToPid, Ref, Reason}
744
FromProc = monitored process pid or name (atom)
745
ToPid = monitoring process
746
Reason = exit reason for the monitored process