1
<?xml version="1.0" encoding="latin1" ?>
2
<!DOCTYPE chapter SYSTEM "chapter.dtd">
7
<year>2002</year><year>2010</year>
8
<holder>Ericsson AB. All Rights Reserved.</holder>
11
The contents of this file are subject to the Erlang Public License,
12
Version 1.1, (the "License"); you may not use this file except in
13
compliance with the License. You should have received a copy of the
14
Erlang Public License along with this software. If not, it can be
15
retrieved online at http://www.erlang.org/.
17
Software distributed under the License is distributed on an "AS IS"
18
basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
19
the License for the specific language governing rights and limitations
24
<title>Description</title>
25
<prepared>Håkan Mattsson</prepared>
26
<responsible>Håkan Mattsson</responsible>
28
<approved>Håkan Mattsson</approved>
32
<file>et_desc.xml</file>
36
<title>Overview</title>
38
<p>The two major components of the <c>Event Tracer (ET)</c> tool
39
is a graphical sequence chart viewer (<c>et_viewer</c>) and its
40
backing storage (<c>et_collector</c>). One <c>Collector</c> may be
41
used as backing storage for several simultaneous <c>Viewers</c>
42
where each one may display a different view of the same trace
45
<p>The interface between the <c>Collector</c> and its
46
<c>Viewers</c> is public in order to enable other types of
47
<c>Viewers</c>. However in the following text we will focus on
48
usage of the <c>et_viewer</c>.</p>
50
<p>The main start function is <c>et_viewer:start/1</c>. By
51
default it will start both an <c>et_collector</c> and an
56
<code type="none"><![CDATA[
58
Erlang R13B03 (erts-5.7.4) [64-bit] [smp:4:4] [rq:4]
59
[async-threads:0] [kernel-poll:false]
61
Eshell V5.7.4 (abort with ^G)
62
1> {ok, Viewer} = et_viewer:start([]).
63
{ok,<0.40.0>}]]></code>
65
<p>A <c>Viewer</c> gets trace <c>Events</c> from its
66
<c>Collector</c> by polling it regularly for more <c>Events</c> to
67
display. <c>Events</c> are for example reported to the
68
<c>Collector</c> with <c>et_collector:report_event/6</c>:</p>
70
<code type="none"><![CDATA[
71
2> Collector = et_viewer:get_collector_pid(Viewer).
73
3> et_collector:report_event(Collector, 60, my_shell, mnesia_tm, start_outer,
74
3> "Start outer transaction"),
75
3> et_collector:report_event(Collector, 40, mnesia_tm, my_shell, new_tid,
76
3> "New transaction id is 4711"),
77
3> et_collector:report_event(Collector, 20, my_shell, mnesia_locker, try_write_lock,
78
3> "Acquire write lock for {my_tab, key}"),
79
3> et_collector:report_event(Collector, 10, mnesia_locker, my_shell, granted,
80
3> "You got the write lock for {my_tab, key}"),
81
3> et_collector:report_event(Collector, 60, my_shell, do_commit,
82
3> "Perform transaction commit"),
83
3> et_collector:report_event(Collector, 40, my_shell, mnesia_locker, release_tid,
84
3> "Release all locks for transaction 4711"),
85
3> et_collector:report_event(Collector, 60, my_shell, mnesia_tm, delete_transaction,
86
3> "End of outer transaction"),
87
3> et_collector:report_event(Collector, 20, my_shell, end_outer,
88
3> "Transaction returned {atomic, ok}").
89
{ok,{table_handle,<0.39.0>,16402,trace_ts,
90
#Fun<et_collector.0.62831470>}}]]></code>
92
<p>This actually is a simulation of the process <c>Events</c>
93
caused by a <c>Mnesia</c> transaction that writes a record in a local
96
<code type="none"><![CDATA[
97
mnesia:transaction(fun() -> mnesia:write({my_tab, key, val}) end).]]></code>
99
<p>At this stage when we have a couple of <c>Events</c>, it is time to
100
show how it looks like in the graphical interface of
101
<c>et_viewer</c>:</p>
105
<image file="sim_trans.png">
106
<icaption>A simulated Mnesia transaction which writes one record</icaption>
109
<p>In the sequence chart, the actors (which symbolically has
110
performed the <c>Event</c>) are shown as named vertical bars. The
111
order of the actors may be altered by dragging (hold mouse button
112
1 pressed during the operation) the name tag of an actor and drop
115
<image file="sim_trans_move_actor.png">
116
<icaption>Two actors has switched places</icaption>
119
<p>An <c>Event</c> may be an action performed by one single actor
120
(blue text label) or it may involve two actors and is then
121
depicted as an arrow directed from one actor to another (red text
122
label). Details of an <c>Event</c> can be shown by clicking (press
123
and release the mouse button 1) on the event label text or on the
124
arrow. When doing that a <c>Contents Viewer</c> window pops up. It
125
may look like this:</p>
129
<image file="sim_trans_write_lock.png">
130
<icaption>Details of a write lock message</icaption>
135
<title>Filters and dictionary</title>
137
<p>The <c>Event Tracer (ET)</c> uses named filters in various
138
contexts. An Event Trace filter is an <c>Erlang fun</c> that takes
139
some trace data as input and returns a possibly modified version
145
<code type="none"><![CDATA[
146
filter(TraceData) -> false | true | {true, NewEvent}
148
TraceData = Event | erlang_trace_data()
150
NewEvent = #event{}]]></code>
152
<p>The interface of the filter function is the same as the the
153
filter functions for the good old <c>lists:zf/2</c>. If the filter
154
returns <c>false</c> it means that the trace data should silently
155
be dropped. <c>true</c> means that the trace data data already is
156
an <c>Event Record</c> and that it should be kept as it is.
157
<c>true</c> means that the <c>TraceData</c> already is an <c>Event
158
Record</c> and that it should be kept as it is. <c>{true,
159
NewEvent}</c> means that the original trace data should be
160
replaced with <c>Event</c>. This provides means to get rid of
161
unwanted <c>Events</c> as well as enabling alternate views of an
164
<p>The first filter that the trace data is exposed for is the
165
<c>Collector Filter</c>. When a trace <c>Event</c> is reported with
166
<c>et_collector:report/2</c> (or
167
<c>et_collector:report_event/5,6</c>) the first thing that
168
happens, is that a message is sent to the <c>Collector</c> process
169
to fetch a handle that contains some useful stuff, such as the
170
<c>Collector Filter Fun</c> and an Ets table identifier. Then the
171
<c>Collector Filter Fun</c> is applied and if it returns
172
<c>true</c> (or <c>{true, NewEvent}</c>), the <c>Event</c> will be stored
173
in an Ets table. As an optimization, subsequent calls to
174
<c>et_collector:report</c>-functions can use the handle directly
175
instead of the <c>Collector Pid</c>.</p>
177
<p>All filters (registered in a <c>Collector</c> or in a
178
<c>Viewer</c>) must be able to handle an <c>Event record</c> as
179
input. The <c>Collector Filter</c> (that is the filter named
180
<c>all</c>) is a little bit special, as its input also may be raw
181
<c>Erlang Trace Data</c></p>
183
<p>The <c>Collector</c> manages a key/value based dictionary,
184
where the filters are stored. Updates of the dictionary is
185
propagated to all subscribing processes. When a <c>Viewer</c> is
186
started it is registered as a subscriber of dictionary
189
<p>In each <c>Viewer</c> there is only one filter that is active
190
and all trace <c>Events</c> that the <c>Viewer</c> gets from the
191
<c>Collector</c> will pass thru that filter. By writing clever
192
filters it is possible to customize how the <c>Events</c> looks
193
like in the viewer. The following filter in
194
<c>et/examples/et_demo.erl</c> replaces the actor names
195
<c>mnesia_tm</c> and <c>mnesia_locker</c> and leaves everything
196
else in the record as it was:</p>
200
<codeinclude file="../../examples/et_demo.erl" tag="%mgr_actors" type="erl"></codeinclude>
201
<p>If we now add the filter to the running <c>Collector</c>:</p>
205
<code type="none"><![CDATA[
206
4> Fun = fun(E) -> et_demo:mgr_actors(E) end.
207
#Fun<erl_eval.6.13229925>
208
5> et_collector:dict_insert(Collector, {filter, mgr_actors}, Fun).
211
<p>you will see that the <c>Filter</c> menu in all viewers have
212
got a new entry called <c>mgr_actors</c>. Select it, and a new
213
<c>Viewer</c> window will pop up:</p>
217
<image file="sim_trans_mgr_actors.png">
218
<icaption>The same trace data in a different view</icaption>
221
<p>In order to see the nitty gritty details of an <c>Event</c> you
222
may click on the <c>Event</c> in order to start a <c>Contents
223
Viewer</c> for that <c>Event</c>. In the <c>Contents Viewer</c>
224
there also is a filter menu that enables inspection of the
225
<c>Event</c> from other views than the one selected in the
226
viewer. A click on the <c>new_tid</c> <c>Event</c> will cause a
227
<c>Contents Viewer</c> window to pop up, showing the <c>Event</c>
228
in the <c>mgr_actors</c> view:</p>
232
<image file="sim_trans_contents_viewer_mgr_actors.png">
233
<icaption>The trace <c>Event</c> in the mgr_actors view</icaption>
236
<p>Select the <c>all</c> entry in the <c>Filters</c> menu
237
and a new <c>Contents Viewer window</c> will pop up showing the
238
same trace <c>Event</c> in the collectors view:</p>
242
<image file="sim_trans_contents_viewer_collector.png">
243
<icaption>The same trace <c>Event</c> in the collectors
249
<title>Trace clients</title>
250
<p>As you have seen, it is possible to use the
251
<c>et_collector:report_event/5,6</c> functions explicitly. By
252
using those functions you can write your own trace client that
253
reads trace data from any source stored in any format and just
254
feed the <c>Collector</c> with it. You may replace the default
255
<c>Collector Filter</c> with a filter that converts new exciting
256
trace data formats to <c>Event Records</c> or you may convert it
257
to an <c>Event Record</c> before you invoke
258
<c>et_collector:report/2</c> and then rely on the default
259
<c>Collector Filter</c> to handle the new format.</p>
261
<p>There are also existing functions in the API that reads from
262
various sources and calls <c>et_collector:report/2</c>:</p>
264
<list type="bulleted">
266
<p>The trace <c>Events</c> that are hosted by the <c>Collector</c> may be
267
stored to file and later be loaded by selecting <c>save</c>
268
and <c>load</c> entries in the <c>Viewers</c> <c>File</c> menu
269
or via the <c>et_collector</c> API.</p>
273
<p>It is also possible to perform live tracing of a running
274
system by making use of the built-in trace support in the
275
Erlang emulator. These Erlang traces can be directed to files
276
or to ports. See the reference manual for
277
<c>erlang:trace/4</c>, <c>erlang:trace_pattern/3</c>,
278
<c>dbg</c> and <c>ttb</c> for more info.</p>
280
<p>There are also corresponding trace client types that can
281
read the Erlang trace data format from such files or ports.
282
The <c>et_collector:start_trace_client/3</c> function makes
283
use of these Erlang trace clients and redirects the trace data
284
to the <c>Collector</c>.</p>
286
<p>The default <c>Collector Filter</c> converts the raw Erlang
287
trace data format into <c>Event Records</c>. If you want to
288
perform this differently you can of course write your own
289
<c>Collector Filter</c> from scratch. But it may probably save
290
you some efforts if you first apply the default filter in
291
<c>et_selector:parse_event/2</c> before you apply your own
292
conversions of its output.</p>
298
<title>Global tracing</title>
300
<p>Setting up an Erlang tracer on a set of nodes and connecting
301
trace clients to the ports of these tracers is not intuitive. In
302
order to make this it easier the <c>Event Tracer</c> has a notion
303
of global tracing. When used, the <c>et_collector</c> process will
304
monitor Erlang nodes and when one connects, an Erlang tracer will
305
automatically be started on the newly connected node. A
306
corresponding trace client will also be started on the
307
<c>Collector</c> node in order to automatically forward the trace
308
<c>Events</c> to the <c>Collector</c>. Set the boolean parameter
309
<c>trace_global</c> to <c>true</c> for either the
310
<c>et_collector</c> or <c>et_viewer</c> in order to activate the
311
global tracing. There is no restriction on how many concurrent
312
(anonymous) collectors you can have, but you can only have one
313
<b>global</b> <c>Collector</c> as its name is registered in
316
<p>In order to further simplify the tracing, you can make use of
317
the <c>et:trace_me/4,5</c> functions. These functions are intended
318
to be invoked from other applications when there are interesting
319
<c>Events</c>, in your application that needs to be
320
highlighted. The functions are extremely light weight as they do
321
nothing besides returning an atom. These functions are
322
specifically designed to be traced for. As the caller explicitly
323
provides the values for the <c>Event Record</c> fields, the
324
default <c>Collector Filter</c> is able to automatically provide a
325
customized <c>Event Record</c> without any user defined filter
328
<p>In normal operation, the <c>et:trace_me/4,5</c> calls are almost
329
for free. When tracing is needed, you can either activate tracing
330
on these functions explicitly. Or you can combine the usage of
331
<c>trace_global</c> with the usage of <c>trace_pattern</c>. When
332
set, the <c>trace_pattern</c> will automatically be activated on
333
all connected nodes. </p>
335
<p>One nice thing with the <c>trace_pattern</c> is that it
336
provides a very simple way of minimizing the amount of generated
337
trace data by allowing you to explicitly control the detail level
338
of the tracing. As you may have seen the <c>et_viewer</c> have a
339
slider called <c>"Detail Level"</c> that allows you to control the
340
detail level of the trace <c>Events</c> displayed in the
341
<c>Viewer</c>. On the other hand if you set a low detail level in
342
the <c>trace_pattern</c>, lots of the trace data will never be
343
generated and thus not sent over the socket to the trace client
344
and stored in the <c>Collector</c>.</p>
348
<title>Viewer window</title>
350
<p>Almost all functionality available in the <c>et_viewer</c> is
351
also available via shortcuts. Which key that has the same effect
352
as selecting a menu entry is shown enclosed in parentheses. For
353
example pressing the key <c>r</c> is equivalent to selecting the
354
menu entry <c>Viewer->Refresh</c>.</p>
358
<list type="bulleted">
360
<p><c>Clear all events in the Collector</c> - Deletes all
361
<c>Events</c> stored in the <c>Collector</c> and notifies all
362
connected <c>Viewers</c> about this.</p>
366
<p><c>Load events to the Collector from file</c> - Loads the
367
<c>Collector</c> with <c>Events</c> from a file and notifies
368
all connected <c>Viewers</c> about this.</p>
372
<p><c>Save all events in the Collector to file</c> - Saves all
373
<c>Events</c> stored in the <c>Collector</c> to file.</p>
377
<p><c>Print setup</c> - Enables editing of printer setting,
378
such as paper and layout.</p>
382
<p><c>Print current page</c> - Prints the events on the
383
current page. The page size is dependent of the selected paper
388
<p><c>Print all pages</c> - Prints all events. The page size
389
is dependent of the selected paper type.</p>
393
<p><c>Close this Viewer</c> - Closes this <c>Viewer</c>
394
window, but keeps all other <c>Viewers</c> windows and the
395
<c>Collector</c> process.</p>
399
<p><c>Close other Viewers, but this</c> - Keeps this
400
<c>Viewer</c> window and its <c>Collector</c> process, but
401
closes all other <c>Viewers</c> windowsconnected to the same
402
<c>Collector</c>.</p>
406
<p><c>Close all Viewers and the Collector</c> - Closes the
407
<c>Collector</c> and all <c>Viewers</c> connected to it.</p>
413
<list type="bulleted">
415
<p><c>First</c> - Scrolls <c>this</c> viewer to the first
416
<c>Event</c> in the <c>Collector</c>.</p>
420
<p><c>Last</c> - Scrolls <c>this</c> viewer to the last
421
<c>Event</c> in the <c>Collector</c>.</p>
425
<p><c>Prev</c> - Scrolls <c>this</c> viewer one page
430
<p><c>Next</c> - Scrolls <c>this</c> viewer one page
435
<p><c>Refresh</c> - Clears <c>this</c> viewer and re-read its
436
<c>Events</c> from the <c>Collector</c>.</p>
440
<p><c>Up</c> - Scrolls a few <c>Events</c> backwards.</p>
444
<p><c>Down</c> - Scrolls a few <c>Events</c> forward.</p>
448
<p><c>Display all actors.</c> - Reset the settings for hidden
449
and/or highlighted actors.</p>
453
<p>Collector menu:</p>
455
<list type="bulleted">
457
<p><c>First</c> - Scrolls<c>all</c> viewers to the first
458
<c>Event</c> in the <c>Collector</c>.</p>
462
<p><c>Last</c> - Scrolls <c>all</c> viewers to the last
463
<c>Event</c> in the <c>Collector</c>.</p>
467
<p><c>Prev</c> - Scrolls <c>all</c> viewers one page
472
<p><c>Next</c> - Scrolls <c>all</c> viewers one page
477
<p><c>Refresh</c> - Clears <c>all</c> viewers and re-read
478
their <c>Events</c> from the <c>Collector</c>.</p>
482
<p>Filters and scaling menu:</p>
484
<list type="bulleted">
486
<p><c>ActiveFilter (=)</c> - Starts a new <c>Viewer</c> window
487
with the same active filter and scale as the current one.</p>
491
<p><c>ActiveFilter (+)</c> - Starts a new <c>Viewer</c> window
492
with the same active filter but a larger scale than the
497
<p><c>ActiveFilter (-)</c> - Starts a new <c>Viewer </c>window
498
with the same active filter but a smaller scale than the
503
<p><c>all (0)</c> - Starts a new <c>Viewer</c> with the
504
<c>Collector Filter</c> as active filter. It will cause all
505
events in the collector to be viewed.</p>
509
<p><c>AnotherFilter (2)</c> - If more filters are inserted
510
into the dictionary, these will turn up here as entries in the
511
<c>Filters</c> menu. The second filter will get the shortcut
512
number 2, the next one number 3 etc. The names are sorted.</p>
516
<p>Slider and radio buttons:</p>
518
<list type="bulleted">
520
<p><c>Hide From=To</c> - When true, this means that the
521
<c>Viewer</c> will hide all <c>Events</c> where the from-actor
522
equals to its to-actor. These events are sometimes called
527
<p><c>Hide (excluded actors)</c> - When true, this means that
528
the <c>Viewer</c> will hide all <c>Events</c> whose actors are
529
marked as excluded. Excluded actors are normally enclosed in
530
round brackets when they are displayed inthe
535
<p><c>Detail level</c> - This slider controls the resolution
536
of the <c>Viewer</c>. Only <c>Events</c> with a detail level
537
<c>smaller</c> than the selected one (default=100=max) are
542
<p>Other features:</p>
544
<list type="bulleted">
546
<p><c>Vertical scroll</c> - Use mouse wheel and up/down arrows
547
to scroll little. Use page up/down and home/end buttons to
552
<p><c>Display details of an event</c> - Left mouse click on
553
the event label or the arrowand a new <c>Contents Viewer</c>
554
window will pop up, displaying the contents of an
559
<p><c>Highlight actor (toggle)</c> - Left mouse click on the
560
actor name tag. The actor name will be enclosed in square
561
brackets <c>[]</c>. When one or more actors are highlighted,
562
only events related to those actors are displayed. All others
567
<p><c>Exclude actor (toggle)</c> - Right mouse click on the
568
actor name tag. The actor name will be enclosed in round
569
brackets <c>()</c>. When an actor is excluded, all events
570
related to this actor is hidden. If the checkbox <c>Hide
571
(excluded actors)</c> is checked, even the name tags and
572
corresponding vertical line of excluded actors will be
577
<p><c>Move actor</c> - Left mouse button drag and drop on
578
actor name tag. Move the actor by first clicking on the actor
579
name, keeping the button pressed while moving the cursor to a
580
new location and release the button where the actor should be
585
<p><c>Display all actors</c> - Press the 'a' button. Reset the
586
settings for hidden and/or highlighted actors.</p>
592
<title>Configuration</title>
594
<p>The <c>Event Records</c> in the Ets table are ordered by their
595
timestamp. Which timestamp that should be used is controlled via
596
the <c>event_order</c> parameter. Default is <c>trace_ts</c> which
597
means the time when the trace data was generated. <c>event_ts</c>
598
means the time when the trace data was parsed (transformed into an
599
<c>Event Record</c>).</p>
603
<title>Contents viewer window</title>
607
<list type="bulleted">
609
<p><c>Close</c> - Close this window.</p>
613
<p><c>Save</c> - Save the contents of this window to file.</p>
619
<list type="bulleted">
621
<p><c>ActiveFilter</c> - Start a new <c>Contents Viewer
622
window</c> with the same active filter.</p>
626
<p><c>AnotherFilter (2)</c> - If more filters are inserted
627
into the dictionary, these will turn up here as entries in the
628
<c>Filters</c> menu. The second filter will be number 2, the
629
next one number 3 etc. The names are sorted.</p>
635
<list type="bulleted">
637
<p><c>Hide actor in viewer</c> - Known actors are shown as a
638
named vertical bars in the <c>Viewer</c> window. By hiding the
639
actor, its vertical bar will be removed and the <c>Viewer</c>
640
will be refreshed.</p>
642
<p><c>Hiding the actor</c> is only useful if the
643
<c>max_actors</c> threshold has been reached, as it then will
644
imply that the "hidden" actor will be displayed as if it were
645
<c>"UNKNOWN"</c>. If the <c>max_actors</c> threshold not have
646
been reached, the actor will re-appear as a vertical bar in
647
the <c>Viewer</c>.</p>
650
<p><c>Show actor in viewer</c> - This implies that the actor
651
will be added as a known actor in the <c>Viewer</c> with its
652
own vertical bar.</p>
658
<list type="bulleted">
660
<p><c>Forward from this event</c> - Set this event to be the first
661
event in the viewer and change its display mode to be enter
662
forward search mode. The actor of this event (from, to or
663
both) will be added to the list of selected actors.</p>
667
<p><c>Reverse from this event</c> - Set this event to be the
668
first <c>Event</c> in the <c>Viewer</c> and change its display
669
mode to be enter reverse search mode. The actor of this
670
<c>Event</c> (from, to or both) will be added to the list of
671
selected actors. Observe, that the <c>Events</c> will be shown
672
in reverse order.</p>
676
<p><c>Abort search. Display all</c> - Switch the display mode
677
of the <c>Viewer</c> to show all <c>Events</c> regardless of
678
any ongoing searches. Abort the searches.</p>