1
.\" Automatically generated by Pod::Man v1.37, Pod::Parser v1.32
4
.\" ========================================================================
5
.de Sh \" Subsection heading
13
.de Sp \" Vertical space (when we can't use .PP)
17
.de Vb \" Begin verbatim text
22
.de Ve \" End verbatim text
26
.\" Set up some character translations and predefined strings. \*(-- will
27
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
28
.\" double quote, and \*(R" will give a right double quote. \*(C+ will
29
.\" give a nicer C++. Capital omega is used to do unbreakable dashes and
30
.\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff,
31
.\" nothing in troff, for use with C<>.
33
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
37
. if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
38
. if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch
51
.\" If the F register is turned on, we'll generate index entries on stderr for
52
.\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index
53
.\" entries marked with X<> in POD. Of course, you'll have to process the
54
.\" output yourself in some meaningful fashion.
57
. tm Index:\\$1\t\\n%\t"\\$2"
63
.\" For nroff, turn off justification. Always turn off hyphenation; it makes
64
.\" way too many mistakes in technical documents.
68
.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
69
.\" Fear. Run. Save yourself. No user-serviceable parts.
70
. \" fudge factors for nroff and troff
79
. ds #H ((1u-(\\\\n(.fu%2u))*.13m)
85
. \" simple accents for nroff and troff
95
. ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
96
. ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
97
. ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
98
. ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
99
. ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
100
. ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
102
. \" troff and (daisy-wheel) nroff accents
103
.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
104
.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
105
.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
106
.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
107
.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
108
.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
109
.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
110
.ds ae a\h'-(\w'a'u*4/10)'e
111
.ds Ae A\h'-(\w'A'u*4/10)'E
112
. \" corrections for vroff
113
.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
114
.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
115
. \" for low resolution devices (crt and lpr)
116
.if \n(.H>23 .if \n(.V>19 \
129
.\" ========================================================================
131
.IX Title "INNOTOP 1p"
132
.TH INNOTOP 1p "2007-11-09" "perl v5.8.8" "User Contributed Perl Documentation"
134
innotop \- MySQL and InnoDB transaction/status monitor.
136
.IX Header "SYNOPSIS"
137
To monitor servers normally:
143
To monitor InnoDB status information from a file:
146
\& innotop /var/log/mysql/mysqld.err
149
To run innotop non-interactively in a pipe-and-filter configuration:
152
\& innotop \-\-count 5 \-d 1 \-n
155
.IX Header "DESCRIPTION"
156
innotop monitors MySQL servers. Each of its modes shows you a different aspect
157
of what's happening in the server. For example, there's a mode for monitoring
158
replication, one for queries, and one for transactions. innotop refreshes its
159
data periodically, so you see an updating view.
161
innotop has lots of features for power users, but you can start and run it with
162
virtually no configuration. If you're just getting started, see
163
\&\*(L"\s-1QUICK\-START\s0\*(R". Press '?' at any time while running innotop for
164
context-sensitive help.
166
.IX Header "QUICK-START"
167
To start innotop, open a terminal or command prompt. If you have installed
168
innotop on your system, you should be able to just type \*(L"innotop\*(R" and press
169
Enter; otherwise, you will need to change to innotop's directory and type \*(L"perl
172
The first thing innotop needs to know is how to connect to a MySQL server. You
173
can just enter the hostname of the server, for example \*(L"localhost\*(R" or
174
\&\*(L"127.0.0.1\*(R" if the server is on the same machine as innotop. After this innotop
175
will prompt you for a \s-1DSN\s0 (data source name). You should be able to just accept
176
the defaults by pressing Enter.
178
When innotop asks you about a table to use when resetting InnoDB deadlock
179
information, just accept the default for now. This is an advanced feature you
180
can configure later (see \*(L"D: InnoDB Deadlocks\*(R" for more).
182
If you have a .my.cnf file with your MySQL connection defaults, innotop can read
183
it, and you won't need to specify a username and password if it's in that file.
184
Otherwise, you should answer 'y' to the next couple of prompts.
186
After this, you should be connected, and innotop should show you something like
190
\& InnoDB Txns (? for help) localhost, 01:11:19, InnoDB 10s :\-), 50 QPS,
194
\& CXN History Versions Undo Dirty Buf Used Bufs Txns MaxTxn
195
\& localhost 7 2035 0 0 0.00% 92.19% 1 07:34
199
\& CXN ID User Host Txn Status Time Undo Query Tex
200
\& localhost 98379 user1 webserver ACTIVE 07:34 0 SELECT `c
201
\& localhost 98450 user1 webserver ACTIVE 01:06 0 INSERT IN
202
\& localhost 97750 user1 webserver not starte 00:00 0
203
\& localhost 98375 user1 appserver not starte 00:00 0
206
(This sample is truncated at the right so it will fit on a terminal when running
209
This sample comes from a quiet server with few transactions active. If your
210
server is busy, you'll see more output. Notice the first line on the screen,
211
which tells you what mode you're in and what server you're connected to. You
212
can change to other modes with keystrokes; press 'Q' to switch to a list of
213
currently running queries.
215
Press the '?' key to see what keys are active in the current mode. You can
216
press any of these keys and innotop will either take the requested action or
217
prompt you for more input. If your system has Term::ReadLine support, you can
218
use \s-1TAB\s0 and other keys to auto-complete and edit input.
220
To quit innotop, press the 'q' key.
223
innotop is mostly configured via its configuration file, but some of the
224
configuration options can come from the command line. You can also specify a
225
file to monitor for InnoDB status output; see \*(L"\s-1MONITORING\s0 A \s-1FILE\s0\*(R" for more
228
You can negate some options by prefixing the option name with \-\-no. For
229
example, \-\-noinc (or \-\-no\-inc) negates \*(L"\-\-inc\*(R".
232
Print a summary of command-line usage and exit.
235
Enable or disable terminal coloring. Corresponds to the \*(L"color\*(R" config file
239
Specifies a configuration file to read. This option is non\-sticky, that is to
240
say it does not persist to the configuration file itself.
243
Enable non-interactive operation. See \*(L"\s-1NON\-INTERACTIVE\s0 \s-1OPERATION\s0\*(R" for more.
246
Refresh only the specified number of times (ticks) before exiting. Each refresh
247
is a pause for \*(L"interval\*(R" seconds, followed by requesting data from MySQL
248
connections and printing it to the terminal.
251
Specifies the amount of time to pause between ticks (refreshes). Corresponds to
252
the configuration option \*(L"interval\*(R".
255
Specifies the mode in which innotop should start. Corresponds to the
256
configuration option \*(L"mode\*(R".
259
Specifies whether innotop should display absolute numbers or relative numbers
260
(offsets from their previous values). Corresponds to the configuration option
261
\&\*(L"status_inc\*(R".
264
Output version information and exit.
267
innotop is interactive, and you control it with key\-presses.
269
Uppercase keys switch between modes.
271
Lowercase keys initiate some action within the current mode.
273
Other keys do something special like change configuration or show the
276
Press '?' at any time to see the currently active keys and what they do.
279
Each of innotop's modes retrieves and displays a particular type of data from
280
the servers you're monitoring. You switch between modes with uppercase keys.
281
The following is a brief description of each mode, in alphabetical order. To
282
switch to the mode, press the key listed in front of its heading in the
284
.IP "B: InnoDB Buffers" 4
285
.IX Item "B: InnoDB Buffers"
286
This mode displays information about the InnoDB buffer pool, page statistics,
287
insert buffer, and adaptive hash index. The data comes from \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0.
289
This mode contains the \*(L"buffer_pool\*(R", \*(L"page_statistics\*(R",
290
\&\*(L"insert_buffers\*(R", and \*(L"adaptive_hash_index\*(R" tables by default.
291
.IP "C: Command Summary" 4
292
.IX Item "C: Command Summary"
293
This mode is similar to mytop's Command Summary mode. It shows the
294
\&\*(L"cmd_summary\*(R" table, which looks something like the following:
297
\& Command Summary (? for help) localhost, 25+07:16:43, 2.45 QPS, 3 thd, 5.0.40
298
\& _____________________ Command Summary _____________________
299
\& Name Value Pct Last Incr Pct
300
\& Select_scan 3244858 69.89% 2 100.00%
301
\& Select_range 1354177 29.17% 0 0.00%
302
\& Select_full_join 39479 0.85% 0 0.00%
303
\& Select_full_range_join 4097 0.09% 0 0.00%
304
\& Select_range_check 0 0.00% 0 0.00%
307
The command summary table is built by extracting variables from
308
\&\*(L"\s-1STATUS_VARIABLES\s0\*(R". The variables must be numeric and must match the prefix
309
given by the \*(L"cmd_filter\*(R" configuration variable. The variables are then
310
sorted by value descending and compared to the last variable, as shown above.
311
The percentage columns are percentage of the total of all variables in the
312
table, so you can see the relative weight of the variables.
314
The example shows what you see if the prefix is \*(L"Select_\*(R". The default
315
prefix is \*(L"Com_\*(R". You can choose a prefix with the 's' key.
317
It's rather like running \s-1SHOW\s0 \s-1VARIABLES\s0 \s-1LIKE\s0 \*(L"prefix%\*(R" with memory and
320
Values are aggregated across all servers. The Pct columns are not correctly
321
aggregated across multiple servers. This is a known limitation of the grouping
322
algorithm that may be fixed in the future.
323
.IP "D: InnoDB Deadlocks" 4
324
.IX Item "D: InnoDB Deadlocks"
325
This mode shows the transactions involved in the last InnoDB deadlock. A second
326
table shows the locks each transaction held and waited for. A deadlock is
327
caused by a cycle in the waits-for graph, so there should be two locks held and
328
one waited for unless the deadlock information is truncated.
330
InnoDB puts deadlock information before some other information in the \s-1SHOW\s0
331
\&\s-1INNODB\s0 \s-1STATUS\s0 output. If there are a lot of locks, the deadlock information can
332
grow very large, and there is a limit on the size of the \s-1SHOW\s0 \s-1INNODB\s0
333
\&\s-1STATUS\s0 output. A large deadlock can fill the entire output, or even be
334
truncated, and prevent you from seeing other information at all. If you are
335
running innotop in another mode, for example T mode, and suddenly you don't see
336
anything, you might want to check and see if a deadlock has wiped out the data
339
If it has, you can create a small deadlock to replace the large one. Use the
340
\&'w' key to 'wipe' the large deadlock with a small one. This will not work
341
unless you have defined a deadlock table for the connection (see \*(L"\s-1SERVER\s0 \s-1CONNECTIONS\s0\*(R").
343
You can also configure innotop to automatically detect when a large deadlock
344
needs to be replaced with a small one (see \*(L"auto_wipe_dl\*(R").
346
This mode displays the \*(L"deadlock_transactions\*(R" and \*(L"deadlock_locks\*(R" tables
348
.IP "F: InnoDB Foreign Key Errors" 4
349
.IX Item "F: InnoDB Foreign Key Errors"
350
This mode shows the last InnoDB foreign key error information, such as the
351
table where it happened, when and who and what query caused it, and so on.
353
InnoDB has a huge variety of foreign key error messages, and many of them are
354
just hard to parse. innotop doesn't always do the best job here, but there's
355
so much code devoted to parsing this messy, unparseable output that innotop is
356
likely never to be perfect in this regard. If innotop doesn't show you what
357
you need to see, just look at the status text directly.
359
This mode displays the \*(L"fk_error\*(R" table by default.
360
.IP "I: InnoDB I/O Info" 4
361
.IX Item "I: InnoDB I/O Info"
362
This mode shows InnoDB's I/O statistics, including the I/O threads, pending I/O,
363
file I/O miscellaneous, and log statistics. It displays the \*(L"io_threads\*(R",
364
\&\*(L"pending_io\*(R", \*(L"file_io_misc\*(R", and \*(L"log_statistics\*(R" tables by default.
367
This mode shows information about current locks. At the moment only InnoDB
368
locks are supported, and by default you'll only see locks for which transactions
369
are waiting. This information comes from the \s-1TRANSACTIONS\s0 section of the InnoDB
370
status text. If you have a very busy server, you may have frequent lock waits;
371
it helps to be able to see which tables and indexes are the \*(L"hot spot\*(R" for
372
locks. If your server is running pretty well, this mode should show nothing.
374
You can configure MySQL and innotop to monitor not only locks for which a
375
transaction is waiting, but those currently held, too. You can do this with the
376
InnoDB Lock Monitor (<http://dev.mysql.com/doc/en/innodb\-monitor.html>). It's
377
not documented in the MySQL manual, but creating the lock monitor with the
378
following statement also affects the output of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0, which innotop
382
\& CREATE TABLE innodb_lock_monitor(a int) ENGINE=INNODB;
385
This causes InnoDB to print its output to the MySQL file every 16 seconds or so,
386
as stated in the manual, but it also makes the normal \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0 output
387
include lock information, which innotop can parse and display (that's the
388
undocumented feature).
390
This means you can do what may have seemed impossible: to a limited extent
391
(InnoDB truncates some information in the output), you can see which transaction
392
holds the locks something else is waiting for. You can also enable and disable
393
the InnoDB Lock Monitor with the key mappings in this mode.
395
This mode displays the \*(L"innodb_locks\*(R" table by default. Here's a sample of
396
the screen when one connection is waiting for locks another connection holds:
399
\& _________________________________ InnoDB Locks __________________________
400
\& CXN ID Type Waiting Wait Active Mode DB Table Index
401
\& localhost 12 RECORD 1 00:10 00:10 X test t1 PRIMARY
402
\& localhost 12 TABLE 0 00:10 00:10 IX test t1
403
\& localhost 12 RECORD 1 00:10 00:10 X test t1 PRIMARY
404
\& localhost 11 TABLE 0 00:00 00:25 IX test t1
405
\& localhost 11 RECORD 0 00:00 00:25 X test t1 PRIMARY
408
You can see the first connection, \s-1ID\s0 12, is waiting for a lock on the \s-1PRIMARY\s0
409
key on test.t1, and has been waiting for 10 seconds. The second connection
410
isn't waiting, because the Waiting column is 0, but it holds locks on the same
411
index. That tells you connection 11 is blocking connection 12.
412
.IP "M: Master/Slave Replication Status" 4
413
.IX Item "M: Master/Slave Replication Status"
414
This mode shows the output of \s-1SHOW\s0 \s-1SLAVE\s0 \s-1STATUS\s0 and \s-1SHOW\s0 \s-1MASTER\s0 \s-1STATUS\s0 in three
415
tables. The first two divide the slave's status into \s-1SQL\s0 and I/O thread status,
416
and the last shows master status. Filters are applied to eliminate non-slave
417
servers from the slave tables, and non-master servers from the master table.
419
This mode displays the \*(L"slave_sql_status\*(R", \*(L"slave_io_status\*(R", and
420
\&\*(L"master_status\*(R" tables by default.
421
.IP "O: Open Tables" 4
422
.IX Item "O: Open Tables"
423
This section comes from MySQL's \s-1SHOW\s0 \s-1OPEN\s0 \s-1TABLES\s0 command. By default it is
424
filtered to show tables which are in use by one or more queries, so you can
425
get a quick look at which tables are 'hot'. You can use this to guess which
426
tables might be locked implicitly.
428
This mode displays the \*(L"open_tables\*(R" mode by default.
429
.IP "Q: Query List" 4
430
.IX Item "Q: Query List"
431
This mode displays the output from \s-1SHOW\s0 \s-1FULL\s0 \s-1PROCESSLIST\s0, much like \fBmytop\fR's
432
query list mode. This mode does \fBnot\fR show InnoDB-related information. This
433
is probably one of the most useful modes for general usage.
435
There is an informative header that shows general status information about
436
your server. You can toggle it on and off with the 'h' key. By default,
437
innotop hides inactive processes and its own process. You can toggle these on
438
and off with the 'i' and 'a' keys.
440
You can \s-1EXPLAIN\s0 a query from this mode with the 'e' key. This displays the
441
query's full text, the results of \s-1EXPLAIN\s0, and in newer MySQL versions, even
442
the optimized query resulting from \s-1EXPLAIN\s0 \s-1EXTENDED\s0. innotop also tries to
443
rewrite certain queries to make them EXPLAIN\-able. For example, \s-1INSERT/SELECT\s0
444
statements are rewritable.
446
This mode displays the \*(L"q_header\*(R" and \*(L"processlist\*(R" tables by default.
447
.IP "R: InnoDB Row Operations and Semaphores" 4
448
.IX Item "R: InnoDB Row Operations and Semaphores"
449
This mode shows InnoDB row operations, row operation miscellaneous, semaphores,
450
and information from the wait array. It displays the \*(L"row_operations\*(R",
451
\&\*(L"row_operation_misc\*(R", \*(L"semaphores\*(R", and \*(L"wait_array\*(R" tables by default.
452
.IP "S: Variables & Status" 4
453
.IX Item "S: Variables & Status"
454
This mode calculates statistics, such as queries per second, and prints them out
455
in several different styles. You can show absolute values, or incremental values
458
You can switch between the views by pressing a key. The 's' key prints a
459
single line each time the screen updates, in the style of \fBvmstat\fR. The 'g'
460
key changes the view to a graph of the same numbers, sort of like \fBtload\fR.
461
The 'v' key changes the view to a pivoted table of variable names on the left,
462
with successive updates scrolling across the screen from left to right. You can
463
choose how many updates to put on the screen with the \*(L"num_status_sets\*(R"
464
configuration variable.
466
Headers may be abbreviated to fit on the screen in interactive operation. You
467
choose which variables to display with the 'c' key, which selects from
468
predefined sets, or lets you create your own sets. You can edit the current set
471
This mode doesn't really display any tables like other modes. Instead, it uses
472
a table definition to extract and format the data, but it then transforms the
473
result in special ways before outputting it. It uses the \*(L"var_status\*(R" table
475
.IP "T: InnoDB Transactions" 4
476
.IX Item "T: InnoDB Transactions"
477
This mode shows transactions from the InnoDB monitor's output, in \fBtop\fR\-like
478
format. This mode is the reason I wrote innotop.
480
You can kill queries or processes with the 'k' and 'x' keys, and \s-1EXPLAIN\s0 a query
481
with the 'e' or 'f' keys. InnoDB doesn't print the full query in transactions,
482
so explaining may not work right if the query is truncated.
484
The informational header can be toggled on and off with the 'h' key. By
485
default, innotop hides inactive transactions and its own transaction. You can
486
toggle this on and off with the 'i' and 'a' keys.
488
This mode displays the \*(L"t_header\*(R" and \*(L"innodb_transactions\*(R" tables by
491
.IX Header "INNOTOP STATUS"
492
The first line innotop displays is a \*(L"status bar\*(R" of sorts. What it contains
493
depends on the mode you're in, and what servers you're monitoring. The first
494
few words are always the innotop mode, such as \*(L"InnoDB Txns\*(R" for T mode,
495
followed by a reminder to press '?' for help at any time.
496
.Sh "\s-1ONE\s0 \s-1SERVER\s0"
497
.IX Subsection "ONE SERVER"
498
The simplest case is when you're monitoring a single server. In this case, the
499
name of the connection is next on the status line. This is the name you gave
500
when you created the connection \*(-- most likely the MySQL server's hostname.
501
This is followed by the server's uptime.
503
If you're in an InnoDB mode, such as T or B, the next word is \*(L"InnoDB\*(R" followed
504
by some information about the \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0 output used to render the
505
screen. The first word is the number of seconds since the last \s-1SHOW\s0 \s-1INNODB\s0
506
\&\s-1STATUS\s0, which InnoDB uses to calculate some per-second statistics. The next is
507
a smiley face indicating whether the InnoDB output is truncated. If the smiley
508
face is a :\-), all is well; there is no truncation. A :^| means the transaction
509
list is so long, InnoDB has only printed out some of the transactions. Finally,
510
a frown :\-( means the output is incomplete, which is probably due to a deadlock
511
printing too much lock information (see \*(L"D: InnoDB Deadlocks\*(R").
513
The next two words indicate the server's queries per second (\s-1QPS\s0) and how many
514
threads (connections) exist. Finally, the server's version number is the last
516
.Sh "\s-1MULTIPLE\s0 \s-1SERVERS\s0"
517
.IX Subsection "MULTIPLE SERVERS"
518
If you are monitoring multiple servers (see \*(L"\s-1SERVER\s0 \s-1CONNECTIONS\s0\*(R"), the status
519
line does not show any details about individual servers. Instead, it shows the
520
names of the connections that are active. Again, these are connection names you
521
specified, which are likely to be the server's hostname. A connection that has
522
an error is prefixed with an exclamation point.
524
If you are monitoring a group of servers (see \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R"), the status
525
line shows the name of the group. If any connection in the group has an
526
error, the group's name is followed by the fraction of the connections that
529
See \*(L"\s-1ERROR\s0 \s-1HANDLING\s0\*(R" for more details about innotop's error handling.
530
.Sh "\s-1MONITORING\s0 A \s-1FILE\s0"
531
.IX Subsection "MONITORING A FILE"
532
If you give a filename on the command line, innotop will not connect to \s-1ANY\s0
533
servers at all. It will watch the specified file for InnoDB status output and
534
use that as its data source. It will always show a single connection called
535
\&'file'. And since it can't connect to a server, it can't determine how long the
536
server it's monitoring has been up; so it calculates the server's uptime as time
537
since innotop started running.
538
.SH "SERVER ADMINISTRATION"
539
.IX Header "SERVER ADMINISTRATION"
540
While innotop is primarily a monitor that lets you watch and analyze your
541
servers, it can also send commands to servers. The most frequently useful
542
commands are killing queries and stopping or starting slaves.
544
You can kill a connection, or in newer versions of MySQL kill a query but not a
545
connection, from \*(L"Q: Query List\*(R" and \*(L"T: InnoDB Transactions\*(R" modes.
546
Press 'k' to issue a \s-1KILL\s0 command, or 'x' to issue a \s-1KILL\s0 \s-1QUERY\s0 command.
547
innotop will prompt you for the server and/or connection \s-1ID\s0 to kill (innotop
548
does not prompt you if there is only one possible choice for any input).
549
innotop pre-selects the longest-running query, or the oldest connection.
550
Confirm the command with 'y'.
552
In \*(L"M: Master/Slave Replication Status\*(R" mode, you can start and stop slaves
553
with the 'a' and 'o' keys, respectively. You can send these commands to many
554
slaves at once. innotop fills in a default command of \s-1START\s0 \s-1SLAVE\s0 or \s-1STOP\s0 \s-1SLAVE\s0
555
for you, but you can actually edit the command and send anything you wish, such
556
as \s-1SET\s0 \s-1GLOBAL\s0 SQL_SLAVE_SKIP_COUNTER=1 to make the slave skip one binlog event
559
You can also ask innotop to calculate the earliest binlog in use by any slave
560
and issue a \s-1PURGE\s0 \s-1MASTER\s0 \s-1LOGS\s0 on the master. Use the 'b' key for this. innotop
561
will prompt you for a master to run the command on, then prompt you for the
562
connection names of that master's slaves (there is no way for innotop to
563
determine this reliably itself). innotop will find the minimum binlog in use by
564
these slave connections and suggest it as the argument to \s-1PURGE\s0 \s-1MASTER\s0 \s-1LOGS\s0.
565
.SH "SERVER CONNECTIONS"
566
.IX Header "SERVER CONNECTIONS"
567
When you create a server connection, innotop asks you for a series of inputs, as
571
A \s-1DSN\s0 is a Data Source Name, which is the initial argument passed to the \s-1DBI\s0
572
module for connecting to a server. It is usually of the form
575
\& DBI:mysql:;mysql_read_default_group=mysql;host=HOSTNAME
578
Since this \s-1DSN\s0 is passed to the DBD::mysql driver, you should read the driver's
579
documentation at \*(L"http://search.cpan.org/dist/DBD\-mysql/lib/DBD/mysql.pm\*(R" for
580
the exact details on all the options you can pass the driver in the \s-1DSN\s0. You
581
can read more about \s-1DBI\s0 at <http://dbi.perl.org/docs/>, and especially at
582
<http://search.cpan.org/~timb/DBI/DBI.pm>.
584
The mysql_read_default_group=mysql option lets the \s-1DBD\s0 driver read your MySQL
585
options files, such as ~/.my.cnf on UNIX-ish systems. You can use this to avoid
586
specifying a username or password for the connection.
587
.IP "InnoDB Deadlock Table" 4
588
.IX Item "InnoDB Deadlock Table"
589
This optional item tells innotop a table name it can use to deliberately create
590
a small deadlock (see \*(L"D: InnoDB Deadlocks\*(R"). If you specify this option,
591
you just need to be sure the table doesn't exist, and that innotop can create
592
and drop the table with the InnoDB storage engine. You can safely omit or just
593
accept the default if you don't intend to use this.
596
innotop will ask you if you want to specify a username. If you say 'y', it will
597
then prompt you for a user name. If you have a MySQL option file that specifies
598
your username, you don't have to specify a username.
600
The username defaults to your login name on the system you're running innotop on.
603
innotop will ask you if you want to specify a password. Like the username, the
604
password is optional, but there's an additional prompt that asks if you want to
605
save the password in the innotop configuration file. If you don't save it in
606
the configuration file, innotop will prompt you for a password each time it
607
starts. Passwords in the innotop configuration file are saved in plain text,
608
not encrypted in any way.
610
Once you finish answering these questions, you should be connected to a server.
611
But innotop isn't limited to monitoring a single server; you can define many
612
server connections and switch between them by pressing the '@' key. See
613
\&\*(L"\s-1SWITCHING\s0 \s-1BETWEEN\s0 \s-1CONNECTIONS\s0\*(R".
615
To create a new connection, press the '@' key and type the name of the new
616
connection, then follow the steps given above.
618
.IX Header "SERVER GROUPS"
619
If you have multiple MySQL instances, you can put them into named groups, such
620
as 'all', 'masters', and 'slaves', which innotop can monitor all together.
622
You can choose which group to monitor with the '#' key, and you can press the
623
\&\s-1TAB\s0 key to switch to the next group. If you're not currently monitoring a
624
group, pressing \s-1TAB\s0 selects the first group.
626
To create a group, press the '#' key and type the name of your new group, then
627
type the names of the connections you want the group to contain.
628
.SH "SWITCHING BETWEEN CONNECTIONS"
629
.IX Header "SWITCHING BETWEEN CONNECTIONS"
630
innotop lets you quickly switch which servers you're monitoring. The most basic
631
way is by pressing the '@' key and typing the name(s) of the connection(s) you
632
want to use. This setting is per\-mode, so you can monitor different connections
633
in each mode, and innotop remembers which connections you choose.
635
You can quickly switch to the 'next' connection in alphabetical order with the
636
\&'n' key. If you're monitoring a server group (see \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R") this will
637
switch to the first connection.
639
You can also type many connection names, and innotop will fetch and display data
640
from them all. Just separate the connection names with spaces, for example
641
\&\*(L"server1 server2.\*(R" Again, if you type the name of a connection that doesn't
642
exist, innotop will prompt you for connection information and create the
645
Another way to monitor multiple connections at once is with server groups. You
646
can use the \s-1TAB\s0 key to switch to the 'next' group in alphabetical order, or if
647
you're not monitoring any groups, \s-1TAB\s0 will switch to the first group.
649
innotop does not fetch data in parallel from connections, so if you are
650
monitoring a large group or many connections, you may notice increased delay
653
When you monitor more than one connection, innotop's status bar changes. See
654
\&\*(L"\s-1INNOTOP\s0 \s-1STATUS\s0\*(R".
656
.IX Header "ERROR HANDLING"
657
Error handling is not that important when monitoring a single connection, but is
658
crucial when you have many active connections. A crashed server or lost
659
connection should not crash innotop. As a result, innotop will continue to run
660
even when there is an error; it just won't display any information from the
661
connection that had an error. Because of this, innotop's behavior might confuse
662
you. It's a feature, not a bug!
664
innotop does not continue to query connections that have errors, because they
665
may slow innotop and make it hard to use, especially if the error is a problem
666
connecting and causes a long time\-out. Instead, innotop retries the connection
667
occasionally to see if the error still exists. If so, it will wait until some
668
point in the future. The wait time increases in ticks as the Fibonacci series,
669
so it tries less frequently as time passes.
671
Since errors might only happen in certain modes because of the \s-1SQL\s0 commands
672
issued in those modes, innotop keeps track of which mode caused the error. If
673
you switch to a different mode, innotop will retry the connection instead of
676
By default innotop will display the problem in red text at the bottom of the
677
first table on the screen. You can disable this behavior with the
678
\&\*(L"show_cxn_errors_in_tbl\*(R" configuration option, which is enabled by default.
679
If the \*(L"debug\*(R" option is enabled, innotop will display the error at the
680
bottom of every table, not just the first. And if \*(L"show_cxn_errors\*(R" is
681
enabled, innotop will print the error text to \s-1STDOUT\s0 as well. Error messages
682
might only display in the mode that caused the error, depending on the mode and
683
whether innotop is avoiding querying that connection.
684
.SH "NON-INTERACTIVE OPERATION"
685
.IX Header "NON-INTERACTIVE OPERATION"
686
You can run innotop in non-interactive mode, in which case it is entirely
687
controlled from the configuration file and command-line options. To start
688
innotop in non-interactive mode, give the L\*(L"<\-\-nonint\*(R"> command-line option.
689
This changes innotop's behavior in the following ways:
691
Certain Perl modules are not loaded. Term::Readline is not loaded, since
692
innotop doesn't prompt interactively. Term::ANSIColor and Win32::Console::ANSI
693
modules are not loaded. Term::ReadKey is still used, since innotop may have to
694
prompt for connection passwords when starting up.
696
innotop does not clear the screen after each tick.
698
innotop does not persist any changes to the configuration file.
700
If \*(L"\-\-count\*(R" is given and innotop is in incremental mode (see \*(L"status_inc\*(R"
701
and \*(L"\-\-inc\*(R"), innotop actually refreshes one more time than specified so it
702
can print incremental statistics. This suppresses output during the first
703
tick, so innotop may appear to hang.
705
innotop only displays the first table in each mode. This is so the output can
706
be easily processed with other command-line utilities such as awk and sed. To
707
change which tables display in each mode, see \*(L"\s-1TABLES\s0\*(R". Since \*(L"Q: Query List\*(R" mode is so important, innotop automatically disables the \*(L"q_header\*(R"
708
table. This ensures you'll see the \*(L"processlist\*(R" table, even if you have
709
innotop configured to show the q_header table during interactive operation.
710
Similarly, in \*(L"T: InnoDB Transactions\*(R" mode, the \*(L"t_header\*(R" table is
711
suppressed so you see only the \*(L"innodb_transactions\*(R" table.
713
All output is tab-separated instead of being column-aligned with whitespace, and
714
innotop prints the full contents of each table instead of only printing one
717
innotop only prints column headers once instead of every tick (see
718
\&\*(L"hide_hdr\*(R"). innotop does not print table captions (see
719
\&\*(L"display_table_captions\*(R"). innotop ensures there are no empty lines in the
722
innotop does not honor the \*(L"shorten\*(R" transformation, which normally shortens
723
some numbers to human-readable formats.
725
innotop does not print a status line (see \*(L"\s-1INNOTOP\s0 \s-1STATUS\s0\*(R").
727
.IX Header "CONFIGURING"
728
Nearly everything about innotop is configurable. Most things are possible to
729
change with built-in commands, but you can also edit the configuration file.
731
While running innotop, press the '$' key to bring up the configuration editing
732
dialog. Press another key to select the type of data you want to edit:
733
.IP "S: Statement Sleep Times" 4
734
.IX Item "S: Statement Sleep Times"
735
Edits \s-1SQL\s0 statement sleep delays, which make innotop pause for the specified
736
amount of time after executing a statement. See \*(L"\s-1SQL\s0 \s-1STATEMENTS\s0\*(R" for a
737
definition of each statement and what it does. By default innotop does not
738
delay after any statements.
740
This feature is included so you can customize the side-effects caused by
741
monitoring your server. You may not see any effects, but some innotop users
742
have noticed that certain MySQL versions under very high load with InnoDB
743
enabled take longer than usual to execute \s-1SHOW\s0 \s-1GLOBAL\s0 \s-1STATUS\s0. If innotop calls
744
\&\s-1SHOW\s0 \s-1FULL\s0 \s-1PROCESSLIST\s0 immediately afterward, the processlist contains more
745
queries than the machine actually averages at any given moment. Configuring
746
innotop to pause briefly after calling \s-1SHOW\s0 \s-1GLOBAL\s0 \s-1STATUS\s0 alleviates this
749
Sleep times are stored in the \*(L"stmt_sleep_times\*(R" section of the configuration
750
file. Fractional-second sleeps are supported, subject to your hardware's
752
.IP "c: Edit Columns" 4
753
.IX Item "c: Edit Columns"
754
Starts the table editor on one of the displayed tables. See \*(L"\s-1TABLE\s0 \s-1EDITOR\s0\*(R".
755
An alternative way to start the table editor without entering the configuration
756
dialog is with the '^' key.
757
.IP "g: General Configuration" 4
758
.IX Item "g: General Configuration"
759
Starts the configuration editor to edit global and mode-specific configuration
760
variables (see \*(L"\s-1MODES\s0\*(R"). innotop prompts you to choose a variable from among
761
the global and mode-specific ones depending on the current mode.
762
.IP "k: Row-Coloring Rules" 4
763
.IX Item "k: Row-Coloring Rules"
764
Starts the row-coloring rules editor on one of the displayed table(s). See
765
\&\*(L"\s-1COLORS\s0\*(R" for details.
766
.IP "p: Manage Plugins" 4
767
.IX Item "p: Manage Plugins"
768
Starts the plugin configuration editor. See \*(L"\s-1PLUGINS\s0\*(R" for details.
769
.IP "s: Server Groups" 4
770
.IX Item "s: Server Groups"
771
Lets you create and edit server groups. See \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R".
772
.IP "t: Choose Displayed Tables" 4
773
.IX Item "t: Choose Displayed Tables"
774
Lets you choose which tables to display in this mode. See \*(L"\s-1MODES\s0\*(R" and
775
\&\*(L"\s-1TABLES\s0\*(R".
776
.SH "CONFIGURATION FILE"
777
.IX Header "CONFIGURATION FILE"
778
innotop's default configuration file location is in \f(CW$HOME\fR/.innotop, but can be
779
overridden with the \*(L"\-\-config\*(R" command-line option. You can edit it by hand
780
safely. innotop reads the configuration file when it starts, and writes it out
781
again when it exits, so any changes you make while innotop is running will be
784
innotop doesn't store its entire configuration in the configuration file. It
785
has a huge set of default configuration that it holds only in memory, and the
786
configuration file only overrides these defaults. When you customize a default
787
setting, innotop notices, and then stores the customizations into the file.
788
This keeps the file size down, makes it easier to edit, and makes upgrades
791
A configuration file can be made read\-only. See \*(L"readonly\*(R".
793
The configuration file is arranged into sections like an \s-1INI\s0 file. Each
794
section begins with [section\-name] and ends with [/section\-name]. Each
795
section's entries have a different syntax depending on the data they need to
796
store. You can put comments in the file; any line that begins with a #
797
character is a comment. innotop will not read the comments, so it won't write
798
them back out to the file when it exits. Comments in read-only configuration
799
files are still useful, though.
801
The first line in the file is innotop's version number. This lets innotop
802
notice when the file format is not backwards\-compatible, and upgrade smoothly
803
without destroying your customized configuration.
805
The following list describes each section of the configuration file and the data
809
The 'general' section contains global configuration variables and variables that
810
may be mode\-specific, but don't belong in any other section. The syntax is a
811
simple key=value list. innotop writes a comment above each value to help you
812
edit the file by hand.
816
Controls S mode presentation (see \*(L"S: Variables & Status\*(R"). If g, values are
817
graphed; if s, values are like vmstat; if p, values are in a pivoted table.
820
Specifies which set of variables to display in \*(L"S: Variables & Status\*(R" mode.
821
See \*(L"\s-1VARIABLE\s0 \s-1SETS\s0\*(R".
823
.IX Item "auto_wipe_dl"
824
Instructs innotop to automatically wipe large deadlocks when it notices them.
825
When this happens you may notice a slight delay. At the next tick, you will
826
usually see the information that was being truncated by the large deadlock.
829
Specifies what kind of characters to allow through the \*(L"no_ctrl_char\*(R"
830
transformation. This keeps non-printable characters from confusing a
831
terminal when you monitor queries that contain binary data, such as images.
833
The default is 'ascii', which considers anything outside normal \s-1ASCII\s0 to be a
834
control character. The other allowable values are 'unicode' and 'none'. 'none'
835
considers every character a control character, which can be useful for
836
collapsing \s-1ALL\s0 text fields in queries.
838
.IX Item "cmd_filter"
839
This is the prefix that filters variables in \*(L"C: Command Summary\*(R" mode.
842
Whether terminal coloring is permitted.
844
.IX Item "cxn_timeout"
845
On MySQL versions 4.0.3 and newer, this variable is used to set the connection's
846
timeout, so MySQL doesn't close the connection if it is not used for a while.
847
This might happen because a connection isn't monitored in a particular mode, for
851
This option enables more verbose errors and makes innotop more strict in some
852
places. It can help in debugging filters and other user-defined code. It also
853
makes innotop write a lot of information to \*(L"debugfile\*(R" when there is a
857
A file to which innotop will write information when there is a crash. See
858
\&\*(L"\s-1FILES\s0\*(R".
859
.IP "display_table_captions" 4
860
.IX Item "display_table_captions"
861
innotop displays a table caption above most tables. This variable suppresses or
862
shows captions on all tables globally. Some tables are configured with the
863
hide_caption property, which overrides this.
866
Whether to show \s-1GLOBAL\s0 variables and status. innotop only tries to do this on
867
servers which support the \s-1GLOBAL\s0 option to \s-1SHOW\s0 \s-1VARIABLES\s0 and \s-1SHOW\s0 \s-1STATUS\s0. In
868
some MySQL versions, you need certain privileges to do this; if you don't have
869
them, innotop will not be able to fetch any variable and status data. This
870
configuration variable lets you run innotop and fetch what data you can even
871
without the elevated privileges.
873
I can no longer find or reproduce the situation where \s-1GLOBAL\s0 wasn't allowed, but
874
I know there was one.
876
.IX Item "graph_char"
877
Defines the character to use when drawing graphs in \*(L"S: Variables & Status\*(R"
879
.IP "header_highlight" 4
880
.IX Item "header_highlight"
881
Defines how to highlight column headers. This only works if Term::ANSIColor is
882
available. Valid values are 'bold' and 'underline'.
885
Hides column headers globally.
888
The interval at which innotop will refresh its data (ticks). The interval is
889
implemented as a sleep time between ticks, so the true interval will vary
890
depending on how long it takes innotop to fetch and render data.
892
This variable accepts fractions of a second.
895
The mode in which innotop should start. Allowable arguments are the same as the
896
key presses that select a mode interactively. See \*(L"\s-1MODES\s0\*(R".
898
.IX Item "num_digits"
899
How many digits to show in fractional numbers and percents. This variable's
900
range is between 0 and 9 and can be set directly from \*(L"S: Variables & Status\*(R"
901
mode with the '+' and '\-' keys. It is used in the \*(L"set_precision\*(R",
902
\&\*(L"shorten\*(R", and \*(L"percent\*(R" transformations.
903
.IP "num_status_sets" 4
904
.IX Item "num_status_sets"
905
Controls how many sets of status variables to display in pivoted \*(L"S: Variables & Status\*(R" mode. It also controls the number of old sets of variables innotop
906
keeps in its memory, so the larger this variable is, the more memory innotop
909
.IX Item "plugin_dir"
910
Specifies where plugins can be found. By default, innotop stores plugins in the
911
\&'plugins' subdirectory of your innotop configuration directory.
914
Whether the configuration file is readonly. This cannot be set interactively,
915
because it would prevent itself from being written to the configuration file.
916
.IP "show_cxn_errors" 4
917
.IX Item "show_cxn_errors"
918
Makes innotop print connection errors to \s-1STDOUT\s0. See \*(L"\s-1ERROR\s0 \s-1HANDLING\s0\*(R".
919
.IP "show_cxn_errors_in_tbl" 4
920
.IX Item "show_cxn_errors_in_tbl"
921
Makes innotop display connection errors as rows in the first table on screen.
922
See \*(L"\s-1ERROR\s0 \s-1HANDLING\s0\*(R".
924
.IX Item "show_percent"
925
Adds a '%' character after the value returned by the \*(L"percent\*(R"
927
.IP "show_statusbar" 4
928
.IX Item "show_statusbar"
929
Controls whether to show the status bar in the display. See \*(L"\s-1INNOTOP\s0 \s-1STATUS\s0\*(R".
931
.IX Item "skip_innodb"
932
Disables fetching \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0, in case your server(s) do not have InnoDB
933
enabled and you don't want innotop to try to fetch it. This can also be useful
934
when you don't have the \s-1SUPER\s0 privilege, required to run \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0.
936
.IX Item "status_inc"
937
Whether to show absolute or incremental values for status variables.
938
Incremental values are calculated as an offset from the last value innotop saw
939
for that variable. This is a global setting, but will probably become
940
mode-specific at some point. Right now it is honored a bit inconsistently; some
941
modes don't pay attention to it.
947
This section holds a list of package names of active plugins. If the plugin
948
exists, innotop will activate it. See \*(L"\s-1PLUGINS\s0\*(R" for more information.
951
This section holds user-defined filters (see \*(L"\s-1FILTERS\s0\*(R"). Each line is in the
952
format filter_name=text='filter text' tbls='table list'.
954
The filter text is the text of the subroutine's code. The table list is a list
955
of tables to which the filter can apply. By default, user-defined filters apply
956
to the table for which they were created, but you can manually override that by
957
editing the definition in the configuration file.
958
.IP "active_filters" 4
959
.IX Item "active_filters"
960
This section stores which filters are active on each table. Each line is in the
961
format table_name=filter_list.
964
This section stores user-defined or user-customized columns (see \*(L"\s-1COLUMNS\s0\*(R").
965
Each line is in the format col_name=properties, where the properties are a
966
name=quoted\-value list.
968
.IX Item "connections"
969
This section holds the server connections you have defined. Each line is in the
970
format name=properties, where the properties are a name=value list. The
971
properties are self\-explanatory, and the only one that is treated specially is
972
\&'pass' which is only present if 'savepass' is set. See \*(L"\s-1SERVER\s0 \s-1CONNECTIONS\s0\*(R".
973
.IP "active_connections" 4
974
.IX Item "active_connections"
975
This section holds a list of which connections are active in each mode. Each
976
line is in the format mode_name=connection_list.
977
.IP "server_groups" 4
978
.IX Item "server_groups"
979
This section holds server groups. Each line is in the format
980
name=connection_list. See \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R".
981
.IP "active_server_groups" 4
982
.IX Item "active_server_groups"
983
This section holds a list of which server group is active in each mode. Each
984
line is in the format mode_name=server_group.
985
.IP "max_values_seen" 4
986
.IX Item "max_values_seen"
987
This section holds the maximum values seen for variables. This is used to scale
988
the graphs in \*(L"S: Variables & Status\*(R" mode. Each line is in the format
990
.IP "active_columns" 4
991
.IX Item "active_columns"
992
This section holds table column lists. Each line is in the format
993
tbl_name=column_list. See \*(L"\s-1COLUMNS\s0\*(R".
996
This section holds the sort definition. Each line is in the format
997
tbl_name=column_list. If a column is prefixed with '\-', that column sorts
998
descending. See \*(L"\s-1SORTING\s0\*(R".
999
.IP "visible_tables" 4
1000
.IX Item "visible_tables"
1001
This section defines which tables are visible in each mode. Each line is in the
1002
format mode_name=table_list. See \*(L"\s-1TABLES\s0\*(R".
1005
This section defines variable sets for use in \*(L"S: Status & Variables\*(R" mode.
1006
Each line is in the format name=variable_list. See \*(L"\s-1VARIABLE\s0 \s-1SETS\s0\*(R".
1009
This section defines colorization rules. Each line is in the format
1010
tbl_name=property_list. See \*(L"\s-1COLORS\s0\*(R".
1011
.IP "stmt_sleep_times" 4
1012
.IX Item "stmt_sleep_times"
1013
This section contains statement sleep times. Each line is in the format
1014
statement_name=sleep_time. See \*(L"S: Statement Sleep Times\*(R".
1017
This section contains column lists for table group_by expressions. Each line is
1018
in the format tbl_name=column_list. See \*(L"\s-1GROUPING\s0\*(R".
1020
.IX Header "CUSTOMIZING"
1021
You can customize innotop a great deal. For example, you can:
1023
Choose which tables to display, and in what order.
1025
Choose which columns are in those tables, and create new columns.
1027
Filter which rows display with built-in filters, user-defined filters, and
1030
Sort the rows to put important data first or group together related rows.
1032
Highlight rows with color.
1034
Customize the alignment, width, and formatting of columns, and apply
1035
transformations to columns to extract parts of their values or format the values
1036
as you wish (for example, shortening large numbers to familiar units).
1038
Design your own expressions to extract and combine data as you need. This gives
1039
you unlimited flexibility.
1041
All these and more are explained in the following sections.
1043
.IX Subsection "TABLES"
1044
A table is what you'd expect: a collection of columns. It also has some other
1045
properties, such as a caption. Filters, sorting rules, and colorization rules
1046
belong to tables and are covered in later sections.
1048
Internally, table meta-data is defined in a data structure called \f(CW%tbl_meta\fR.
1049
This hash holds all built-in table definitions, which contain a lot of default
1050
instructions to innotop. The meta-data includes the caption, a list of columns
1051
the user has customized, a list of columns, a list of visible columns, a list of
1052
filters, color rules, a sort-column list, sort direction, and some information
1053
about the table's data sources. Most of this is customizable via the table
1054
editor (see \*(L"\s-1TABLE\s0 \s-1EDITOR\s0\*(R").
1056
You can choose which tables to show by pressing the '$' key. See \*(L"\s-1MODES\s0\*(R" and
1057
\&\*(L"\s-1TABLES\s0\*(R".
1059
The table life-cycle is as follows:
1061
Each table begins with a data source, which is an array of hashes. See below
1062
for details on data sources.
1064
Each element of the data source becomes a row in the final table.
1066
For each element in the data source, innotop extracts values from the source and
1067
creates a row. This row is another hash, which later steps will refer to as
1068
\&\f(CW$set\fR. The values innotop extracts are determined by the table's columns. Each
1069
column has an extraction subroutine, compiled from an expression (see
1070
\&\*(L"\s-1EXPRESSIONS\s0\*(R"). The resulting row is a hash whose keys are named the same as
1073
innotop filters the rows, removing those that don't need to be displayed. See
1074
\&\*(L"\s-1FILTERS\s0\*(R".
1076
innotop sorts the rows. See \*(L"\s-1SORTING\s0\*(R".
1078
innotop groups the rows together, if specified. See \*(L"\s-1GROUPING\s0\*(R".
1080
innotop colorizes the rows. See \*(L"\s-1COLORS\s0\*(R".
1082
innotop transforms the column values in each row. See \*(L"\s-1TRANSFORMATIONS\s0\*(R".
1084
innotop optionally pivots the rows (see \*(L"\s-1PIVOTING\s0\*(R"), then filters and sorts
1087
innotop formats and justifies the rows as a table. During this step, innotop
1088
applies further formatting to the column values, including alignment, maximum
1089
and minimum widths. innotop also does final error checking to ensure there are
1090
no crashes due to undefined values. innotop then adds a caption if specified,
1091
and the table is ready to print.
1093
The lifecycle is slightly different if the table is pivoted, as noted above. To
1094
clarify, if the table is pivoted, the process is extract, group, transform,
1095
pivot, filter, sort, create. If it's not pivoted, the process is extract,
1096
filter, sort, group, color, transform, create. This slightly convoluted process
1097
doesn't map all that well to \s-1SQL\s0, but pivoting complicates things pretty
1098
thoroughly. Roughly speaking, filtering and sorting happen as late as needed to
1099
effect the final result as you might expect, but as early as possible for
1102
Each built-in table is described below:
1103
.IP "adaptive_hash_index" 4
1104
.IX Item "adaptive_hash_index"
1105
Displays data about InnoDB's adaptive hash index. Data source:
1106
\&\*(L"\s-1STATUS_VARIABLES\s0\*(R".
1108
.IX Item "buffer_pool"
1109
Displays data about InnoDB's buffer pool. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1111
.IX Item "cmd_summary"
1112
Displays weighted status variables. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1113
.IP "deadlock_locks" 4
1114
.IX Item "deadlock_locks"
1115
Shows which locks were held and waited for by the last detected deadlock. Data
1116
source: \*(L"\s-1DEADLOCK_LOCKS\s0\*(R".
1117
.IP "deadlock_transactions" 4
1118
.IX Item "deadlock_transactions"
1119
Shows transactions involved in the last detected deadlock. Data source:
1120
\&\*(L"\s-1DEADLOCK_TRANSACTIONS\s0\*(R".
1123
Shows the output of \s-1EXPLAIN\s0. Data source: \*(L"\s-1EXPLAIN\s0\*(R".
1124
.IP "file_io_misc" 4
1125
.IX Item "file_io_misc"
1126
Displays data about InnoDB's file and I/O operations. Data source:
1127
\&\*(L"\s-1STATUS_VARIABLES\s0\*(R".
1130
Displays various data about InnoDB's last foreign key error. Data source:
1131
\&\*(L"\s-1STATUS_VARIABLES\s0\*(R".
1132
.IP "innodb_locks" 4
1133
.IX Item "innodb_locks"
1134
Displays InnoDB locks. Data source: \*(L"\s-1INNODB_LOCKS\s0\*(R".
1135
.IP "innodb_transactions" 4
1136
.IX Item "innodb_transactions"
1137
Displays data about InnoDB's current transactions. Data source:
1138
\&\*(L"\s-1INNODB_TRANSACTIONS\s0\*(R".
1139
.IP "insert_buffers" 4
1140
.IX Item "insert_buffers"
1141
Displays data about InnoDB's insert buffer. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1143
.IX Item "io_threads"
1144
Displays data about InnoDB's I/O threads. Data source: \*(L"\s-1IO_THREADS\s0\*(R".
1145
.IP "log_statistics" 4
1146
.IX Item "log_statistics"
1147
Displays data about InnoDB's logging system. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1148
.IP "master_status" 4
1149
.IX Item "master_status"
1150
Displays replication master status. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1152
.IX Item "open_tables"
1153
Displays open tables. Data source: \*(L"\s-1OPEN_TABLES\s0\*(R".
1154
.IP "page_statistics" 4
1155
.IX Item "page_statistics"
1156
Displays InnoDB page statistics. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1158
.IX Item "pending_io"
1159
Displays InnoDB pending I/O operations. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1161
.IX Item "processlist"
1162
Displays current MySQL processes (threads/connections). Data source:
1163
\&\*(L"\s-1PROCESSLIST\s0\*(R".
1166
Displays various status values. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1167
.IP "row_operation_misc" 4
1168
.IX Item "row_operation_misc"
1169
Displays data about InnoDB's row operations. Data source:
1170
\&\*(L"\s-1STATUS_VARIABLES\s0\*(R".
1171
.IP "row_operations" 4
1172
.IX Item "row_operations"
1173
Displays data about InnoDB's row operations. Data source:
1174
\&\*(L"\s-1STATUS_VARIABLES\s0\*(R".
1176
.IX Item "semaphores"
1177
Displays data about InnoDB's semaphores and mutexes. Data source:
1178
\&\*(L"\s-1STATUS_VARIABLES\s0\*(R".
1179
.IP "slave_io_status" 4
1180
.IX Item "slave_io_status"
1181
Displays data about the slave I/O thread. Data source:
1182
\&\*(L"\s-1STATUS_VARIABLES\s0\*(R".
1183
.IP "slave_sql_status" 4
1184
.IX Item "slave_sql_status"
1185
Displays data about the slave \s-1SQL\s0 thread. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1188
Displays various InnoDB status values. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1190
.IX Item "var_status"
1191
Displays user-configurable data. Data source: \*(L"\s-1STATUS_VARIABLES\s0\*(R".
1193
.IX Item "wait_array"
1194
Displays data about InnoDB's \s-1OS\s0 wait array. Data source: \*(L"\s-1OS_WAIT_ARRAY\s0\*(R".
1195
.Sh "\s-1COLUMNS\s0"
1196
.IX Subsection "COLUMNS"
1197
Columns belong to tables. You can choose a table's columns by pressing the '^'
1198
key, which starts the \*(L"\s-1TABLE\s0 \s-1EDITOR\s0\*(R" and lets you choose and edit columns.
1199
Pressing 'e' from within the table editor lets you edit the column's properties:
1201
hdr: a column header. This appears in the first row of the table.
1203
just: justification. '\-' means left-justified and '' means right\-justified,
1204
just as with printf formatting codes (not a coincidence).
1206
dec: whether to further align the column on the decimal point.
1208
num: whether the column is numeric. This affects how values are sorted
1209
(lexically or numerically).
1211
label: a small note about the column, which appears in dialogs that help the
1212
user choose columns.
1214
src: an expression that innotop uses to extract the column's data from its
1215
source (see \*(L"\s-1DATA\s0 \s-1SOURCES\s0\*(R"). See \*(L"\s-1EXPRESSIONS\s0\*(R" for more on expressions.
1217
minw: specifies a minimum display width. This helps stabilize the display,
1218
which makes it easier to read if the data is changing frequently.
1220
maxw: similar to minw.
1222
trans: a list of column transformations. See \*(L"\s-1TRANSFORMATIONS\s0\*(R".
1224
agg: an aggregate function. See \*(L"\s-1GROUPING\s0\*(R". The default is \*(L"first\*(R".
1226
aggonly: controls whether the column only shows when grouping is enabled on the
1227
table (see \*(L"\s-1GROUPING\s0\*(R"). By default, this is disabled. This means columns
1228
will always be shown by default, whether grouping is enabled or not. If a
1229
column's aggonly is set true, the column will appear when you toggle grouping on
1230
the table. Several columns are set this way, such as the count column on
1231
\&\*(L"processlist\*(R" and \*(L"innodb_transactions\*(R", so you don't see a count when the
1232
grouping isn't enabled, but you do when it is.
1233
.Sh "\s-1FILTERS\s0"
1234
.IX Subsection "FILTERS"
1235
Filters remove rows from the display. They behave much like a \s-1WHERE\s0 clause in
1236
\&\s-1SQL\s0. innotop has several built-in filters, which remove irrelevant information
1237
like inactive queries, but you can define your own as well. innotop also lets
1238
you create quick\-filters, which do not get saved to the configuration file, and
1239
are just an easy way to quickly view only some rows.
1241
You can enable or disable a filter on any table. Press the '%' key (mnemonic: %
1242
looks kind of like a line being filtered between two circles) and choose which
1243
table you want to filter, if asked. You'll then see a list of possible filters
1244
and a list of filters currently enabled for that table. Type the names of
1245
filters you want to apply and press Enter.
1247
\fIUSER-DEFINED \s-1FILTERS\s0\fR
1248
.IX Subsection "USER-DEFINED FILTERS"
1250
If you type a name that doesn't exist, innotop will prompt you to create the
1251
filter. Filters are easy to create if you know Perl, and not hard if you don't.
1252
What you're doing is creating a subroutine that returns true if the row should
1253
be displayed. The row is a hash reference passed to your subroutine as \f(CW$set\fR.
1255
For example, imagine you want to filter the processlist table so you only see
1256
queries that have been running more than five minutes. Type a new name for your
1257
filter, and when prompted for the subroutine body, press \s-1TAB\s0 to initiate your
1258
terminal's auto\-completion. You'll see the names of the columns in the
1259
\&\*(L"processlist\*(R" table (innotop generally tries to help you with auto-completion
1260
lists). You want to filter on the 'time' column. Type the text \*(L"$set\->{time} >
1261
300\*(R" to return true when the query is more than five minutes old. That's all
1264
In other words, the code you're typing is surrounded by an implicit context,
1265
which looks like this:
1269
\& my ( $set ) = @_;
1274
If your filter doesn't work, or if something else suddenly behaves differently,
1275
you might have made an error in your filter, and innotop is silently catching
1276
the error. Try enabling \*(L"debug\*(R" to make innotop throw an error instead.
1279
.IX Subsection "QUICK-FILTERS"
1281
innotop's quick-filters are a shortcut to create a temporary filter that doesn't
1282
persist when you restart innotop. To create a quick\-filter, press the '/' key.
1283
innotop will prompt you for the column name and filter text. Again, you can use
1284
auto-completion on column names. The filter text can be just the text you want
1285
to \*(L"search for.\*(R" For example, to filter the \*(L"processlist\*(R" table on queries
1286
that refer to the products table, type '/' and then 'info product'.
1288
The filter text can actually be any Perl regular expression, but of course a
1289
literal string like 'product' works fine as a regular expression.
1291
Behind the scenes innotop compiles the quick-filter into a specially tagged
1292
filter that is otherwise like any other filter. It just isn't saved to the
1295
To clear quick\-filters, press the '\e' key and innotop will clear them all at
1297
.Sh "\s-1SORTING\s0"
1298
.IX Subsection "SORTING"
1299
innotop has sensible built-in defaults to sort the most important rows to the
1300
top of the table. Like anything else in innotop, you can customize how any
1303
To start the sort dialog, start the \*(L"\s-1TABLE\s0 \s-1EDITOR\s0\*(R" with the '^' key, choose a
1304
table if necessary, and press the 's' key. You'll see a list of columns you can
1305
use in the sort expression and the current sort expression, if any. Enter a
1306
list of columns by which you want to sort and press Enter. If you want to
1307
reverse sort, prefix the column name with a minus sign. For example, if you
1308
want to sort by column a ascending, then column b descending, type 'a \-b'. You
1309
can also explicitly add a + in front of columns you want to sort ascending, but
1312
Some modes have keys mapped to open this dialog directly, and to quickly reverse
1313
sort direction. Press '?' as usual to see which keys are mapped in any mode.
1314
.Sh "\s-1GROUPING\s0"
1315
.IX Subsection "GROUPING"
1316
innotop can group, or aggregate, rows together (I use the terms
1317
interchangeably). This is quite similar to an \s-1SQL\s0 \s-1GROUP\s0 \s-1BY\s0 clause. You can
1318
specify to group on certain columns, or if you don't specify any, the entire set
1319
of rows is treated as one group. This is quite like \s-1SQL\s0 so far, but unlike \s-1SQL\s0,
1320
you can also select un-grouped columns. innotop actually aggregates every
1321
column. If you don't explicitly specify a grouping function, the default is
1322
\&'first'. This is basically a convenience so you don't have to specify an
1323
aggregate function for every column you want in the result.
1325
You can quickly toggle grouping on a table with the '=' key, which toggles its
1326
aggregate property. This property doesn't persist to the config file.
1328
The columns by which the table is grouped are specified in its group_by
1329
property. When you turn grouping on, innotop places the group_by columns at the
1330
far left of the table, even if they're not supposed to be visible. The rest of
1331
the visible columns appear in order after them.
1333
Two tables have default group_by lists and a count column built in:
1334
\&\*(L"processlist\*(R" and \*(L"innodb_transactions\*(R". The grouping is by connection
1335
and status, so you can quickly see how many queries or transactions are in a
1336
given status on each server you're monitoring. The time columns are aggregated
1337
as a sum; other columns are left at the default 'first' aggregation.
1339
By default, the table shown in \*(L"S: Variables & Status\*(R" mode also uses
1340
grouping so you can monitor variables and status across many servers. The
1341
default aggregation function in this mode is 'avg'.
1343
Valid grouping functions are defined in the \f(CW%agg_funcs\fR hash. They include
1346
Returns the first element in the group.
1349
Returns the number of elements in the group, including undefined elements, much
1350
like \s-1SQL\s0's \s-1COUNT\s0(*).
1353
Returns the average of defined elements in the group.
1356
Returns the sum of elements in the group.
1358
Here's an example of grouping at work. Suppose you have a very busy server with
1359
hundreds of open connections, and you want to see how many connections are in
1360
what status. Using the built-in grouping rules, you can press 'Q' to enter
1361
\&\*(L"Q: Query List\*(R" mode. Press '=' to toggle grouping (if necessary, select the
1362
\&\*(L"processlist\*(R" table when prompted).
1364
Your display might now look like the following:
1367
\& Query List (? for help) localhost, 32:33, 0.11 QPS, 1 thd, 5.0.38\-log
1371
\& CXN Cmd Cnt ID User Host Time Query
1372
\& localhost Query 49 12933 webusr localhost 19:38 SELECT * FROM
1373
\& localhost Sending Da 23 2383 webusr localhost 12:43 SELECT col1,
1374
\& localhost Sleep 120 140 webusr localhost 5:18:12
1375
\& localhost Statistics 12 19213 webusr localhost 01:19 SELECT * FROM
1378
That's actually quite a worrisome picture. You've got a lot of idle connections
1379
(Sleep), and some connections executing queries (Query and Sending Data).
1380
That's okay, but you also have a lot in Statistics status, collectively spending
1381
over a minute. That means the query optimizer is having a really hard time
1382
optimizing your statements. Something is wrong; it should normally take
1383
milliseconds to optimize queries. You might not have seen this pattern if you
1384
didn't look at your connections in aggregate. (This is a made-up example, but
1385
it can happen in real life).
1386
.Sh "\s-1PIVOTING\s0"
1387
.IX Subsection "PIVOTING"
1388
innotop can pivot a table for more compact display, similar to a Pivot Table in
1389
a spreadsheet (also known as a crosstab). Pivoting a table makes columns into
1390
rows. Assume you start with this table:
1399
After pivoting, the table will look like this:
1408
To get reasonable results, you might need to group as well as pivoting.
1409
innotop currently does this for \*(L"S: Variables & Status\*(R" mode.
1411
.IX Subsection "COLORS"
1412
By default, innotop highlights rows with color so you can see at a glance which
1413
rows are more important. You can customize the colorization rules and add your
1414
own to any table. Open the table editor with the '^' key, choose a table if
1415
needed, and press 'o' to open the color editor dialog.
1417
The color editor dialog displays the rules applied to the table, in the order
1418
they are evaluated. Each row is evaluated against each rule to see if the rule
1419
matches the row; if it does, the row gets the specified color, and no further
1420
rules are evaluated. The rules look like the following:
1423
\& state eq Locked black on_red
1424
\& cmd eq Sleep white
1425
\& user eq system user white
1426
\& cmd eq Connect white
1427
\& cmd eq Binlog Dump white
1429
\& time > 120 yellow
1434
This is the default rule set for the \*(L"processlist\*(R" table. In order of
1435
priority, these rules make locked queries black on a red background, \*(L"gray out\*(R"
1436
connections from replication and sleeping queries, and make queries turn from
1437
cyan to red as they run longer.
1439
(For some reason, the \s-1ANSI\s0 color code \*(L"white\*(R" is actually a light gray. Your
1440
terminal's display may vary; experiment to find colors you like).
1442
You can use keystrokes to move the rules up and down, which re-orders their
1443
priority. You can also delete rules and add new ones. If you add a new rule,
1444
innotop prompts you for the column, an operator for the comparison, a value
1445
against which to compare the column, and a color to assign if the rule matches.
1446
There is auto-completion and prompting at each step.
1448
The value in the third step needs to be correctly quoted. innotop does not try
1449
to quote the value because it doesn't know whether it should treat the value as
1450
a string or a number. If you want to compare the column against a string, as
1451
for example in the first rule above, you should enter 'Locked' surrounded by
1452
quotes. If you get an error message about a bareword, you probably should have
1454
.Sh "\s-1EXPRESSIONS\s0"
1455
.IX Subsection "EXPRESSIONS"
1456
Expressions are at the core of how innotop works, and are what enables you to
1457
extend innotop as you wish. Recall the table lifecycle explained in
1458
\&\*(L"\s-1TABLES\s0\*(R". Expressions are used in the earliest step, where it extracts
1459
values from a data source to form rows.
1461
It does this by calling a subroutine for each column, passing it the source data
1462
set, a set of current values, and a set of previous values. These are all
1463
needed so the subroutine can calculate things like the difference between this
1464
tick and the previous tick.
1466
The subroutines that extract the data from the set are compiled from
1467
expressions. This gives significantly more power than just naming the values to
1468
fill the columns, because it allows the column's value to be calculated from
1469
whatever data is necessary, but avoids the need to write complicated and lengthy
1472
innotop begins with a string of text that can look as simple as a value's name
1473
or as complicated as a full-fledged Perl expression. It looks at each
1474
\&'bareword' token in the string and decides whether it's supposed to be a key
1475
into the \f(CW$set\fR hash. A bareword is an unquoted value that isn't already
1476
surrounded by code-ish things like dollar signs or curly brackets. If innotop
1477
decides that the bareword isn't a function or other valid Perl code, it converts
1478
it into a hash access. After the whole string is processed, innotop compiles a
1479
subroutine, like this:
1482
\& sub compute_column_value {
1483
\& my ( $set, $cur, $pre ) = @_;
1484
\& my $val = # EXPANDED STRING GOES HERE
1489
Here's a concrete example, taken from the header table \*(L"q_header\*(R" in \*(L"Q: Query List\*(R" mode. This expression calculates the qps, or Queries Per Second,
1490
column's values, from the values returned by \s-1SHOW\s0 \s-1STATUS:\s0
1493
\& Questions/Uptime_hires
1496
innotop decides both words are barewords, and transforms this expression into
1497
the following Perl code:
1500
\& $set\->{Questions}/$set\->{Uptime_hires}
1503
When surrounded by the rest of the subroutine's code, this is executable Perl
1504
that calculates a high-resolution queries-per-second value.
1506
The arguments to the subroutine are named \f(CW$set\fR, \f(CW$cur\fR, and \f(CW$pre\fR. In most cases,
1507
\&\f(CW$set\fR and \f(CW$cur\fR will be the same values. However, if \*(L"status_inc\*(R" is set, \f(CW$cur\fR
1508
will not be the same as \f(CW$set\fR, because \f(CW$set\fR will already contain values that are
1509
the incremental difference between \f(CW$cur\fR and \f(CW$pre\fR.
1511
Every column in innotop is computed by subroutines compiled in the same fashion.
1512
There is no difference between innotop's built-in columns and user-defined
1513
columns. This keeps things consistent and predictable.
1514
.Sh "\s-1TRANSFORMATIONS\s0"
1515
.IX Subsection "TRANSFORMATIONS"
1516
Transformations change how a value is rendered. For example, they can take a
1517
number of seconds and display it in H:M:S format. The following transformations
1521
Adds commas to large numbers every three decimal places.
1522
.IP "dulint_to_int" 4
1523
.IX Item "dulint_to_int"
1524
Accepts two unsigned integers and converts them into a single longlong. This is
1525
useful for certain operations with InnoDB, which uses two integers as
1526
transaction identifiers, for example.
1527
.IP "no_ctrl_char" 4
1528
.IX Item "no_ctrl_char"
1529
Removes quoted control characters from the value. This is affected by the
1530
\&\*(L"charset\*(R" configuration variable.
1532
This transformation only operates within quoted strings, for example, values to
1533
a \s-1SET\s0 clause in an \s-1UPDATE\s0 statement. It will not alter the \s-1UPDATE\s0 statement,
1534
but will collapse the quoted string to [\s-1BINARY\s0] or [\s-1TEXT\s0], depending on the
1538
Converts a number to a percentage by multiplying it by two, formatting it with
1539
\&\*(L"num_digits\*(R" digits after the decimal point, and optionally adding a percent
1540
sign (see \*(L"show_percent\*(R").
1541
.IP "secs_to_time" 4
1542
.IX Item "secs_to_time"
1543
Formats a number of seconds as time in days+hours:minutes:seconds format.
1544
.IP "set_precision" 4
1545
.IX Item "set_precision"
1546
Formats numbers with \*(L"num_digits\*(R" number of digits after the decimal point.
1549
Formats a number as a unit of 1024 (k/M/G/T) and with \*(L"num_digits\*(R" number of
1550
digits after the decimal point.
1551
.Sh "\s-1TABLE\s0 \s-1EDITOR\s0"
1552
.IX Subsection "TABLE EDITOR"
1553
The innotop table editor lets you customize tables with keystrokes. You start
1554
the table editor with the '^' key. If there's more than one table on the
1555
screen, it will prompt you to choose one of them. Once you do, innotop will
1556
show you something like this:
1559
\& Editing table definition for Buffer Pool. Press ? for help, q to quit.
1563
\& name hdr label src
1564
\& cxn CXN Connection from which cxn
1565
\& buf_pool_size Size Buffer pool size IB_bp_buf_poo
1566
\& buf_free Free Bufs Buffers free in the b IB_bp_buf_fre
1567
\& pages_total Pages Pages total IB_bp_pages_t
1568
\& pages_modified Dirty Pages Pages modified (dirty IB_bp_pages_m
1569
\& buf_pool_hit_rate Hit Rate Buffer pool hit rate IB_bp_buf_poo
1570
\& total_mem_alloc Memory Total memory allocate IB_bp_total_m
1571
\& add_pool_alloc Add\(aql Pool Additonal pool alloca IB_bp_add_poo
1574
The first line shows which table you're editing, and reminds you again to press
1575
\&'?' for a list of key mappings. The rest is a tabular representation of the
1576
table's columns, because that's likely what you're trying to edit. However, you
1577
can edit more than just the table's columns; this screen can start the filter
1578
editor, color rule editor, and more.
1580
Each row in the display shows a single column in the table you're editing, along
1581
with a couple of its properties such as its header and source expression (see
1582
\&\*(L"\s-1EXPRESSIONS\s0\*(R").
1584
The key mappings are Vim\-style, as in many other places. Pressing 'j' and 'k'
1585
moves the highlight up or down. You can then (d)elete or (e)dit the highlighted
1586
column. You can also (a)dd a column to the table. This actually just activates
1587
one of the columns already defined for the table; it prompts you to choose from
1588
among the columns available but not currently displayed. Finally, you can
1589
re-order the columns with the '+' and '\-' keys.
1591
You can do more than just edit the columns with the table editor, you can also
1592
edit other properties, such as the table's sort expression and group-by
1593
expression. Press '?' to see the full list, of course.
1595
If you want to really customize and create your own column, as opposed to just
1596
activating a built-in one that's not currently displayed, press the (n)ew key,
1597
and innotop will prompt you for the information it needs:
1599
The column name: this needs to be a word without any funny characters, e.g. just
1600
letters, numbers and underscores.
1602
The column header: this is the label that appears at the top of the column, in
1603
the table header. This can have spaces and funny characters, but be careful not
1604
to make it too wide and waste space on\-screen.
1606
The column's data source: this is an expression that determines what data from
1607
the source (see \*(L"\s-1TABLES\s0\*(R") innotop will put into the column. This can just be
1608
the name of an item in the source, or it can be a more complex expression, as
1609
described in \*(L"\s-1EXPRESSIONS\s0\*(R".
1611
Once you've entered the required data, your table has a new column. There is no
1612
difference between this column and the built-in ones; it can have all the same
1613
properties and behaviors. innotop will write the column's definition to the
1614
configuration file, so it will persist across sessions.
1616
Here's an example: suppose you want to track how many times your slaves have
1617
retried transactions. According to the MySQL manual, the
1618
Slave_retried_transactions status variable gives you that data: \*(L"The total
1619
number of times since startup that the replication slave \s-1SQL\s0 thread has retried
1620
transactions. This variable was added in version 5.0.4.\*(R" This is appropriate to
1621
add to the \*(L"slave_sql_status\*(R" table.
1623
To add the column, switch to the replication-monitoring mode with the 'M' key,
1624
and press the '^' key to start the table editor. When prompted, choose
1625
slave_sql_status as the table, then press 'n' to create the column. Type
1626
\&'retries' as the column name, 'Retries' as the column header, and
1627
\&'Slave_retried_transactions' as the source. Now the column is created, and you
1628
see the table editor screen again. Press 'q' to exit the table editor, and
1629
you'll see your column at the end of the table.
1631
.IX Header "VARIABLE SETS"
1632
Variable sets are used in \*(L"S: Variables & Status\*(R" mode to define more easily
1633
what variables you want to monitor. Behind the scenes they are compiled to a
1634
list of expressions, and then into a column list so they can be treated just
1635
like columns in any other table, in terms of data extraction and
1636
transformations. However, you're protected from the tedious details by a syntax
1637
that ought to feel very natural to you: a \s-1SQL\s0 \s-1SELECT\s0 list.
1639
The data source for variable sets, and indeed the entire S mode, is the
1640
combination of \s-1SHOW\s0 \s-1STATUS\s0, \s-1SHOW\s0 \s-1VARIABLES\s0, and \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. Imagine
1641
that you had a huge table with one column per variable returned from those
1642
statements. That's the data source for variable sets. You can now query this
1643
data source just like you'd expect. For example:
1646
\& Questions, Uptime, Questions/Uptime as QPS
1649
Behind the scenes innotop will split that variable set into three expressions,
1650
compile them and turn them into a table definition, then extract as usual. This
1651
becomes a \*(L"variable set,\*(R" or a \*(L"list of variables you want to monitor.\*(R"
1653
innotop lets you name and save your variable sets, and writes them to the
1654
configuration file. You can choose which variable set you want to see with the
1655
\&'c' key, or activate the next and previous sets with the '>' and '<' keys.
1656
There are many built-in variable sets as well, which should give you a good
1657
start for creating your own. Press 'e' to edit the current variable set, or
1658
just to see how it's defined. To create a new one, just press 'c' and type its
1661
You may want to use some of the functions listed in \*(L"\s-1TRANSFORMATIONS\s0\*(R" to help
1662
format the results. In particular, \*(L"set_precision\*(R" is often useful to limit
1663
the number of digits you see. Extending the above example, here's how:
1666
\& Questions, Uptime, set_precision(Questions/Uptime) as QPS
1669
Actually, this still needs a little more work. If your \*(L"interval\*(R" is less
1670
than one second, you might be dividing by zero because Uptime is incremental in
1671
this mode by default. Instead, use Uptime_hires:
1674
\& Questions, Uptime, set_precision(Questions/Uptime_hires) as QPS
1677
This example is simple, but it shows how easy it is to choose which variables
1678
you want to monitor.
1680
.IX Header "PLUGINS"
1681
innotop has a simple but powerful plugin mechanism by which you can extend
1682
or modify its existing functionality, and add new functionality. innotop's
1683
plugin functionality is event\-based: plugins register themselves to be called
1684
when events happen. They then have a chance to influence the event.
1686
An innotop plugin is a Perl module placed in innotop's \*(L"plugin_dir\*(R"
1687
directory. On \s-1UNIX\s0 systems, you can place a symbolic link to the module instead
1688
of putting the actual file there. innotop automatically discovers the file. If
1689
there is a corresponding entry in the \*(L"plugins\*(R" configuration file section,
1690
innotop loads and activates the plugin.
1692
The module must conform to innotop's plugin interface. Additionally, the source
1693
code of the module must be written in such a way that innotop can inspect the
1694
file and determine the package name and description.
1695
.Sh "Package Source Convention"
1696
.IX Subsection "Package Source Convention"
1697
innotop inspects the plugin module's source to determine the Perl package name.
1698
It looks for a line of the form \*(L"package Foo;\*(R" and if found, considers the
1699
plugin's package name to be Foo. Of course the package name can be a valid Perl
1700
package name, with double semicolons and so on.
1702
It also looks for a description in the source code, to make the plugin editor
1703
more human\-friendly. The description is a comment line of the form \*(L"#
1704
description: Foo\*(R", where \*(L"Foo\*(R" is the text innotop will consider to be the
1705
plugin's description.
1706
.Sh "Plugin Interface"
1707
.IX Subsection "Plugin Interface"
1708
The innotop plugin interface is quite simple: innotop expects the plugin to be
1709
an object-oriented module it can call certain methods on. The methods are
1710
.IP "new(%variables)" 4
1711
.IX Item "new(%variables)"
1712
This is the plugin's constructor. It is passed a hash of innotop's variables,
1713
which it can manipulate (see \*(L"Plugin Variables\*(R"). It must return a reference
1714
to the newly created plugin object.
1716
At construction time, innotop has only loaded the general configuration and
1717
created the default built-in variables with their default contents (which is
1718
quite a lot). Therefore, the state of the program is exactly as in the innotop
1719
source code, plus the configuration variables from the \*(L"general\*(R" section in
1722
If your plugin manipulates the variables, it is changing global data, which is
1723
shared by innotop and all plugins. Plugins are loaded in the order they're
1724
listed in the config file. Your plugin may load before or after another plugin,
1725
so there is a potential for conflict or interaction between plugins if they
1726
modify data other plugins use or modify.
1727
.IP "\fIregister_for_events()\fR" 4
1728
.IX Item "register_for_events()"
1729
This method must return a list of events in which the plugin is interested, if
1730
any. See \*(L"Plugin Events\*(R" for the defined events. If the plugin returns an
1731
event that's not defined, the event is ignored.
1732
.IP "event handlers" 4
1733
.IX Item "event handlers"
1734
The plugin must implement a method named the same as each event for which it has
1735
registered. In other words, if the plugin returns qw(foo bar) from
1736
\&\fIregister_for_events()\fR, it must have \fIfoo()\fR and \fIbar()\fR methods. These methods are
1737
callbacks for the events. See \*(L"Plugin Events\*(R" for more details about each
1739
.Sh "Plugin Variables"
1740
.IX Subsection "Plugin Variables"
1741
The plugin's constructor is passed a hash of innotop's variables, which it can
1742
manipulate. It is probably a good idea if the plugin object saves a copy of it
1743
for later use. The variables are defined in the innotop variable
1744
\&\f(CW%pluggable_vars\fR, and are as follows:
1746
.IX Item "action_for"
1747
A hashref of key mappings. These are innotop's global hot\-keys.
1749
.IX Item "agg_funcs"
1750
A hashref of functions that can be used for grouping. See \*(L"\s-1GROUPING\s0\*(R".
1753
The global configuration hash.
1755
.IX Item "connections"
1756
A hashref of connection specifications. These are just specifications of how to
1757
connect to a server.
1760
A hashref of innotop's database connections. These are actual \s-1DBI\s0 connection
1764
A hashref of filters applied to table rows. See \*(L"\s-1FILTERS\s0\*(R" for more.
1767
A hashref of modes. See \*(L"\s-1MODES\s0\*(R" for more.
1768
.IP "server_groups" 4
1769
.IX Item "server_groups"
1770
A hashref of server groups. See \*(L"\s-1SERVER\s0 \s-1GROUPS\s0\*(R".
1773
A hashref of innotop's table meta\-data, with one entry per table (see
1774
\&\*(L"\s-1TABLES\s0\*(R" for more information).
1776
.IX Item "trans_funcs"
1777
A hashref of transformation functions. See \*(L"\s-1TRANSFORMATIONS\s0\*(R".
1780
A hashref of variable sets. See \*(L"\s-1VARIABLE\s0 \s-1SETS\s0\*(R".
1782
.IX Subsection "Plugin Events"
1783
Each event is defined somewhere in the innotop source code. When innotop runs
1784
that code, it executes the callback function for each plugin that expressed its
1785
interest in the event. innotop passes some data for each event. The events are
1786
defined in the \f(CW%event_listener_for\fR variable, and are as follows:
1787
.ie n .IP "extract_values($set, $cur\fR, \f(CW$pre\fR, \f(CW$tbl)" 4
1788
.el .IP "extract_values($set, \f(CW$cur\fR, \f(CW$pre\fR, \f(CW$tbl\fR)" 4
1789
.IX Item "extract_values($set, $cur, $pre, $tbl)"
1790
This event occurs inside the function that extracts values from a data source.
1791
The arguments are the set of values, the current values, the previous values,
1794
.IX Item "set_to_tbl"
1795
Events are defined at many places in this subroutine, which is responsible for
1796
turning an arrayref of hashrefs into an arrayref of lines that can be printed to
1797
the screen. The events all pass the same data: an arrayref of rows and the name
1798
of the table being created. The events are set_to_tbl_pre_filter,
1799
set_to_tbl_pre_sort,set_to_tbl_pre_group, set_to_tbl_pre_colorize,
1800
set_to_tbl_pre_transform, set_to_tbl_pre_pivot, set_to_tbl_pre_create,
1801
set_to_tbl_post_create.
1802
.IP "draw_screen($lines)" 4
1803
.IX Item "draw_screen($lines)"
1804
This event occurs inside the subroutine that prints the lines to the screen.
1805
\&\f(CW$lines\fR is an arrayref of strings.
1806
.Sh "Simple Plugin Example"
1807
.IX Subsection "Simple Plugin Example"
1808
The easiest way to explain the plugin functionality is probably with a simple
1809
example. The following module adds a column to the beginning of every table and
1810
sets its value to 1.
1814
\& use warnings FATAL => \(aqall\(aq;
1818
\& package Innotop::Plugin::Example;
1819
\& # description: Adds an \(aqexample\(aq column to every table
1824
\& my ( $class, %vars ) = @_;
1825
\& # Store reference to innotop\(aqs variables in $self
1826
\& my $self = bless { %vars }, $class;
1830
\& # Design the example column
1832
\& hdr => \(aqExample\(aq,
1833
\& just => \(aq\(aq,
1836
\& label => \(aqExample\(aq,
1837
\& src => \(aqexample\(aq, # Get data from this column in the data source
1844
\& # Add the column to every table.
1845
\& my $tbl_meta = $vars{tbl_meta};
1846
\& foreach my $tbl ( values %$tbl_meta ) {
1847
\& # Add the column to the list of defined columns
1848
\& $tbl\->{cols}\->{example} = $col;
1849
\& # Add the column to the list of visible columns
1850
\& unshift @{$tbl\->{visible}}, \(aqexample\(aq;
1855
\& # Be sure to return a reference to the object.
1861
\& # I\(aqd like to be called when a data set is being rendered into a table, please.
1862
\& sub register_for_events {
1863
\& my ( $self ) = @_;
1864
\& return qw(set_to_tbl_pre_filter);
1869
\& # This method will be called when the event fires.
1870
\& sub set_to_tbl_pre_filter {
1871
\& my ( $self, $rows, $tbl ) = @_;
1872
\& # Set the example column\(aqs data source to the value 1.
1873
\& foreach my $row ( @$rows ) {
1874
\& $row\->{example} = 1;
1883
.IX Subsection "Plugin Editor"
1884
The plugin editor lets you view the plugins innotop discovered and activate or
1885
deactivate them. Start the editor by pressing $ to start the configuration
1886
editor from any mode. Press the 'p' key to start the plugin editor. You'll see
1887
a list of plugins innotop discovered. You can use the 'j' and 'k' keys to move
1888
the highlight to the desired one, then press the * key to toggle it active or
1889
inactive. Exit the editor and restart innotop for the changes to take effect.
1890
.SH "SQL STATEMENTS"
1891
.IX Header "SQL STATEMENTS"
1892
innotop uses a limited set of \s-1SQL\s0 statements to retrieve data from MySQL for
1893
display. The statements are customized depending on the server version against
1894
which they are executed; for example, on MySQL 5 and newer, \s-1INNODB_STATUS\s0
1895
executes \*(L"\s-1SHOW\s0 \s-1ENGINE\s0 \s-1INNODB\s0 \s-1STATUS\s0\*(R", while on earlier versions it executes
1896
\&\*(L"\s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0\*(R". The statements are as follows:
1899
\& Statement SQL executed
1900
\& =================== ===============================
1901
\& INNODB_STATUS SHOW [ENGINE] INNODB STATUS
1902
\& KILL_CONNECTION KILL
1903
\& KILL_QUERY KILL QUERY
1904
\& OPEN_TABLES SHOW OPEN TABLES
1905
\& PROCESSLIST SHOW FULL PROCESSLIST
1906
\& SHOW_MASTER_LOGS SHOW MASTER LOGS
1907
\& SHOW_MASTER_STATUS SHOW MASTER STATUS
1908
\& SHOW_SLAVE_STATUS SHOW SLAVE STATUS
1909
\& SHOW_STATUS SHOW [GLOBAL] STATUS
1910
\& SHOW_VARIABLES SHOW [GLOBAL] VARIABLES
1913
.IX Header "DATA SOURCES"
1914
Each time innotop extracts values to create a table (see \*(L"\s-1EXPRESSIONS\s0\*(R" and
1915
\&\*(L"\s-1TABLES\s0\*(R"), it does so from a particular data source. Largely because of the
1916
complex data extracted from \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0, this is slightly messy. \s-1SHOW\s0
1917
\&\s-1INNODB\s0 \s-1STATUS\s0 contains a mixture of single values and repeated values that form
1920
Whenever innotop fetches data from MySQL, it adds two extra bits to each set:
1921
cxn and Uptime_hires. cxn is the name of the connection from which the data
1922
came. Uptime_hires is a high-resolution version of the server's Uptime status
1923
variable, which is important if your \*(L"interval\*(R" setting is sub\-second.
1925
Here are the kinds of data sources from which data is extracted:
1926
.IP "\s-1STATUS_VARIABLES\s0" 4
1927
.IX Item "STATUS_VARIABLES"
1928
This is the broadest category, into which the most kinds of data fall. It
1929
begins with the combination of \s-1SHOW\s0 \s-1STATUS\s0 and \s-1SHOW\s0 \s-1VARIABLES\s0, but other sources
1930
may be included as needed, for example, \s-1SHOW\s0 \s-1MASTER\s0 \s-1STATUS\s0 and \s-1SHOW\s0 \s-1SLAVE\s0
1931
\&\s-1STATUS\s0, as well as many of the non-repeated values from \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0.
1932
.IP "\s-1DEADLOCK_LOCKS\s0" 4
1933
.IX Item "DEADLOCK_LOCKS"
1934
This data is extracted from the transaction list in the \s-1LATEST\s0 \s-1DETECTED\s0 \s-1DEADLOCK\s0
1935
section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. It is nested two levels deep: transactions, then
1937
.IP "\s-1DEADLOCK_TRANSACTIONS\s0" 4
1938
.IX Item "DEADLOCK_TRANSACTIONS"
1939
This data is from the transaction list in the \s-1LATEST\s0 \s-1DETECTED\s0 \s-1DEADLOCK\s0
1940
section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0. It is nested one level deep.
1941
.IP "\s-1EXPLAIN\s0" 4
1943
This data is from the result set returned by \s-1EXPLAIN\s0.
1944
.IP "\s-1INNODB_TRANSACTIONS\s0" 4
1945
.IX Item "INNODB_TRANSACTIONS"
1946
This data is from the \s-1TRANSACTIONS\s0 section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0.
1947
.IP "\s-1IO_THREADS\s0" 4
1948
.IX Item "IO_THREADS"
1949
This data is from the list of threads in the the \s-1FILE\s0 I/O section of \s-1SHOW\s0 \s-1INNODB\s0
1951
.IP "\s-1INNODB_LOCKS\s0" 4
1952
.IX Item "INNODB_LOCKS"
1953
This data is from the \s-1TRANSACTIONS\s0 section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0 and is nested
1955
.IP "\s-1OPEN_TABLES\s0" 4
1956
.IX Item "OPEN_TABLES"
1957
This data is from \s-1SHOW\s0 \s-1OPEN\s0 \s-1TABLES\s0.
1958
.IP "\s-1PROCESSLIST\s0" 4
1959
.IX Item "PROCESSLIST"
1960
This data is from \s-1SHOW\s0 \s-1FULL\s0 \s-1PROCESSLIST\s0.
1961
.IP "\s-1OS_WAIT_ARRAY\s0" 4
1962
.IX Item "OS_WAIT_ARRAY"
1963
This data is from the \s-1SEMAPHORES\s0 section of \s-1SHOW\s0 \s-1INNODB\s0 \s-1STATUS\s0 and is nested one
1964
level deep. It comes from the lines that look like this:
1967
\& \-\-Thread 1568861104 has waited at btr0cur.c line 424 ....
1969
.SH "MYSQL PRIVILEGES"
1970
.IX Header "MYSQL PRIVILEGES"
1972
You must connect to MySQL as a user who has the \s-1SUPER\s0 privilege for many of the
1975
If you don't have the \s-1SUPER\s0 privilege, you can still run some functions, but you
1976
won't necessarily see all the same data.
1978
You need the \s-1PROCESS\s0 privilege to see the list of currently running queries in Q
1981
You need special privileges to start and stop slave servers.
1983
You need appropriate privileges to create and drop the deadlock tables if needed
1984
(see \*(L"\s-1SERVER\s0 \s-1CONNECTIONS\s0\*(R").
1985
.SH "SYSTEM REQUIREMENTS"
1986
.IX Header "SYSTEM REQUIREMENTS"
1987
You need Perl to run innotop, of course. You also need a few Perl modules: \s-1DBI\s0,
1988
DBD::mysql, Term::ReadKey, and Time::HiRes. These should be included with most
1989
Perl distributions, but in case they are not, I recommend using versions
1990
distributed with your operating system or Perl distribution, not from \s-1CPAN\s0.
1991
Term::ReadKey in particular has been known to cause problems if installed from
1994
If you have Term::ANSIColor, innotop will use it to format headers more readably
1995
and compactly. (Under Microsoft Windows, you also need Win32::Console::ANSI for
1996
terminal formatting codes to be honored). If you install Term::ReadLine,
1997
preferably Term::ReadLine::Gnu, you'll get nice auto-completion support.
1999
I run innotop on Gentoo GNU/Linux, Debian and Ubuntu, and I've had feedback from
2000
people successfully running it on Red Hat, CentOS, Solaris, and Mac \s-1OSX\s0. I
2001
don't see any reason why it won't work on other UNIX-ish operating systems, but
2002
I don't know for sure. It also runs on Windows under ActivePerl without
2005
I use innotop on MySQL versions 3.23.58, 4.0.27, 4.1.0, 4.1.22, 5.0.26, 5.1.15,
2006
and 5.2.3. If it doesn't run correctly for you, that is a bug and I hope you
2010
$HOMEDIR/.innotop is used to store configuration information. Files include the
2011
configuration file innotop.ini, the core_dump file which contains verbose error
2012
messages if \*(L"debug\*(R" is enabled, and the plugins/ subdirectory.
2013
.SH "GLOSSARY OF TERMS"
2014
.IX Header "GLOSSARY OF TERMS"
2017
A tick is a refresh event, when innotop re-fetches data from connections and
2019
.SH "ACKNOWLEDGEMENTS"
2020
.IX Header "ACKNOWLEDGEMENTS"
2021
I'm grateful to the following people for various reasons, and hope I haven't
2022
forgotten to include anyone:
2025
Aurimas Mikalauskas,
2034
Google.com Site Reliability Engineers,
2049
The Gentoo MySQL Team,
2052
and probably more people I've neglected to include.
2054
(If I misspelled your name, it's probably because I'm afraid of putting
2055
international characters into this documentation; earlier versions of Perl might
2056
not be able to compile it then).
2057
.SH "COPYRIGHT, LICENSE AND WARRANTY"
2058
.IX Header "COPYRIGHT, LICENSE AND WARRANTY"
2059
This program is copyright (c) 2006 Baron Schwartz.
2060
Feedback and improvements are welcome.
2062
\&\s-1THIS\s0 \s-1PROGRAM\s0 \s-1IS\s0 \s-1PROVIDED\s0 \*(L"\s-1AS\s0 \s-1IS\s0\*(R" \s-1AND\s0 \s-1WITHOUT\s0 \s-1ANY\s0 \s-1EXPRESS\s0 \s-1OR\s0 \s-1IMPLIED\s0
2063
\&\s-1WARRANTIES\s0, \s-1INCLUDING\s0, \s-1WITHOUT\s0 \s-1LIMITATION\s0, \s-1THE\s0 \s-1IMPLIED\s0 \s-1WARRANTIES\s0 \s-1OF\s0
2064
\&\s-1MERCHANTIBILITY\s0 \s-1AND\s0 \s-1FITNESS\s0 \s-1FOR\s0 A \s-1PARTICULAR\s0 \s-1PURPOSE\s0.
2066
This program is free software; you can redistribute it and/or modify it under
2067
the terms of the \s-1GNU\s0 General Public License as published by the Free Software
2068
Foundation, version 2; \s-1OR\s0 the Perl Artistic License. On \s-1UNIX\s0 and similar
2069
systems, you can issue `man perlgpl' or `man perlartistic' to read these
2072
You should have received a copy of the \s-1GNU\s0 General Public License along with
2073
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
2074
Place, Suite 330, Boston, \s-1MA\s0 02111\-1307 \s-1USA\s0.
2076
Execute innotop and press '!' to see this information at any time.
2082
You can report bugs, ask for improvements, and get other help and support at
2083
<http://sourceforge.net/projects/innotop>. There are mailing lists, forums,
2084
a bug tracker, etc. Please use these instead of contacting me directly, as it
2085
makes my job easier and benefits others if the discussions are permanent and
2086
public. Of course, if you need to contact me in private, please do.