14
pt-log-player - Replay MySQL query logs.
22
Usage: pt-log-player [OPTION...] [DSN]
24
pt-log-player splits and plays slow log files.
26
Split slow.log on Thread_id into 16 session files, save in ./sessions:
31
pt-log-player --split Thread_id --session-files 16 --base-dir ./sessions slow.log
34
Play all those sessions on host1, save results in ./results:
39
pt-log-player --play ./sessions --base-dir ./results h=host1
42
Use pt-query-digest to summarize the results:
47
pt-query-digest ./results/*
56
The following section is included to inform users about the potential risks,
57
whether known or unknown, of using this tool. The two main categories of risks
58
are those created by the nature of the tool (e.g. read-only tools vs. read-write
59
tools) and those created by bugs.
61
This tool is meant to load a server as much as possible, for stress-testing
62
purposes. It is not designed to be used on production servers.
64
At the time of this release there is a bug which causes pt-log-player to
65
exceed max open files during "--split".
67
The authoritative source for updated information is always the online issue
68
tracking system. Issues that affect this tool will be marked as such. You can
69
see a list of such issues at the following URL:
70
`http://www.percona.com/bugs/pt-log-player <http://www.percona.com/bugs/pt-log-player>`_.
72
See also "BUGS" for more information on filing bugs and getting help.
80
pt-log-player does two things: it splits MySQL query logs into session files
81
and it plays (executes) queries in session files on a MySQL server. Only
82
session files can be played; slow logs cannot be played directly without
85
A session is a group of queries from the slow log that all share a common
86
attribute, usually Thread_id. The common attribute is specified with
87
"--split". Multiple sessions are saved into a single session file.
88
See "--session-files", "--max-sessions", "--base-file-name" and
89
"--base-dir". These session files are played with "--play".
91
pt-log-player will "--play" session files in parallel using N number of
92
"--threads". (They're not technically threads, but we call them that
93
anyway.) Each thread will play all the sessions in its given session files.
94
The sessions are played as fast as possible--there are no delays--because the
95
goal is to stress-test and load-test the server. So be careful using this
96
script on a production server!
98
Each "--play" thread writes its results to a separate file. These result
99
files are in slow log format so they can be aggregated and summarized with
100
pt-query-digest. See "OUTPUT".
108
Both "--split" and "--play" have two outputs: status messages printed to
109
STDOUT to let you know what the script is doing, and session or result files
110
written to separate files saved in "--base-dir". You can suppress all
111
output to STDOUT for each with "--quiet", or increase output with
114
The session files written by "--split" are simple text files containing
115
queries grouped into sessions. For example:
124
SELECT col FROM foo_tbl
127
The format of these session files is important: each query must be a single
128
line separated by a single blank line. And the "-- START SESSION" comment
129
tells pt-log-player where individual sessions begin and end so that "--play"
130
can correctly fake Thread_id in its result files.
132
The result files written by "--play" are in slow log format with a minimal
133
header: the only attributes printed are Thread_id, Query_time and Schema.
141
Specify at least one of "--play", "--split" or "--split-random".
143
"--play" and "--split" are mutually exclusive.
145
This tool accepts additional command-line arguments. Refer to the
146
"SYNOPSIS" and usage information for details.
153
Prompt for a password when connecting to MySQL.
159
type: string; default: ./
161
Base directory for "--split" session files and "--play" result file.
167
type: string; default: session
169
Base file name for "--split" session files and "--play" result file.
171
Each "--split" session file will be saved as <base-file-name>-N.txt, where
172
N is a four digit, zero-padded session ID. For example: session-0003.txt.
174
Each "--play" result file will be saved as <base-file-name>-results-PID.txt,
175
where PID is the process ID of the executing thread.
177
All files are saved in "--base-dir".
183
short form: -A; type: string; group: Play
185
Default character set. If the value is utf8, sets Perl's binmode on STDOUT to
186
utf8, passes the mysql_enable_utf8 option to DBD::mysql, and runs SET NAMES UTF8
187
after connecting to MySQL. Any other value sets binmode on STDOUT without the
188
utf8 layer, and runs SET NAMES after connecting to MySQL.
196
Read this comma-separated list of config files; if specified, this must be the
197
first option on the command line.
203
short form: -F; type: string
205
Only read mysql options from the given file.
211
Print which processes play which session files then exit.
217
type: string; group: Split
219
Discard "--split" events for which this Perl code doesn't return true.
221
This option only works with "--split".
223
This option allows you to inject Perl code into the tool to affect how the
224
tool runs. Usually your code should examine \ ``$event``\ to decided whether
225
or not to allow the event. \ ``$event``\ is a hashref of attributes and values of
226
the event being filtered. Or, your code could add new attribute-value pairs
227
to \ ``$event``\ for use by other options that accept event attributes as their
228
value. You can find an explanation of the structure of \ ``$event``\ at
229
`http://code.google.com/p/maatkit/wiki/EventAttributes <http://code.google.com/p/maatkit/wiki/EventAttributes>`_.
231
There are two ways to supply your code: on the command line or in a file.
232
If you supply your code on the command line, it is injected into the following
233
subroutine where \ ``$filter``\ is your code:
239
MKDEBUG && _d('callback: filter');
240
my( $event ) = shift;
241
( $filter ) && return $event;
245
Therefore you must ensure two things: first, that you correctly escape any
246
special characters that need to be escaped on the command line for your
247
shell, and two, that your code is syntactically valid when injected into
248
the subroutine above.
250
Here's an example filter supplied on the command line that discards
251
events that are not SELECT statements:
256
--filter '$event->{arg} =~ m/^select/i'
259
The second way to supply your code is in a file. If your code is too complex
260
to be expressed on the command line that results in valid syntax in the
261
subroutine above, then you need to put the code in a file and give the file
262
name as the value to "--filter". The file should not contain a shebang
263
(\ ``#!/usr/bin/perl``\ ) line. The entire contents of the file is injected into
264
the following subroutine:
270
MKDEBUG && _d('callback: filter');
271
my( $event ) = shift;
272
$filter && return $event;
276
That subroutine is almost identical to the one above except your code is
277
not wrapped in parentheses. This allows you to write multi-line code like:
292
Notice that the last line is not syntactically valid by itself, but it
293
becomes syntactically valid when injected into the subroutine because it
299
$event_ok && return $event;
302
If your code doesn't compile, the tool will die with an error. Even if your
303
code compiles, it may crash to tool during runtime if, for example, it tries
304
a pattern match an undefined value. No safeguards of any kind of provided so
317
short form: -h; type: string; group: Play
325
type: int; default: 1; group: Play
327
How many times each thread should play all its session files.
333
type: int; default: 5000000; group: Split
335
Maximum number of sessions to "--split".
337
By default, \ ``pt-log-player``\ tries to split every session from the log file.
338
For huge logs, however, this can result in millions of sessions. This
339
option causes only the first N number of sessions to be saved. All sessions
340
after this number are ignored, but sessions split before this number will
341
continue to have their queries split even if those queries appear near the end
342
of the log and after this number has been reached.
350
Play only SELECT and USE queries; ignore all others.
356
short form: -p; type: string; group: Play
358
Password to use when connecting.
366
Create the given PID file. The file contains the process ID of the script.
367
The PID file is removed when the script exits. Before starting, the script
368
checks if the PID file already exists. If it does not, then the script creates
369
and writes its own PID to it. If it does, then the script checks the following:
370
if the file contains a PID and a process is running with that PID, then
371
the script dies; or, if there is no process running with that PID, then the
372
script overwrites the file with its own PID and starts; else, if the file
373
contains no PID, then the script dies.
379
type: string; group: Play
381
Play (execute) session files created by "--split".
383
The argument to play must be a comma-separated list of session files
384
created by "--split" or a directory. If the argument is a directory,
385
ALL files in that directory will be played.
391
short form: -P; type: int; group: Play
393
Port number to use for connection.
401
Print queries instead of playing them; requires "--play".
403
You must also specify "--play" with "--print". Although the queries
404
will not be executed, "--play" is required to specify which session files to
413
Do not print anything; disables "--verbose".
421
Print "--play" results to files in "--base-dir".
427
type: int; default: 8; group: Split
429
Number of session files to create with "--split".
431
The number of session files should either be equal to the number of
432
"--threads" you intend to "--play" or be an even multiple of
433
"--threads". This number is important for maximum performance because it:
438
* allows each thread to have roughly the same amount of sessions to play
439
* avoids having to open/close many session files
440
* avoids disk IO overhead by doing large sequential reads
443
You may want to increase this number beyond "--threads" if each session
444
file becomes too large. For example, splitting a 20G log into 8 sessions
445
files may yield roughly eight 2G session files.
447
See also "--max-sessions".
453
type: string; group: Play; default: wait_timeout=10000
455
Set these MySQL variables. Immediately after connecting to MySQL, this string
456
will be appended to SET and executed.
462
short form: -S; type: string; group: Play
464
Socket file to use for connection.
470
type: string; group: Split
472
Split log by given attribute to create session files.
474
Valid attributes are any which appear in the log: Thread_id, Schema,
483
Split log without an attribute, write queries round-robin to session files.
485
This option, if specified, overrides "--split" and causes the log to be
486
split query-by-query, writing each query to the next session file in round-robin
487
style. If you don't care about "sessions" and just want to split a lot into
488
N many session files and the relation or order of the queries does not matter,
489
then use this option.
495
type: int; default: 2; group: Play
497
Number of threads used to play sessions concurrently.
499
Specifies the number of parallel processes to run. The default is 2. On
500
GNU/Linux machines, the default is the number of times 'processor' appears in
501
\ */proc/cpuinfo*\ . On Windows, the default is read from the environment.
502
In any case, the default is at least 2, even when there's only a single
505
See also "--session-files".
511
type: string; group: Split
513
The type of log to "--split" (default slowlog). The permitted types are
518
Split the output of running \ ``mysqlbinlog``\ against a binary log file.
519
Currently, splitting binary logs does not always work well depending
520
on what the binary logs contain. Be sure to check the session files
521
after splitting to ensure proper "OUTPUT".
523
If the binary log contains row-based replication data, you need to run
524
\ ``mysqlbinlog``\ with options \ ``--base64-output=decode-rows --verbose``\ ,
525
else invalid statements will be written to the session files.
531
Split a general log file.
537
Split a log file in any variation of MySQL slow-log format.
545
short form: -u; type: string; group: Play
547
User for login if not current user.
553
short form: -v; cumulative: yes; default: 0
555
Increase verbosity; can be specified multiple times.
557
This option is disabled by "--quiet".
563
Show version and exit.
567
--wait-between-sessions
569
type: array; default: 0; group: Play
573
The wait time is given in seconds with microsecond precision and can be either
574
a single value or a range. A single value causes an exact wait; example:
575
0.010 = wait 10 milliseconds. A range causes a random wait between
576
the given value times; example: 0.001,1 = random wait from 1 millisecond to
583
default: no; group: Play
585
Print warnings about SQL errors such as invalid queries to STDERR.
595
These DSN options are used to create a DSN. Each option is given like
596
\ ``option=value``\ . The options are case-sensitive, so P and p are not the
597
same option. There cannot be whitespace before or after the \ ``=``\ and
598
if the value contains whitespace it must be quoted. DSN options are
599
comma-separated. See the percona-toolkit manpage for full details.
604
dsn: charset; copy: yes
606
Default character set.
612
dsn: database; copy: yes
620
dsn: mysql_read_default_file; copy: yes
622
Only read default options from the given file
636
dsn: password; copy: yes
638
Password to use when connecting.
646
Port number to use for connection.
652
dsn: mysql_socket; copy: yes
654
Socket file to use for connection.
662
User for login if not current user.
672
Visit `http://www.percona.com/software/ <http://www.percona.com/software/>`_ to download the latest release of
673
Percona Toolkit. Or, to get the latest release from the command line:
678
wget percona.com/latest/percona-toolkit/PKG
681
Replace \ ``PKG``\ with \ ``tar``\ , \ ``rpm``\ , or \ ``deb``\ to download the package in that
682
format. You can also get individual tools from the latest release:
687
wget percona.com/latest/percona-toolkit/TOOL
690
Replace \ ``TOOL``\ with the name of any tool.
698
The environment variable \ ``PTDEBUG``\ enables verbose debugging output to STDERR.
699
To enable debugging and capture all output to a file, run the tool like:
704
PTDEBUG=1 pt-log-player ... > FILE 2>&1
707
Be careful: debugging output is voluminous and can generate several megabytes
716
You need Perl, DBI, DBD::mysql, and some core packages that ought to be
717
installed in any reasonably new version of Perl.
725
For a list of known bugs, see `http://www.percona.com/bugs/pt-log-player <http://www.percona.com/bugs/pt-log-player>`_.
727
Please report bugs at `https://bugs.launchpad.net/percona-toolkit <https://bugs.launchpad.net/percona-toolkit>`_.
728
Include the following information in your bug report:
731
\* Complete command-line used to run the tool
739
\* MySQL version of all servers involved
743
\* Output from the tool including STDERR
747
\* Input files (log/dump/config files, etc.)
751
If possible, include debugging output by running the tool with \ ``PTDEBUG``\ ;
763
*********************
764
ABOUT PERCONA TOOLKIT
765
*********************
768
This tool is part of Percona Toolkit, a collection of advanced command-line
769
tools developed by Percona for MySQL support and consulting. Percona Toolkit
770
was forked from two projects in June, 2011: Maatkit and Aspersa. Those
771
projects were created by Baron Schwartz and developed primarily by him and
772
Daniel Nichter, both of whom are employed by Percona. Visit
773
`http://www.percona.com/software/ <http://www.percona.com/software/>`_ for more software developed by Percona.
776
********************************
777
COPYRIGHT, LICENSE, AND WARRANTY
778
********************************
781
This program is copyright 2008-2011 Percona Inc.
782
Feedback and improvements are welcome.
784
THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED
785
WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
786
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
788
This program is free software; you can redistribute it and/or modify it under
789
the terms of the GNU General Public License as published by the Free Software
790
Foundation, version 2; OR the Perl Artistic License. On UNIX and similar
791
systems, you can issue \`man perlgpl' or \`man perlartistic' to read these
794
You should have received a copy of the GNU General Public License along with
795
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
796
Place, Suite 330, Boston, MA 02111-1307 USA.
804
Percona Toolkit v1.0.0 released 2011-08-01
added
removed
docs/user/pt-log-player.rst
