1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
|
.TH MAKEDUMPFILE 8 "April 2009" "makedumpfile v1.3.3" "Linux System Administrator's Manual"
.SH NAME
makedumpfile \- make a small dumpfile of kdump
.SH SYNOPSIS
\fBmakedumpfile\fR [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE\fR
.br
\fBmakedumpfile\fR \-F [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR
.br
\fBmakedumpfile\fR \-R \fIDUMPFILE\fR
.br
\fBmakedumpfile\fR \-\-split [\fIOPTION\fR] [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE1\fR \fIDUMPFILE2\fR [\fIDUMPFILE3\fR ..]
.br
\fBmakedumpfile\fR \-\-reassemble \fIDUMPFILE1\fR \fIDUMPFILE2\fR [\fIDUMPFILE3\fR ..] \fIDUMPFILE\fR
.br
\fBmakedumpfile\fR \-g \fIVMCOREINFO\fR \-x \fIVMLINUX\fR
.br
\fBmakedumpfile\fR \-E [\-\-xen-syms \fIXEN-SYMS\fR|\-\-xen-vmcoreinfo \fIVMCOREINFO\fR] \fIVMCORE\fR \fIDUMPFILE\fR
.br
\fBmakedumpfile\fR \-\-dump-dmesg [\-x \fIVMLINUX\fR|\-i \fIVMCOREINFO\fR] \fIVMCORE\fR \fILOGFILE\fR
.br
.B makedumpfile
\-h
.br
.B makedumpfile
\-v
.br
.SH DESCRIPTION
.PP
With kdump, the memory image of the first kernel (called "panicked kernel") can
be taken as /proc/vmcore while the second kernel (called "kdump kernel" or
"capture kernel") is running. This document represents /proc/vmcore as
\fIVMCORE\fR. makedumpfile makes a small \fIDUMPFILE\fR by compressing dump
data or by excluding unnecessary pages for analysis, or both. makedumpfile
needs the first kernel's \fIVMLINUX\fR with debug information, so that it can
distinguish unnecessary pages by analyzing how the first kernel uses the memory.
.PP
makedumpfile can exclude the following types of pages while copying
\fIVMCORE\fR to \fIDUMPFILE\fR, and a user can choose which type of pages will
be excluded.
.br
.B \- Pages filled with zero
.br
.B \- Cache pages
.br
.B \- User process data pages
.br
.B \- Free pages
.PP
makedumpfile provides two \fIDUMPFILE\fR formats (the ELF format and the
kdump\-compressed format). By default, makedumpfile makes a \fIDUMPFILE\fR in
the kdump\-compressed format. The kdump\-compressed format is readable only with
the crash utility, and it can be smaller than the ELF format because of the
compression support. The ELF format is readable with GDB and the crash utility.
If a user wants to use GDB, \fIDUMPFILE\fR format has to be explicitly
specified to be the ELF format.
.PP
To analyze the first kernel's memory usage, makedumpfile can refer to
\fIVMCOREINFO\fR instead of \fIVMLINUX\fR. \fIVMCOREINFO\fR contains the first
kernel's information (structure size, field offset, etc.), and \fIVMCOREINFO\fR
is small enough to be included into the second kernel's initrd.
.br
If the second kernel is running on its initrd without mounting a root file
system, makedumpfile cannot refer to \fIVMLINUX\fR because the second kernel's
initrd cannot include a large file like \fIVMLINUX\fR. To solve the problem,
makedumpfile makes \fIVMCOREINFO\fR beforehand, and it refers to
\fIVMCOREINFO\fR instead of \fIVMLINUX\fR while the second kernel is running.
.PP
If the second kernel is running on its initrd without mounting any file system,
a user needs to transport the dump data to a remote host. To transport the dump
data by SSH, makedumpfile outputs the dump data in the intermediate format (the
flattened format) to the standard output. By piping the output data to SSH,
a user can transport the dump data to a remote host. Note that analysis tools
cannot read the flattened format directly, so on a remote host the received
data in the flattened format needs to be rearranged to a readable
\fIDUMPFILE\fR format by makedumpfile (or makedumpfile\-R.pl).
.PP
.SH OPTIONS
.TP
\fB\-c\fR
Compress dump data by each page.
.br
A user cannot specify this option with \-E option, because the ELF format does
not support compressed data.
.br
.B Example:
.br
# makedumpfile \-c \-d 31 \-x vmlinux /proc/vmcore dumpfile
.TP
.BI \-d \ dump_level
Specify the type of unnecessary page for analysis.
.br
Pages of the specified type are not copied to \fIDUMPFILE\fR. The page type
marked in the following table is excluded. A user can specify multiple page
types by setting the sum of each page type for dump_level. The maximum of
dump_level is 31. Note that dump_level for Xen dump filtering is 0 or 1.
.br
If specifying multiple dump_levels with the delimiter ',', makedumpfile retries
to create a \fIDUMPFILE\fR by other dump_level when "No space on device" error
happens. For example, if dump_level is "11,31" and makedumpfile fails
by dump_level 11, makedumpfile retries it by dump_level 31.
.br
.B Example:
.br
# makedumpfile \-d 11 \-x vmlinux /proc/vmcore dumpfile
.br
# makedumpfile \-d 11,31 \-x vmlinux /proc/vmcore dumpfile
dump | zero | cache|cache | user | free
level | page | page |private| data | page
.br
\-\-\-\-\-\-\-+\-\-\-\-\-\-+\-\-\-\-\-\-+\-\-\-\-\-\-\-+\-\-\-\-\-\-+\-\-\-\-\-\-
0 | | | | |
1 | X | | | |
2 | | X | | |
3 | X | X | | |
4 | | X | X | |
5 | X | X | X | |
6 | | X | X | |
7 | X | X | X | |
8 | | | | X |
9 | X | | | X |
10 | | X | | X |
11 | X | X | | X |
12 | | X | X | X |
13 | X | X | X | X |
14 | | X | X | X |
15 | X | X | X | X |
16 | | | | | X
17 | X | | | | X
18 | | X | | | X
19 | X | X | | | X
20 | | X | X | | X
21 | X | X | X | | X
22 | | X | X | | X
23 | X | X | X | | X
24 | | | | X | X
25 | X | | | X | X
26 | | X | | X | X
27 | X | X | | X | X
28 | | X | X | X | X
29 | X | X | X | X | X
30 | | X | X | X | X
31 | X | X | X | X | X
.TP
\fB\-E\fR
Create \fIDUMPFILE\fR in the ELF format.
.br
This option cannot be specified with \-c option, because the ELF format does not
support compressed data.
.br
.B Example:
.br
# makedumpfile \-E \-d 31 \-x vmlinux /proc/vmcore dumpfile
.TP
\fB\-f\fR
Force existing DUMPFILE to be overwritten.
.br
.B Example:
.br
# makedumpfile \-f \-d 31 \-x vmlinux /proc/vmcore dumpfile
.br
This command overwrites \fIDUMPFILE\fR even if it already exists.
.TP
\fB\-x\fR \fIVMLINUX\fR
Specify the first kernel's \fIVMLINUX\fR with debug information to analyze the
first kernel's memory usage.
.br
The page size of the first kernel and the second kernel should match.
.br
.B Example:
.br
# makedumpfile \-d 31 \-x vmlinux /proc/vmcore dumpfile
.TP
\fB\-i\fR \fIVMCOREINFO\fR
Specify \fIVMCOREINFO\fR instead of \fIVMLINUX\fR for analyzing the first kernel's memory usage.
.br
\fIVMCOREINFO\fR should be made beforehand by makedumpfile with \-g option, and
it contains the first kernel's information. If dump_level is 2 or more and
[\-x \fIVMLINUX\fR] is not specified, this option is necessary.
.br
.B Example:
.br
# makedumpfile \-d 31 \-i vmcoreinfo /proc/vmcore dumpfile
.TP
\fB\-g\fR \fIVMCOREINFO\fR
Generate \fIVMCOREINFO\fR from the first kernel's \fIVMLINUX\fR with debug
information.
.br
\fIVMCOREINFO\fR must be generated on the system that is running the first
kernel. With \-i option, a user can specify \fIVMCOREINFO\fR generated on the
other system that is running the same first kernel. [\-x \fIVMLINUX\fR] must be
specified.
.br
.B Example:
.br
# makedumpfile \-g vmcoreinfo \-x vmlinux
.TP
\fB\-F\fR
Output the dump data in the flattened format to the standard output for
transporting the dump data by SSH.
.br
Analysis tools cannot read the flattened format directly. For analysis, the
dump data in the flattened format should be rearranged to a normal
\fIDUMPFILE\fR (readable with analysis tools) by \-R option. By which option is
specified with \-F option, the format of the rearranged \fIDUMPFILE\fR is fixed.
In other words, it is impossible to specify the \fIDUMPFILE\fR format when the
dump data is rearranged with \-R option. If specifying \-E option with \-F option,
the format of the rearranged \fIDUMPFILE\fR is the ELF format. Otherwise, it
is the kdump\-compressed format. All the messages are output to standard error
output by \-F option because standard output is used for the dump data.
.br
.B Example:
.br
# makedumpfile \-F \-c \-d 31 \-x vmlinux /proc/vmcore \e
.br
| ssh user@host "cat > dumpfile.tmp"
.br
# makedumpfile \-F \-c \-d 31 \-x vmlinux /proc/vmcore \e
.br
| ssh user@host "makedumpfile \-R dumpfile"
.br
# makedumpfile \-F \-E \-d 31 \-i vmcoreinfo /proc/vmcore \e
.br
| ssh user@host "makedumpfile \-R dumpfile"
.br
# makedumpfile \-F \-E \-\-xen-vmcoreinfo \fIVMCOREINFO\fR /proc/vmcore \e
.br
| ssh user@host "makedumpfile \-R dumpfile"
.TP
\fB\-R\fR
Rearrange the dump data in the flattened format from the standard input to a
normal \fIDUMPFILE\fR (readable with analysis tools).
.br
.B Example:
.br
# makedumpfile \-R dumpfile < dumpfile.tmp
.br
# makedumpfile \-F \-d 31 \-x vmlinux /proc/vmcore \e
.br
| ssh user@host "makedumpfile \-R dumpfile"
Instead of using \-R option, a perl script "makedumpfile\-R.pl" rearranges the
dump data in the flattened format to a normal \fIDUMPFILE\fR, too. The perl
script does not depend on architecture, and most systems have perl command.
Even if a remote host does not have makedumpfile, it is possible to rearrange
the dump data in the flattened format to a readable \fIDUMPFILE\fR on a remote
host by running this script.
.br
.B Example:
.br
# makedumpfile \-F \-d 31 \-x vmlinux /proc/vmcore \e
.br
| ssh user@host "makedumpfile\-R.pl dumpfile"
.TP
\fB\-\-split\fR
Split the dump data to multiple \fIDUMPFILE\fRs in parallel. If specifying
\fIDUMPFILE\fRs on different storage devices, a device can share I/O load
with other devices and it reduces time for saving the dump data. The file
size of each \fIDUMPFILE\fR is smaller than the system memory size which
is divided by the number of \fIDUMPFILE\fRs. This feature supports only
the kdump\-compressed format.
.br
.B Example:
.br
# makedumpfile \-\-split \-d 31 \-x vmlinux /proc/vmcore dumpfile1 dumpfile2
.TP
\fB\-\-reassemble\fR
Reassemble multiple \fIDUMPFILE\fRs, which are created by \-\-split option,
into one \fIDUMPFILE\fR. dumpfile1 and dumpfile2 are reassembled into dumpfile
on the following example.
.br
.B Example:
.br
# makedumpfile \-\-reassemble dumpfile1 dumpfile2 dumpfile
.TP
\fB\-\-xen-syms\fR \fIXEN-SYMS\fR
Specify the \fIXEN-SYMS\fR with debug information to analyze the xen's memory usage.
This option extracts the part of xen and domain-0.
\-E option must be specified with this option.
.br
.B Example:
.br
# makedumpfile \-E \-\-xen-syms xen-syms /proc/vmcore dumpfile
.TP
\fB\-\-xen-vmcoreinfo\fR \fIVMCOREINFO\fR
Specify \fIVMCOREINFO\fR instead of \fIXEN-SYMS\fR for analyzing the xen's memory usage.
.br
\fIVMCOREINFO\fR should be made beforehand by makedumpfile with \-g option, and
it contains the xen's information.
\-E option must be specified with this option.
.br
.B Example:
.br
# makedumpfile \-E \-\-xen-vmcoreinfo \fIVMCOREINFO\fR /proc/vmcore dumpfile
.TP
\fB\-X\fR
Exclude all the user domain pages from Xen kdump's \fIVMCORE\fR, and extracts the
part of xen and domain-0. If \fIVMCORE\fR contains \fIVMCOREINFO\fR for Xen, it is
not necessary to specify \fI\-\-xen-syms\fR and \fI\-\-xen-vmcoreinfo\fR.
\-E option must be specified with this option.
.br
.B Example:
.br
# makedumpfile \-E \-X /proc/vmcore dumpfile
.TP
\fB\-\-xen_phys_start\fR \fIxen_phys_start_address\fR
This option is only for x86_64.
Specify the \fIxen_phys_start_address\fR, if the xen code/data is relocatable
and \fIVMCORE\fR does not contain \fIxen_phys_start_address\fR in the CRASHINFO.
\fIxen_phys_start_address\fR can be token from the line of "Hypervisor code
and data" in /proc/iomem. For example, specify 0xcee00000 as \fIxen_phys_start_address\fR
if /proc/iomem is the following:
-------------------------------------------------------
# cat /proc/iomem
...
cee00000-cfd99999 : Hypervisor code and data
...
-------------------------------------------------------
.br
.B Example:
.br
# makedumpfile \-E \-X \-\-xen_phys_start 0xcee00000 /proc/vmcore dumpfile
.TP
\fB\-\-message-level\fR \fImessage_level\fR
Specify the message types.
.br
Users can restrict outputs printed by specifying \fImessage_level\fR
with this option. The message type marked with an X in the following
table is printed. For example, according to the table, specifying 7 as
\fImessage_level\fR means progress indicator, common message, and error
message are printed, and this is a default value. Note that the maximum
value of \fImessage_level\fR is 31.
.br
message | progress | common | error | debug | report
level | indicator| message | message | message | message
.br
\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-+\-\-\-\-\-\-\-\-\-
0 | | | | |
1 | X | | | |
2 | | X | | |
3 | X | X | | |
4 | | | X | |
5 | X | | X | |
6 | | X | X | |
* 7 | X | X | X | |
8 | | | | X |
9 | X | | | X |
10 | | X | | X |
11 | X | X | | X |
12 | | | X | X |
13 | X | | X | X |
14 | | X | X | X |
15 | X | X | X | X |
16 | | | | | X
17 | X | | | | X
18 | | X | | | X
19 | X | X | | | X
20 | | | X | | X
21 | X | | X | | X
22 | | X | X | | X
23 | X | X | X | | X
24 | | | | X | X
25 | X | | | X | X
26 | | X | | X | X
27 | X | X | | X | X
28 | | | X | X | X
29 | X | | X | X | X
30 | | X | X | X | X
31 | X | X | X | X | X
.TP
\fB\-\-vtop\fR \fIvirtual_address\fR
This option is useful, when user debugs the translation problem
of virtual address. If specifing \fIvirtual_address\fR, its physical
address is printed. It makes debugging easy by comparing the
output of this option with the one of "vtop" subcommand of the
crash utility.
"--vtop" option only prints the translation output, and it does
not affect the dumpfile creation.
.TP
\fB\-\-dump-dmesg\fR
This option overrides the normal behavior of makedumpfile. Instead of
compressing and filtering a \fIVMCORE\fR to make it smaller, it simply
extracts the dmesg log from a \fIVMCORE\fR and writes it to the specified
\fILOGFILE\fR. If a \fIVMCORE\fR does not contain \fIVMCOREINFO\fR for dmesg,
it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i \fIVMCOREINFO\fR].
.br
.B Example:
.br
# makedumpfile \-\-dump-dmesg /proc/vmcore dmesgfile
.br
# makedumpfile \-\-dump-dmesg -x vmlinux /proc/vmcore dmesgfile
.br
.TP
\fB\-D\fR
Print debugging message.
.TP
\fB\-h\fR
Show help message.
.TP
\fB\-v\fR
Show the version of makedumpfile.
.SH DIAGNOSTICS
makedumpfile exits with the following value.
.TP
\fB0\fR : makedumpfile succeeded.
.TP
\fB1\fR : makedumpfile failed without the following reasons.
.TP
\fB2\fR : makedumpfile failed due to the different version between \fIVMLINUX\fR and \fIVMCORE\fR.
.TP
\fB3\fR : makedumpfile failed due to the analysis error of the memory.
.SH AUTHORS
.PP
Written by Masaki Tachibana, and Ken'ichi Ohmichi.
.SH SEE ALSO
.PP
crash(8), gdb(1), kexec(8)
|