3
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
4
<title>bzip2 and libbzip2, version 1.0.6</title>
5
<meta name="generator" content="DocBook XSL Stylesheets V1.75.2">
6
<style type="text/css" media="screen">/* Colours:
7
#74240f dark brown h1, h2, h3, h4
8
#336699 medium blue links
9
#339999 turquoise link hover colour
10
#202020 almost black general text
11
#761596 purple md5sum text
12
#626262 dark gray pre border
13
#eeeeee very light gray pre background
14
#f2f2f9 very light blue nav table background
15
#3366cc medium blue nav table border
18
a, a:link, a:visited, a:active { color: #336699; }
19
a:hover { color: #339999; }
21
body { font: 80%/126% sans-serif; }
22
h1, h2, h3, h4 { color: #74240f; }
24
dt { color: #336699; font-weight: bold }
27
padding-bottom: 0.8em;
33
background:#ffffff url("/images/hr_blue.png") repeat-x; }
34
div.hr_blue hr { display:none; }
37
#release p { margin-top: 0.4em; }
38
#release .md5sum { color: #761596; }
41
/* ------ styles for docs|manuals|howto ------ */
44
margin: 0px 4px 16px 16px;
46
list-style: url("/images/li-blue.png");
52
list-style-type: none;
53
list-style-image: none;
57
/* header / footer nav tables */
59
border: solid 1px #3366cc;
61
background-color: #f2f2f9;
64
/* don't have underlined links in chunked nav menus */
65
table.nav a { text-decoration: none; }
66
table.nav a:hover { text-decoration: underline; }
67
table.nav td { font-size: 85%; }
69
code, tt, pre { font-size: 120%; }
70
code, tt { color: #761596; }
72
div.literallayout, pre.programlisting, pre.screen {
76
border: 1px solid #626262;
77
background-color: #eeeeee;
78
margin: 4px 0px 4px 0px;
82
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div lang="en" class="book" title="bzip2 and libbzip2, version 1.0.6">
83
<div class="titlepage">
85
<div><h1 class="title">
86
<a name="userman"></a>bzip2 and libbzip2, version 1.0.6</h1></div>
87
<div><h2 class="subtitle">A program and library for data compression</h2></div>
88
<div><div class="authorgroup"><div class="author">
90
<span class="firstname">Julian</span> <span class="surname">Seward</span>
92
<div class="affiliation"><span class="orgname">http://www.bzip.org<br></span></div>
94
<div><p class="releaseinfo">Version 1.0.6 of 6 September 2010</p></div>
95
<div><p class="copyright">Copyright � 1996-2010 Julian Seward</p></div>
96
<div><div class="legalnotice" title="Legal Notice">
97
<a name="id537185"></a><p>This program, <code class="computeroutput">bzip2</code>, the
98
associated library <code class="computeroutput">libbzip2</code>, and
99
all documentation, are copyright � 1996-2010 Julian Seward.
100
All rights reserved.</p>
101
<p>Redistribution and use in source and binary forms, with
102
or without modification, are permitted provided that the
103
following conditions are met:</p>
104
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
105
<li class="listitem" style="list-style-type: disc"><p>Redistributions of source code must retain the
106
above copyright notice, this list of conditions and the
107
following disclaimer.</p></li>
108
<li class="listitem" style="list-style-type: disc"><p>The origin of this software must not be
109
misrepresented; you must not claim that you wrote the original
110
software. If you use this software in a product, an
111
acknowledgment in the product documentation would be
112
appreciated but is not required.</p></li>
113
<li class="listitem" style="list-style-type: disc"><p>Altered source versions must be plainly marked
114
as such, and must not be misrepresented as being the original
116
<li class="listitem" style="list-style-type: disc"><p>The name of the author may not be used to
117
endorse or promote products derived from this software without
118
specific prior written permission.</p></li>
120
<p>THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY
121
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
122
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
123
PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
124
AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
125
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
126
TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
127
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
128
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
129
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
130
IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
131
THE POSSIBILITY OF SUCH DAMAGE.</p>
132
<p>PATENTS: To the best of my knowledge,
133
<code class="computeroutput">bzip2</code> and
134
<code class="computeroutput">libbzip2</code> do not use any patented
135
algorithms. However, I do not have the resources to carry
136
out a patent search. Therefore I cannot give any guarantee of
144
<p><b>Table of Contents</b></p>
146
<dt><span class="chapter"><a href="#intro">1. Introduction</a></span></dt>
147
<dt><span class="chapter"><a href="#using">2. How to use bzip2</a></span></dt>
149
<dt><span class="sect1"><a href="#name">2.1. NAME</a></span></dt>
150
<dt><span class="sect1"><a href="#synopsis">2.2. SYNOPSIS</a></span></dt>
151
<dt><span class="sect1"><a href="#description">2.3. DESCRIPTION</a></span></dt>
152
<dt><span class="sect1"><a href="#options">2.4. OPTIONS</a></span></dt>
153
<dt><span class="sect1"><a href="#memory-management">2.5. MEMORY MANAGEMENT</a></span></dt>
154
<dt><span class="sect1"><a href="#recovering">2.6. RECOVERING DATA FROM DAMAGED FILES</a></span></dt>
155
<dt><span class="sect1"><a href="#performance">2.7. PERFORMANCE NOTES</a></span></dt>
156
<dt><span class="sect1"><a href="#caveats">2.8. CAVEATS</a></span></dt>
157
<dt><span class="sect1"><a href="#author">2.9. AUTHOR</a></span></dt>
159
<dt><span class="chapter"><a href="#libprog">3.
160
Programming with <code class="computeroutput">libbzip2</code>
163
<dt><span class="sect1"><a href="#top-level">3.1. Top-level structure</a></span></dt>
165
<dt><span class="sect2"><a href="#ll-summary">3.1.1. Low-level summary</a></span></dt>
166
<dt><span class="sect2"><a href="#hl-summary">3.1.2. High-level summary</a></span></dt>
167
<dt><span class="sect2"><a href="#util-fns-summary">3.1.3. Utility functions summary</a></span></dt>
169
<dt><span class="sect1"><a href="#err-handling">3.2. Error handling</a></span></dt>
170
<dt><span class="sect1"><a href="#low-level">3.3. Low-level interface</a></span></dt>
172
<dt><span class="sect2"><a href="#bzcompress-init">3.3.1. BZ2_bzCompressInit</a></span></dt>
173
<dt><span class="sect2"><a href="#bzCompress">3.3.2. BZ2_bzCompress</a></span></dt>
174
<dt><span class="sect2"><a href="#bzCompress-end">3.3.3. BZ2_bzCompressEnd</a></span></dt>
175
<dt><span class="sect2"><a href="#bzDecompress-init">3.3.4. BZ2_bzDecompressInit</a></span></dt>
176
<dt><span class="sect2"><a href="#bzDecompress">3.3.5. BZ2_bzDecompress</a></span></dt>
177
<dt><span class="sect2"><a href="#bzDecompress-end">3.3.6. BZ2_bzDecompressEnd</a></span></dt>
179
<dt><span class="sect1"><a href="#hl-interface">3.4. High-level interface</a></span></dt>
181
<dt><span class="sect2"><a href="#bzreadopen">3.4.1. BZ2_bzReadOpen</a></span></dt>
182
<dt><span class="sect2"><a href="#bzread">3.4.2. BZ2_bzRead</a></span></dt>
183
<dt><span class="sect2"><a href="#bzreadgetunused">3.4.3. BZ2_bzReadGetUnused</a></span></dt>
184
<dt><span class="sect2"><a href="#bzreadclose">3.4.4. BZ2_bzReadClose</a></span></dt>
185
<dt><span class="sect2"><a href="#bzwriteopen">3.4.5. BZ2_bzWriteOpen</a></span></dt>
186
<dt><span class="sect2"><a href="#bzwrite">3.4.6. BZ2_bzWrite</a></span></dt>
187
<dt><span class="sect2"><a href="#bzwriteclose">3.4.7. BZ2_bzWriteClose</a></span></dt>
188
<dt><span class="sect2"><a href="#embed">3.4.8. Handling embedded compressed data streams</a></span></dt>
189
<dt><span class="sect2"><a href="#std-rdwr">3.4.9. Standard file-reading/writing code</a></span></dt>
191
<dt><span class="sect1"><a href="#util-fns">3.5. Utility functions</a></span></dt>
193
<dt><span class="sect2"><a href="#bzbufftobuffcompress">3.5.1. BZ2_bzBuffToBuffCompress</a></span></dt>
194
<dt><span class="sect2"><a href="#bzbufftobuffdecompress">3.5.2. BZ2_bzBuffToBuffDecompress</a></span></dt>
196
<dt><span class="sect1"><a href="#zlib-compat">3.6. zlib compatibility functions</a></span></dt>
197
<dt><span class="sect1"><a href="#stdio-free">3.7. Using the library in a stdio-free environment</a></span></dt>
199
<dt><span class="sect2"><a href="#stdio-bye">3.7.1. Getting rid of stdio</a></span></dt>
200
<dt><span class="sect2"><a href="#critical-error">3.7.2. Critical error handling</a></span></dt>
202
<dt><span class="sect1"><a href="#win-dll">3.8. Making a Windows DLL</a></span></dt>
204
<dt><span class="chapter"><a href="#misc">4. Miscellanea</a></span></dt>
206
<dt><span class="sect1"><a href="#limits">4.1. Limitations of the compressed file format</a></span></dt>
207
<dt><span class="sect1"><a href="#port-issues">4.2. Portability issues</a></span></dt>
208
<dt><span class="sect1"><a href="#bugs">4.3. Reporting bugs</a></span></dt>
209
<dt><span class="sect1"><a href="#package">4.4. Did you get the right package?</a></span></dt>
210
<dt><span class="sect1"><a href="#reading">4.5. Further Reading</a></span></dt>
214
<div class="chapter" title="1.�Introduction">
215
<div class="titlepage"><div><div><h2 class="title">
216
<a name="intro"></a>1.�Introduction</h2></div></div></div>
217
<p><code class="computeroutput">bzip2</code> compresses files
218
using the Burrows-Wheeler block-sorting text compression
219
algorithm, and Huffman coding. Compression is generally
220
considerably better than that achieved by more conventional
221
LZ77/LZ78-based compressors, and approaches the performance of
222
the PPM family of statistical compressors.</p>
223
<p><code class="computeroutput">bzip2</code> is built on top of
224
<code class="computeroutput">libbzip2</code>, a flexible library for
225
handling compressed data in the
226
<code class="computeroutput">bzip2</code> format. This manual
227
describes both how to use the program and how to work with the
228
library interface. Most of the manual is devoted to this
229
library, not the program, which is good news if your interest is
230
only in the program.</p>
231
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
232
<li class="listitem" style="list-style-type: disc"><p><a class="xref" href="#using" title="2.�How to use bzip2">How to use bzip2</a> describes how to use
233
<code class="computeroutput">bzip2</code>; this is the only part
234
you need to read if you just want to know how to operate the
236
<li class="listitem" style="list-style-type: disc"><p><a class="xref" href="#libprog" title="3.� Programming with libbzip2">Programming with libbzip2</a> describes the
237
programming interfaces in detail, and</p></li>
238
<li class="listitem" style="list-style-type: disc"><p><a class="xref" href="#misc" title="4.�Miscellanea">Miscellanea</a> records some
239
miscellaneous notes which I thought ought to be recorded
243
<div class="chapter" title="2.�How to use bzip2">
244
<div class="titlepage"><div><div><h2 class="title">
245
<a name="using"></a>2.�How to use bzip2</h2></div></div></div>
247
<p><b>Table of Contents</b></p>
249
<dt><span class="sect1"><a href="#name">2.1. NAME</a></span></dt>
250
<dt><span class="sect1"><a href="#synopsis">2.2. SYNOPSIS</a></span></dt>
251
<dt><span class="sect1"><a href="#description">2.3. DESCRIPTION</a></span></dt>
252
<dt><span class="sect1"><a href="#options">2.4. OPTIONS</a></span></dt>
253
<dt><span class="sect1"><a href="#memory-management">2.5. MEMORY MANAGEMENT</a></span></dt>
254
<dt><span class="sect1"><a href="#recovering">2.6. RECOVERING DATA FROM DAMAGED FILES</a></span></dt>
255
<dt><span class="sect1"><a href="#performance">2.7. PERFORMANCE NOTES</a></span></dt>
256
<dt><span class="sect1"><a href="#caveats">2.8. CAVEATS</a></span></dt>
257
<dt><span class="sect1"><a href="#author">2.9. AUTHOR</a></span></dt>
260
<p>This chapter contains a copy of the
261
<code class="computeroutput">bzip2</code> man page, and nothing
263
<div class="sect1" title="2.1.�NAME">
264
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
265
<a name="name"></a>2.1.�NAME</h2></div></div></div>
266
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
267
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2</code>,
268
<code class="computeroutput">bunzip2</code> - a block-sorting file
269
compressor, v1.0.6</p></li>
270
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzcat</code> -
271
decompresses files to stdout</p></li>
272
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2recover</code> -
273
recovers data from damaged bzip2 files</p></li>
276
<div class="sect1" title="2.2.�SYNOPSIS">
277
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
278
<a name="synopsis"></a>2.2.�SYNOPSIS</h2></div></div></div>
279
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
280
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2</code> [
281
-cdfkqstvzVL123456789 ] [ filenames ... ]</p></li>
282
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bunzip2</code> [
283
-fkvsVL ] [ filenames ... ]</p></li>
284
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzcat</code> [ -s ] [
285
filenames ... ]</p></li>
286
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2recover</code>
290
<div class="sect1" title="2.3.�DESCRIPTION">
291
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
292
<a name="description"></a>2.3.�DESCRIPTION</h2></div></div></div>
293
<p><code class="computeroutput">bzip2</code> compresses files
294
using the Burrows-Wheeler block sorting text compression
295
algorithm, and Huffman coding. Compression is generally
296
considerably better than that achieved by more conventional
297
LZ77/LZ78-based compressors, and approaches the performance of
298
the PPM family of statistical compressors.</p>
299
<p>The command-line options are deliberately very similar to
300
those of GNU <code class="computeroutput">gzip</code>, but they are
302
<p><code class="computeroutput">bzip2</code> expects a list of
303
file names to accompany the command-line flags. Each file is
304
replaced by a compressed version of itself, with the name
305
<code class="computeroutput">original_name.bz2</code>. Each
306
compressed file has the same modification date, permissions, and,
307
when possible, ownership as the corresponding original, so that
308
these properties can be correctly restored at decompression time.
309
File name handling is naive in the sense that there is no
310
mechanism for preserving original file names, permissions,
311
ownerships or dates in filesystems which lack these concepts, or
312
have serious file name length restrictions, such as
314
<p><code class="computeroutput">bzip2</code> and
315
<code class="computeroutput">bunzip2</code> will by default not
316
overwrite existing files. If you want this to happen, specify
317
the <code class="computeroutput">-f</code> flag.</p>
318
<p>If no file names are specified,
319
<code class="computeroutput">bzip2</code> compresses from standard
320
input to standard output. In this case,
321
<code class="computeroutput">bzip2</code> will decline to write
322
compressed output to a terminal, as this would be entirely
323
incomprehensible and therefore pointless.</p>
324
<p><code class="computeroutput">bunzip2</code> (or
325
<code class="computeroutput">bzip2 -d</code>) decompresses all
326
specified files. Files which were not created by
327
<code class="computeroutput">bzip2</code> will be detected and
328
ignored, and a warning issued.
329
<code class="computeroutput">bzip2</code> attempts to guess the
330
filename for the decompressed file from that of the compressed
332
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
333
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.bz2 </code>
335
<code class="computeroutput">filename</code></p></li>
336
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.bz </code>
338
<code class="computeroutput">filename</code></p></li>
339
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.tbz2</code>
341
<code class="computeroutput">filename.tar</code></p></li>
342
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">filename.tbz </code>
344
<code class="computeroutput">filename.tar</code></p></li>
345
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">anyothername </code>
347
<code class="computeroutput">anyothername.out</code></p></li>
349
<p>If the file does not end in one of the recognised endings,
350
<code class="computeroutput">.bz2</code>,
351
<code class="computeroutput">.bz</code>,
352
<code class="computeroutput">.tbz2</code> or
353
<code class="computeroutput">.tbz</code>,
354
<code class="computeroutput">bzip2</code> complains that it cannot
355
guess the name of the original file, and uses the original name
356
with <code class="computeroutput">.out</code> appended.</p>
357
<p>As with compression, supplying no filenames causes
358
decompression from standard input to standard output.</p>
359
<p><code class="computeroutput">bunzip2</code> will correctly
360
decompress a file which is the concatenation of two or more
361
compressed files. The result is the concatenation of the
362
corresponding uncompressed files. Integrity testing
363
(<code class="computeroutput">-t</code>) of concatenated compressed
364
files is also supported.</p>
365
<p>You can also compress or decompress files to the standard
366
output by giving the <code class="computeroutput">-c</code> flag.
367
Multiple files may be compressed and decompressed like this. The
368
resulting outputs are fed sequentially to stdout. Compression of
369
multiple files in this manner generates a stream containing
370
multiple compressed file representations. Such a stream can be
371
decompressed correctly only by
372
<code class="computeroutput">bzip2</code> version 0.9.0 or later.
373
Earlier versions of <code class="computeroutput">bzip2</code> will
374
stop after decompressing the first file in the stream.</p>
375
<p><code class="computeroutput">bzcat</code> (or
376
<code class="computeroutput">bzip2 -dc</code>) decompresses all
377
specified files to the standard output.</p>
378
<p><code class="computeroutput">bzip2</code> will read arguments
379
from the environment variables
380
<code class="computeroutput">BZIP2</code> and
381
<code class="computeroutput">BZIP</code>, in that order, and will
382
process them before any arguments read from the command line.
383
This gives a convenient way to supply default arguments.</p>
384
<p>Compression is always performed, even if the compressed
385
file is slightly larger than the original. Files of less than
386
about one hundred bytes tend to get larger, since the compression
387
mechanism has a constant overhead in the region of 50 bytes.
388
Random data (including the output of most file compressors) is
389
coded at about 8.05 bits per byte, giving an expansion of around
391
<p>As a self-check for your protection,
392
<code class="computeroutput">bzip2</code> uses 32-bit CRCs to make
393
sure that the decompressed version of a file is identical to the
394
original. This guards against corruption of the compressed data,
395
and against undetected bugs in
396
<code class="computeroutput">bzip2</code> (hopefully very unlikely).
397
The chances of data corruption going undetected is microscopic,
398
about one chance in four billion for each file processed. Be
399
aware, though, that the check occurs upon decompression, so it
400
can only tell you that something is wrong. It can't help you
401
recover the original uncompressed data. You can use
402
<code class="computeroutput">bzip2recover</code> to try to recover
403
data from damaged files.</p>
404
<p>Return values: 0 for a normal exit, 1 for environmental
405
problems (file not found, invalid flags, I/O errors, etc.), 2
406
to indicate a corrupt compressed file, 3 for an internal
407
consistency error (eg, bug) which caused
408
<code class="computeroutput">bzip2</code> to panic.</p>
410
<div class="sect1" title="2.4.�OPTIONS">
411
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
412
<a name="options"></a>2.4.�OPTIONS</h2></div></div></div>
413
<div class="variablelist"><dl>
414
<dt><span class="term"><code class="computeroutput">-c --stdout</code></span></dt>
415
<dd><p>Compress or decompress to standard
417
<dt><span class="term"><code class="computeroutput">-d --decompress</code></span></dt>
418
<dd><p>Force decompression.
419
<code class="computeroutput">bzip2</code>,
420
<code class="computeroutput">bunzip2</code> and
421
<code class="computeroutput">bzcat</code> are really the same
422
program, and the decision about what actions to take is done on
423
the basis of which name is used. This flag overrides that
424
mechanism, and forces bzip2 to decompress.</p></dd>
425
<dt><span class="term"><code class="computeroutput">-z --compress</code></span></dt>
426
<dd><p>The complement to
427
<code class="computeroutput">-d</code>: forces compression,
428
regardless of the invokation name.</p></dd>
429
<dt><span class="term"><code class="computeroutput">-t --test</code></span></dt>
430
<dd><p>Check integrity of the specified file(s), but
431
don't decompress them. This really performs a trial
432
decompression and throws away the result.</p></dd>
433
<dt><span class="term"><code class="computeroutput">-f --force</code></span></dt>
435
<p>Force overwrite of output files. Normally,
436
<code class="computeroutput">bzip2</code> will not overwrite
437
existing output files. Also forces
438
<code class="computeroutput">bzip2</code> to break hard links to
439
files, which it otherwise wouldn't do.</p>
440
<p><code class="computeroutput">bzip2</code> normally declines
441
to decompress files which don't have the correct magic header
442
bytes. If forced (<code class="computeroutput">-f</code>),
443
however, it will pass such files through unmodified. This is
444
how GNU <code class="computeroutput">gzip</code> behaves.</p>
446
<dt><span class="term"><code class="computeroutput">-k --keep</code></span></dt>
447
<dd><p>Keep (don't delete) input files during
448
compression or decompression.</p></dd>
449
<dt><span class="term"><code class="computeroutput">-s --small</code></span></dt>
451
<p>Reduce memory usage, for compression,
452
decompression and testing. Files are decompressed and tested
453
using a modified algorithm which only requires 2.5 bytes per
454
block byte. This means any file can be decompressed in 2300k
455
of memory, albeit at about half the normal speed.</p>
456
<p>During compression, <code class="computeroutput">-s</code>
457
selects a block size of 200k, which limits memory use to around
458
the same figure, at the expense of your compression ratio. In
459
short, if your machine is low on memory (8 megabytes or less),
460
use <code class="computeroutput">-s</code> for everything. See
461
<a class="xref" href="#memory-management" title="2.5.�MEMORY MANAGEMENT">MEMORY MANAGEMENT</a> below.</p>
463
<dt><span class="term"><code class="computeroutput">-q --quiet</code></span></dt>
464
<dd><p>Suppress non-essential warning messages.
465
Messages pertaining to I/O errors and other critical events
466
will not be suppressed.</p></dd>
467
<dt><span class="term"><code class="computeroutput">-v --verbose</code></span></dt>
468
<dd><p>Verbose mode -- show the compression ratio for
469
each file processed. Further
470
<code class="computeroutput">-v</code>'s increase the verbosity
471
level, spewing out lots of information which is primarily of
472
interest for diagnostic purposes.</p></dd>
473
<dt><span class="term"><code class="computeroutput">-L --license -V --version</code></span></dt>
474
<dd><p>Display the software version, license terms and
476
<dt><span class="term"><code class="computeroutput">-1</code> (or
477
<code class="computeroutput">--fast</code>) to
478
<code class="computeroutput">-9</code> (or
479
<code class="computeroutput">-best</code>)</span></dt>
480
<dd><p>Set the block size to 100 k, 200 k ... 900 k
481
when compressing. Has no effect when decompressing. See <a class="xref" href="#memory-management" title="2.5.�MEMORY MANAGEMENT">MEMORY MANAGEMENT</a> below. The
482
<code class="computeroutput">--fast</code> and
483
<code class="computeroutput">--best</code> aliases are primarily
484
for GNU <code class="computeroutput">gzip</code> compatibility.
485
In particular, <code class="computeroutput">--fast</code> doesn't
486
make things significantly faster. And
487
<code class="computeroutput">--best</code> merely selects the
488
default behaviour.</p></dd>
489
<dt><span class="term"><code class="computeroutput">--</code></span></dt>
490
<dd><p>Treats all subsequent arguments as file names,
491
even if they start with a dash. This is so you can handle
492
files with names beginning with a dash, for example:
493
<code class="computeroutput">bzip2 --
494
-myfilename</code>.</p></dd>
496
<span class="term"><code class="computeroutput">--repetitive-fast</code>, </span><span class="term"><code class="computeroutput">--repetitive-best</code></span>
498
<dd><p>These flags are redundant in versions 0.9.5 and
499
above. They provided some coarse control over the behaviour of
500
the sorting algorithm in earlier versions, which was sometimes
501
useful. 0.9.5 and above have an improved algorithm which
502
renders these flags irrelevant.</p></dd>
505
<div class="sect1" title="2.5.�MEMORY MANAGEMENT">
506
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
507
<a name="memory-management"></a>2.5.�MEMORY MANAGEMENT</h2></div></div></div>
508
<p><code class="computeroutput">bzip2</code> compresses large
509
files in blocks. The block size affects both the compression
510
ratio achieved, and the amount of memory needed for compression
511
and decompression. The flags <code class="computeroutput">-1</code>
512
through <code class="computeroutput">-9</code> specify the block
513
size to be 100,000 bytes through 900,000 bytes (the default)
514
respectively. At decompression time, the block size used for
515
compression is read from the header of the compressed file, and
516
<code class="computeroutput">bunzip2</code> then allocates itself
517
just enough memory to decompress the file. Since block sizes are
518
stored in compressed files, it follows that the flags
519
<code class="computeroutput">-1</code> to
520
<code class="computeroutput">-9</code> are irrelevant to and so
521
ignored during decompression.</p>
522
<p>Compression and decompression requirements, in bytes, can be
524
<pre class="programlisting">Compression: 400k + ( 8 x block size )
526
Decompression: 100k + ( 4 x block size ), or
527
100k + ( 2.5 x block size )</pre>
528
<p>Larger block sizes give rapidly diminishing marginal
529
returns. Most of the compression comes from the first two or
530
three hundred k of block size, a fact worth bearing in mind when
531
using <code class="computeroutput">bzip2</code> on small machines.
532
It is also important to appreciate that the decompression memory
533
requirement is set at compression time by the choice of block
535
<p>For files compressed with the default 900k block size,
536
<code class="computeroutput">bunzip2</code> will require about 3700
537
kbytes to decompress. To support decompression of any file on a
538
4 megabyte machine, <code class="computeroutput">bunzip2</code> has
539
an option to decompress using approximately half this amount of
540
memory, about 2300 kbytes. Decompression speed is also halved,
541
so you should use this option only where necessary. The relevant
542
flag is <code class="computeroutput">-s</code>.</p>
543
<p>In general, try and use the largest block size memory
544
constraints allow, since that maximises the compression achieved.
545
Compression and decompression speed are virtually unaffected by
547
<p>Another significant point applies to files which fit in a
548
single block -- that means most files you'd encounter using a
549
large block size. The amount of real memory touched is
550
proportional to the size of the file, since the file is smaller
551
than a block. For example, compressing a file 20,000 bytes long
552
with the flag <code class="computeroutput">-9</code> will cause the
553
compressor to allocate around 7600k of memory, but only touch
554
400k + 20000 * 8 = 560 kbytes of it. Similarly, the decompressor
555
will allocate 3700k but only touch 100k + 20000 * 4 = 180
557
<p>Here is a table which summarises the maximum memory usage
558
for different block sizes. Also recorded is the total compressed
559
size for 14 files of the Calgary Text Compression Corpus
560
totalling 3,141,622 bytes. This column gives some feel for how
561
compression varies with block size. These figures tend to
562
understate the advantage of larger block sizes for larger files,
563
since the Corpus is dominated by smaller files.</p>
564
<pre class="programlisting"> Compress Decompress Decompress Corpus
565
Flag usage usage -s usage Size
567
-1 1200k 500k 350k 914704
568
-2 2000k 900k 600k 877703
569
-3 2800k 1300k 850k 860338
570
-4 3600k 1700k 1100k 846899
571
-5 4400k 2100k 1350k 845160
572
-6 5200k 2500k 1600k 838626
573
-7 6100k 2900k 1850k 834096
574
-8 6800k 3300k 2100k 828642
575
-9 7600k 3700k 2350k 828642</pre>
577
<div class="sect1" title="2.6.�RECOVERING DATA FROM DAMAGED FILES">
578
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
579
<a name="recovering"></a>2.6.�RECOVERING DATA FROM DAMAGED FILES</h2></div></div></div>
580
<p><code class="computeroutput">bzip2</code> compresses files in
581
blocks, usually 900kbytes long. Each block is handled
582
independently. If a media or transmission error causes a
583
multi-block <code class="computeroutput">.bz2</code> file to become
584
damaged, it may be possible to recover data from the undamaged
585
blocks in the file.</p>
586
<p>The compressed representation of each block is delimited by
587
a 48-bit pattern, which makes it possible to find the block
588
boundaries with reasonable certainty. Each block also carries
589
its own 32-bit CRC, so damaged blocks can be distinguished from
591
<p><code class="computeroutput">bzip2recover</code> is a simple
592
program whose purpose is to search for blocks in
593
<code class="computeroutput">.bz2</code> files, and write each block
594
out into its own <code class="computeroutput">.bz2</code> file. You
595
can then use <code class="computeroutput">bzip2 -t</code> to test
596
the integrity of the resulting files, and decompress those which
598
<p><code class="computeroutput">bzip2recover</code> takes a
599
single argument, the name of the damaged file, and writes a
600
number of files <code class="computeroutput">rec0001file.bz2</code>,
601
<code class="computeroutput">rec0002file.bz2</code>, etc, containing
602
the extracted blocks. The output filenames are designed so that
603
the use of wildcards in subsequent processing -- for example,
604
<code class="computeroutput">bzip2 -dc rec*file.bz2 >
605
recovered_data</code> -- lists the files in the correct
607
<p><code class="computeroutput">bzip2recover</code> should be of
608
most use dealing with large <code class="computeroutput">.bz2</code>
609
files, as these will contain many blocks. It is clearly futile
610
to use it on damaged single-block files, since a damaged block
611
cannot be recovered. If you wish to minimise any potential data
612
loss through media or transmission errors, you might consider
613
compressing with a smaller block size.</p>
615
<div class="sect1" title="2.7.�PERFORMANCE NOTES">
616
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
617
<a name="performance"></a>2.7.�PERFORMANCE NOTES</h2></div></div></div>
618
<p>The sorting phase of compression gathers together similar
619
strings in the file. Because of this, files containing very long
620
runs of repeated symbols, like "aabaabaabaab ..." (repeated
621
several hundred times) may compress more slowly than normal.
622
Versions 0.9.5 and above fare much better than previous versions
623
in this respect. The ratio between worst-case and average-case
624
compression time is in the region of 10:1. For previous
625
versions, this figure was more like 100:1. You can use the
626
<code class="computeroutput">-vvvv</code> option to monitor progress
627
in great detail, if you want.</p>
628
<p>Decompression speed is unaffected by these
630
<p><code class="computeroutput">bzip2</code> usually allocates
631
several megabytes of memory to operate in, and then charges all
632
over it in a fairly random fashion. This means that performance,
633
both for compressing and decompressing, is largely determined by
634
the speed at which your machine can service cache misses.
635
Because of this, small changes to the code to reduce the miss
636
rate have been observed to give disproportionately large
637
performance improvements. I imagine
638
<code class="computeroutput">bzip2</code> will perform best on
639
machines with very large caches.</p>
641
<div class="sect1" title="2.8.�CAVEATS">
642
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
643
<a name="caveats"></a>2.8.�CAVEATS</h2></div></div></div>
644
<p>I/O error messages are not as helpful as they could be.
645
<code class="computeroutput">bzip2</code> tries hard to detect I/O
646
errors and exit cleanly, but the details of what the problem is
647
sometimes seem rather misleading.</p>
648
<p>This manual page pertains to version 1.0.6 of
649
<code class="computeroutput">bzip2</code>. Compressed data created by
650
this version is entirely forwards and backwards compatible with the
651
previous public releases, versions 0.1pl2, 0.9.0 and 0.9.5, 1.0.0,
652
1.0.1, 1.0.2 and 1.0.3, but with the following exception: 0.9.0 and
653
above can correctly decompress multiple concatenated compressed files.
654
0.1pl2 cannot do this; it will stop after decompressing just the first
655
file in the stream.</p>
656
<p><code class="computeroutput">bzip2recover</code> versions
657
prior to 1.0.2 used 32-bit integers to represent bit positions in
658
compressed files, so it could not handle compressed files more
659
than 512 megabytes long. Versions 1.0.2 and above use 64-bit ints
660
on some platforms which support them (GNU supported targets, and
661
Windows). To establish whether or not
662
<code class="computeroutput">bzip2recover</code> was built with such
663
a limitation, run it without arguments. In any event you can
664
build yourself an unlimited version if you can recompile it with
665
<code class="computeroutput">MaybeUInt64</code> set to be an
666
unsigned 64-bit integer.</p>
668
<div class="sect1" title="2.9.�AUTHOR">
669
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
670
<a name="author"></a>2.9.�AUTHOR</h2></div></div></div>
672
<code class="computeroutput">jseward@bzip.org</code></p>
673
<p>The ideas embodied in
674
<code class="computeroutput">bzip2</code> are due to (at least) the
675
following people: Michael Burrows and David Wheeler (for the
676
block sorting transformation), David Wheeler (again, for the
677
Huffman coder), Peter Fenwick (for the structured coding model in
678
the original <code class="computeroutput">bzip</code>, and many
679
refinements), and Alistair Moffat, Radford Neal and Ian Witten
680
(for the arithmetic coder in the original
681
<code class="computeroutput">bzip</code>). I am much indebted for
682
their help, support and advice. See the manual in the source
683
distribution for pointers to sources of documentation. Christian
684
von Roques encouraged me to look for faster sorting algorithms,
685
so as to speed up compression. Bela Lubkin encouraged me to
686
improve the worst-case compression performance.
687
Donna Robinson XMLised the documentation.
689
patches, helped with portability problems, lent machines, gave
690
advice and were generally helpful.</p>
693
<div class="chapter" title="3.� Programming with libbzip2">
694
<div class="titlepage"><div><div><h2 class="title">
695
<a name="libprog"></a>3.�
696
Programming with <code class="computeroutput">libbzip2</code>
697
</h2></div></div></div>
699
<p><b>Table of Contents</b></p>
701
<dt><span class="sect1"><a href="#top-level">3.1. Top-level structure</a></span></dt>
703
<dt><span class="sect2"><a href="#ll-summary">3.1.1. Low-level summary</a></span></dt>
704
<dt><span class="sect2"><a href="#hl-summary">3.1.2. High-level summary</a></span></dt>
705
<dt><span class="sect2"><a href="#util-fns-summary">3.1.3. Utility functions summary</a></span></dt>
707
<dt><span class="sect1"><a href="#err-handling">3.2. Error handling</a></span></dt>
708
<dt><span class="sect1"><a href="#low-level">3.3. Low-level interface</a></span></dt>
710
<dt><span class="sect2"><a href="#bzcompress-init">3.3.1. BZ2_bzCompressInit</a></span></dt>
711
<dt><span class="sect2"><a href="#bzCompress">3.3.2. BZ2_bzCompress</a></span></dt>
712
<dt><span class="sect2"><a href="#bzCompress-end">3.3.3. BZ2_bzCompressEnd</a></span></dt>
713
<dt><span class="sect2"><a href="#bzDecompress-init">3.3.4. BZ2_bzDecompressInit</a></span></dt>
714
<dt><span class="sect2"><a href="#bzDecompress">3.3.5. BZ2_bzDecompress</a></span></dt>
715
<dt><span class="sect2"><a href="#bzDecompress-end">3.3.6. BZ2_bzDecompressEnd</a></span></dt>
717
<dt><span class="sect1"><a href="#hl-interface">3.4. High-level interface</a></span></dt>
719
<dt><span class="sect2"><a href="#bzreadopen">3.4.1. BZ2_bzReadOpen</a></span></dt>
720
<dt><span class="sect2"><a href="#bzread">3.4.2. BZ2_bzRead</a></span></dt>
721
<dt><span class="sect2"><a href="#bzreadgetunused">3.4.3. BZ2_bzReadGetUnused</a></span></dt>
722
<dt><span class="sect2"><a href="#bzreadclose">3.4.4. BZ2_bzReadClose</a></span></dt>
723
<dt><span class="sect2"><a href="#bzwriteopen">3.4.5. BZ2_bzWriteOpen</a></span></dt>
724
<dt><span class="sect2"><a href="#bzwrite">3.4.6. BZ2_bzWrite</a></span></dt>
725
<dt><span class="sect2"><a href="#bzwriteclose">3.4.7. BZ2_bzWriteClose</a></span></dt>
726
<dt><span class="sect2"><a href="#embed">3.4.8. Handling embedded compressed data streams</a></span></dt>
727
<dt><span class="sect2"><a href="#std-rdwr">3.4.9. Standard file-reading/writing code</a></span></dt>
729
<dt><span class="sect1"><a href="#util-fns">3.5. Utility functions</a></span></dt>
731
<dt><span class="sect2"><a href="#bzbufftobuffcompress">3.5.1. BZ2_bzBuffToBuffCompress</a></span></dt>
732
<dt><span class="sect2"><a href="#bzbufftobuffdecompress">3.5.2. BZ2_bzBuffToBuffDecompress</a></span></dt>
734
<dt><span class="sect1"><a href="#zlib-compat">3.6. zlib compatibility functions</a></span></dt>
735
<dt><span class="sect1"><a href="#stdio-free">3.7. Using the library in a stdio-free environment</a></span></dt>
737
<dt><span class="sect2"><a href="#stdio-bye">3.7.1. Getting rid of stdio</a></span></dt>
738
<dt><span class="sect2"><a href="#critical-error">3.7.2. Critical error handling</a></span></dt>
740
<dt><span class="sect1"><a href="#win-dll">3.8. Making a Windows DLL</a></span></dt>
743
<p>This chapter describes the programming interface to
744
<code class="computeroutput">libbzip2</code>.</p>
745
<p>For general background information, particularly about
746
memory use and performance aspects, you'd be well advised to read
747
<a class="xref" href="#using" title="2.�How to use bzip2">How to use bzip2</a> as well.</p>
748
<div class="sect1" title="3.1.�Top-level structure">
749
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
750
<a name="top-level"></a>3.1.�Top-level structure</h2></div></div></div>
751
<p><code class="computeroutput">libbzip2</code> is a flexible
752
library for compressing and decompressing data in the
753
<code class="computeroutput">bzip2</code> data format. Although
754
packaged as a single entity, it helps to regard the library as
755
three separate parts: the low level interface, and the high level
756
interface, and some utility functions.</p>
758
<code class="computeroutput">libbzip2</code>'s interfaces is similar
759
to that of Jean-loup Gailly's and Mark Adler's excellent
760
<code class="computeroutput">zlib</code> library.</p>
761
<p>All externally visible symbols have names beginning
762
<code class="computeroutput">BZ2_</code>. This is new in version
763
1.0. The intention is to minimise pollution of the namespaces of
765
<p>To use any part of the library, you need to
766
<code class="computeroutput">#include <bzlib.h></code>
767
into your sources.</p>
768
<div class="sect2" title="3.1.1.�Low-level summary">
769
<div class="titlepage"><div><div><h3 class="title">
770
<a name="ll-summary"></a>3.1.1.�Low-level summary</h3></div></div></div>
771
<p>This interface provides services for compressing and
772
decompressing data in memory. There's no provision for dealing
773
with files, streams or any other I/O mechanisms, just straight
774
memory-to-memory work. In fact, this part of the library can be
775
compiled without inclusion of
776
<code class="computeroutput">stdio.h</code>, which may be helpful
777
for embedded applications.</p>
778
<p>The low-level part of the library has no global variables
779
and is therefore thread-safe.</p>
780
<p>Six routines make up the low level interface:
781
<code class="computeroutput">BZ2_bzCompressInit</code>,
782
<code class="computeroutput">BZ2_bzCompress</code>, and
783
<code class="computeroutput">BZ2_bzCompressEnd</code> for
784
compression, and a corresponding trio
785
<code class="computeroutput">BZ2_bzDecompressInit</code>,
786
<code class="computeroutput">BZ2_bzDecompress</code> and
787
<code class="computeroutput">BZ2_bzDecompressEnd</code> for
788
decompression. The <code class="computeroutput">*Init</code>
789
functions allocate memory for compression/decompression and do
790
other initialisations, whilst the
791
<code class="computeroutput">*End</code> functions close down
792
operations and release memory.</p>
793
<p>The real work is done by
794
<code class="computeroutput">BZ2_bzCompress</code> and
795
<code class="computeroutput">BZ2_bzDecompress</code>. These
796
compress and decompress data from a user-supplied input buffer to
797
a user-supplied output buffer. These buffers can be any size;
798
arbitrary quantities of data are handled by making repeated calls
799
to these functions. This is a flexible mechanism allowing a
800
consumer-pull style of activity, or producer-push, or a mixture
803
<div class="sect2" title="3.1.2.�High-level summary">
804
<div class="titlepage"><div><div><h3 class="title">
805
<a name="hl-summary"></a>3.1.2.�High-level summary</h3></div></div></div>
806
<p>This interface provides some handy wrappers around the
807
low-level interface to facilitate reading and writing
808
<code class="computeroutput">bzip2</code> format files
809
(<code class="computeroutput">.bz2</code> files). The routines
810
provide hooks to facilitate reading files in which the
811
<code class="computeroutput">bzip2</code> data stream is embedded
812
within some larger-scale file structure, or where there are
813
multiple <code class="computeroutput">bzip2</code> data streams
814
concatenated end-to-end.</p>
815
<p>For reading files,
816
<code class="computeroutput">BZ2_bzReadOpen</code>,
817
<code class="computeroutput">BZ2_bzRead</code>,
818
<code class="computeroutput">BZ2_bzReadClose</code> and
819
<code class="computeroutput">BZ2_bzReadGetUnused</code> are
820
supplied. For writing files,
821
<code class="computeroutput">BZ2_bzWriteOpen</code>,
822
<code class="computeroutput">BZ2_bzWrite</code> and
823
<code class="computeroutput">BZ2_bzWriteFinish</code> are
825
<p>As with the low-level library, no global variables are used
826
so the library is per se thread-safe. However, if I/O errors
827
occur whilst reading or writing the underlying compressed files,
828
you may have to consult <code class="computeroutput">errno</code> to
829
determine the cause of the error. In that case, you'd need a C
830
library which correctly supports
831
<code class="computeroutput">errno</code> in a multithreaded
833
<p>To make the library a little simpler and more portable,
834
<code class="computeroutput">BZ2_bzReadOpen</code> and
835
<code class="computeroutput">BZ2_bzWriteOpen</code> require you to
836
pass them file handles (<code class="computeroutput">FILE*</code>s)
837
which have previously been opened for reading or writing
838
respectively. That avoids portability problems associated with
839
file operations and file attributes, whilst not being much of an
840
imposition on the programmer.</p>
842
<div class="sect2" title="3.1.3.�Utility functions summary">
843
<div class="titlepage"><div><div><h3 class="title">
844
<a name="util-fns-summary"></a>3.1.3.�Utility functions summary</h3></div></div></div>
845
<p>For very simple needs,
846
<code class="computeroutput">BZ2_bzBuffToBuffCompress</code> and
847
<code class="computeroutput">BZ2_bzBuffToBuffDecompress</code> are
848
provided. These compress data in memory from one buffer to
849
another buffer in a single function call. You should assess
850
whether these functions fulfill your memory-to-memory
851
compression/decompression requirements before investing effort in
852
understanding the more general but more complex low-level
855
(<code class="computeroutput">tsuneo@rr.iij4u.or.jp</code>) has
856
contributed some functions to give better
857
<code class="computeroutput">zlib</code> compatibility. These
858
functions are <code class="computeroutput">BZ2_bzopen</code>,
859
<code class="computeroutput">BZ2_bzread</code>,
860
<code class="computeroutput">BZ2_bzwrite</code>,
861
<code class="computeroutput">BZ2_bzflush</code>,
862
<code class="computeroutput">BZ2_bzclose</code>,
863
<code class="computeroutput">BZ2_bzerror</code> and
864
<code class="computeroutput">BZ2_bzlibVersion</code>. You may find
865
these functions more convenient for simple file reading and
866
writing, than those in the high-level interface. These functions
867
are not (yet) officially part of the library, and are minimally
868
documented here. If they break, you get to keep all the pieces.
869
I hope to document them properly when time permits.</p>
870
<p>Yoshioka also contributed modifications to allow the
871
library to be built as a Windows DLL.</p>
874
<div class="sect1" title="3.2.�Error handling">
875
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
876
<a name="err-handling"></a>3.2.�Error handling</h2></div></div></div>
877
<p>The library is designed to recover cleanly in all
878
situations, including the worst-case situation of decompressing
879
random data. I'm not 100% sure that it can always do this, so
880
you might want to add a signal handler to catch segmentation
881
violations during decompression if you are feeling especially
882
paranoid. I would be interested in hearing more about the
883
robustness of the library to corrupted compressed data.</p>
884
<p>Version 1.0.3 more robust in this respect than any
885
previous version. Investigations with Valgrind (a tool for detecting
886
problems with memory management) indicate
887
that, at least for the few files I tested, all single-bit errors
888
in the decompressed data are caught properly, with no
889
segmentation faults, no uses of uninitialised data, no out of
890
range reads or writes, and no infinite looping in the decompressor.
891
So it's certainly pretty robust, although
892
I wouldn't claim it to be totally bombproof.</p>
893
<p>The file <code class="computeroutput">bzlib.h</code> contains
894
all definitions needed to use the library. In particular, you
895
should definitely not include
896
<code class="computeroutput">bzlib_private.h</code>.</p>
897
<p>In <code class="computeroutput">bzlib.h</code>, the various
898
return values are defined. The following list is not intended as
899
an exhaustive description of the circumstances in which a given
900
value may be returned -- those descriptions are given later.
901
Rather, it is intended to convey the rough meaning of each return
902
value. The first five actions are normal and not intended to
903
denote an error situation.</p>
904
<div class="variablelist"><dl>
905
<dt><span class="term"><code class="computeroutput">BZ_OK</code></span></dt>
906
<dd><p>The requested action was completed
907
successfully.</p></dd>
908
<dt><span class="term"><code class="computeroutput">BZ_RUN_OK, BZ_FLUSH_OK,
909
BZ_FINISH_OK</code></span></dt>
911
<code class="computeroutput">BZ2_bzCompress</code>, the requested
912
flush/finish/nothing-special action was completed
913
successfully.</p></dd>
914
<dt><span class="term"><code class="computeroutput">BZ_STREAM_END</code></span></dt>
915
<dd><p>Compression of data was completed, or the
916
logical stream end was detected during
917
decompression.</p></dd>
919
<p>The following return values indicate an error of some
921
<div class="variablelist"><dl>
922
<dt><span class="term"><code class="computeroutput">BZ_CONFIG_ERROR</code></span></dt>
923
<dd><p>Indicates that the library has been improperly
924
compiled on your platform -- a major configuration error.
925
Specifically, it means that
926
<code class="computeroutput">sizeof(char)</code>,
927
<code class="computeroutput">sizeof(short)</code> and
928
<code class="computeroutput">sizeof(int)</code> are not 1, 2 and
929
4 respectively, as they should be. Note that the library
930
should still work properly on 64-bit platforms which follow
931
the LP64 programming model -- that is, where
932
<code class="computeroutput">sizeof(long)</code> and
933
<code class="computeroutput">sizeof(void*)</code> are 8. Under
934
LP64, <code class="computeroutput">sizeof(int)</code> is still 4,
935
so <code class="computeroutput">libbzip2</code>, which doesn't
936
use the <code class="computeroutput">long</code> type, is
938
<dt><span class="term"><code class="computeroutput">BZ_SEQUENCE_ERROR</code></span></dt>
939
<dd><p>When using the library, it is important to call
940
the functions in the correct sequence and with data structures
941
(buffers etc) in the correct states.
942
<code class="computeroutput">libbzip2</code> checks as much as it
943
can to ensure this is happening, and returns
944
<code class="computeroutput">BZ_SEQUENCE_ERROR</code> if not.
945
Code which complies precisely with the function semantics, as
946
detailed below, should never receive this value; such an event
947
denotes buggy code which you should
948
investigate.</p></dd>
949
<dt><span class="term"><code class="computeroutput">BZ_PARAM_ERROR</code></span></dt>
950
<dd><p>Returned when a parameter to a function call is
951
out of range or otherwise manifestly incorrect. As with
952
<code class="computeroutput">BZ_SEQUENCE_ERROR</code>, this
953
denotes a bug in the client code. The distinction between
954
<code class="computeroutput">BZ_PARAM_ERROR</code> and
955
<code class="computeroutput">BZ_SEQUENCE_ERROR</code> is a bit
956
hazy, but still worth making.</p></dd>
957
<dt><span class="term"><code class="computeroutput">BZ_MEM_ERROR</code></span></dt>
958
<dd><p>Returned when a request to allocate memory
959
failed. Note that the quantity of memory needed to decompress
960
a stream cannot be determined until the stream's header has
962
<code class="computeroutput">BZ2_bzDecompress</code> and
963
<code class="computeroutput">BZ2_bzRead</code> may return
964
<code class="computeroutput">BZ_MEM_ERROR</code> even though some
965
of the compressed data has been read. The same is not true
966
for compression; once
967
<code class="computeroutput">BZ2_bzCompressInit</code> or
968
<code class="computeroutput">BZ2_bzWriteOpen</code> have
969
successfully completed,
970
<code class="computeroutput">BZ_MEM_ERROR</code> cannot
972
<dt><span class="term"><code class="computeroutput">BZ_DATA_ERROR</code></span></dt>
973
<dd><p>Returned when a data integrity error is
974
detected during decompression. Most importantly, this means
975
when stored and computed CRCs for the data do not match. This
976
value is also returned upon detection of any other anomaly in
977
the compressed data.</p></dd>
978
<dt><span class="term"><code class="computeroutput">BZ_DATA_ERROR_MAGIC</code></span></dt>
979
<dd><p>As a special case of
980
<code class="computeroutput">BZ_DATA_ERROR</code>, it is
981
sometimes useful to know when the compressed stream does not
982
start with the correct magic bytes (<code class="computeroutput">'B' 'Z'
983
'h'</code>).</p></dd>
984
<dt><span class="term"><code class="computeroutput">BZ_IO_ERROR</code></span></dt>
986
<code class="computeroutput">BZ2_bzRead</code> and
987
<code class="computeroutput">BZ2_bzWrite</code> when there is an
988
error reading or writing in the compressed file, and by
989
<code class="computeroutput">BZ2_bzReadOpen</code> and
990
<code class="computeroutput">BZ2_bzWriteOpen</code> for attempts
991
to use a file for which the error indicator (viz,
992
<code class="computeroutput">ferror(f)</code>) is set. On
993
receipt of <code class="computeroutput">BZ_IO_ERROR</code>, the
994
caller should consult <code class="computeroutput">errno</code>
995
and/or <code class="computeroutput">perror</code> to acquire
996
operating-system specific information about the
998
<dt><span class="term"><code class="computeroutput">BZ_UNEXPECTED_EOF</code></span></dt>
1000
<code class="computeroutput">BZ2_bzRead</code> when the
1001
compressed file finishes before the logical end of stream is
1003
<dt><span class="term"><code class="computeroutput">BZ_OUTBUFF_FULL</code></span></dt>
1005
<code class="computeroutput">BZ2_bzBuffToBuffCompress</code> and
1006
<code class="computeroutput">BZ2_bzBuffToBuffDecompress</code> to
1007
indicate that the output data will not fit into the output
1008
buffer provided.</p></dd>
1011
<div class="sect1" title="3.3.�Low-level interface">
1012
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
1013
<a name="low-level"></a>3.3.�Low-level interface</h2></div></div></div>
1014
<div class="sect2" title="3.3.1.�BZ2_bzCompressInit">
1015
<div class="titlepage"><div><div><h3 class="title">
1016
<a name="bzcompress-init"></a>3.3.1.�BZ2_bzCompressInit</h3></div></div></div>
1017
<pre class="programlisting">typedef struct {
1019
unsigned int avail_in;
1020
unsigned int total_in_lo32;
1021
unsigned int total_in_hi32;
1024
unsigned int avail_out;
1025
unsigned int total_out_lo32;
1026
unsigned int total_out_hi32;
1030
void *(*bzalloc)(void *,int,int);
1031
void (*bzfree)(void *,void *);
1035
int BZ2_bzCompressInit ( bz_stream *strm,
1038
int workFactor );</pre>
1039
<p>Prepares for compression. The
1040
<code class="computeroutput">bz_stream</code> structure holds all
1041
data pertaining to the compression activity. A
1042
<code class="computeroutput">bz_stream</code> structure should be
1043
allocated and initialised prior to the call. The fields of
1044
<code class="computeroutput">bz_stream</code> comprise the entirety
1045
of the user-visible data. <code class="computeroutput">state</code>
1046
is a pointer to the private data structures required for
1048
<p>Custom memory allocators are supported, via fields
1049
<code class="computeroutput">bzalloc</code>,
1050
<code class="computeroutput">bzfree</code>, and
1051
<code class="computeroutput">opaque</code>. The value
1052
<code class="computeroutput">opaque</code> is passed to as the first
1053
argument to all calls to <code class="computeroutput">bzalloc</code>
1054
and <code class="computeroutput">bzfree</code>, but is otherwise
1055
ignored by the library. The call <code class="computeroutput">bzalloc (
1056
opaque, n, m )</code> is expected to return a pointer
1057
<code class="computeroutput">p</code> to <code class="computeroutput">n *
1058
m</code> bytes of memory, and <code class="computeroutput">bzfree (
1059
opaque, p )</code> should free that memory.</p>
1060
<p>If you don't want to use a custom memory allocator, set
1061
<code class="computeroutput">bzalloc</code>,
1062
<code class="computeroutput">bzfree</code> and
1063
<code class="computeroutput">opaque</code> to
1064
<code class="computeroutput">NULL</code>, and the library will then
1065
use the standard <code class="computeroutput">malloc</code> /
1066
<code class="computeroutput">free</code> routines.</p>
1068
<code class="computeroutput">BZ2_bzCompressInit</code>, fields
1069
<code class="computeroutput">bzalloc</code>,
1070
<code class="computeroutput">bzfree</code> and
1071
<code class="computeroutput">opaque</code> should be filled
1072
appropriately, as just described. Upon return, the internal
1073
state will have been allocated and initialised, and
1074
<code class="computeroutput">total_in_lo32</code>,
1075
<code class="computeroutput">total_in_hi32</code>,
1076
<code class="computeroutput">total_out_lo32</code> and
1077
<code class="computeroutput">total_out_hi32</code> will have been
1078
set to zero. These four fields are used by the library to inform
1079
the caller of the total amount of data passed into and out of the
1080
library, respectively. You should not try to change them. As of
1081
version 1.0, 64-bit counts are maintained, even on 32-bit
1082
platforms, using the <code class="computeroutput">_hi32</code>
1083
fields to store the upper 32 bits of the count. So, for example,
1084
the total amount of data in is <code class="computeroutput">(total_in_hi32
1085
<< 32) + total_in_lo32</code>.</p>
1086
<p>Parameter <code class="computeroutput">blockSize100k</code>
1087
specifies the block size to be used for compression. It should
1088
be a value between 1 and 9 inclusive, and the actual block size
1089
used is 100000 x this figure. 9 gives the best compression but
1090
takes most memory.</p>
1091
<p>Parameter <code class="computeroutput">verbosity</code> should
1092
be set to a number between 0 and 4 inclusive. 0 is silent, and
1093
greater numbers give increasingly verbose monitoring/debugging
1094
output. If the library has been compiled with
1095
<code class="computeroutput">-DBZ_NO_STDIO</code>, no such output
1096
will appear for any verbosity setting.</p>
1097
<p>Parameter <code class="computeroutput">workFactor</code>
1098
controls how the compression phase behaves when presented with
1099
worst case, highly repetitive, input data. If compression runs
1100
into difficulties caused by repetitive data, the library switches
1101
from the standard sorting algorithm to a fallback algorithm. The
1102
fallback is slower than the standard algorithm by perhaps a
1103
factor of three, but always behaves reasonably, no matter how bad
1105
<p>Lower values of <code class="computeroutput">workFactor</code>
1106
reduce the amount of effort the standard algorithm will expend
1107
before resorting to the fallback. You should set this parameter
1108
carefully; too low, and many inputs will be handled by the
1109
fallback algorithm and so compress rather slowly, too high, and
1110
your average-to-worst case compression times can become very
1111
large. The default value of 30 gives reasonable behaviour over a
1112
wide range of circumstances.</p>
1113
<p>Allowable values range from 0 to 250 inclusive. 0 is a
1114
special case, equivalent to using the default value of 30.</p>
1115
<p>Note that the compressed output generated is the same
1116
regardless of whether or not the fallback algorithm is
1118
<p>Be aware also that this parameter may disappear entirely in
1119
future versions of the library. In principle it should be
1120
possible to devise a good way to automatically choose which
1121
algorithm to use. Such a mechanism would render the parameter
1123
<p>Possible return values:</p>
1124
<pre class="programlisting">BZ_CONFIG_ERROR
1125
if the library has been mis-compiled
1128
or blockSize < 1 or blockSize > 9
1129
or verbosity < 0 or verbosity > 4
1130
or workFactor < 0 or workFactor > 250
1132
if not enough memory is available
1135
<p>Allowable next actions:</p>
1136
<pre class="programlisting">BZ2_bzCompress
1137
if BZ_OK is returned
1138
no specific action needed in case of error</pre>
1140
<div class="sect2" title="3.3.2.�BZ2_bzCompress">
1141
<div class="titlepage"><div><div><h3 class="title">
1142
<a name="bzCompress"></a>3.3.2.�BZ2_bzCompress</h3></div></div></div>
1143
<pre class="programlisting">int BZ2_bzCompress ( bz_stream *strm, int action );</pre>
1144
<p>Provides more input and/or output buffer space for the
1145
library. The caller maintains input and output buffers, and
1146
calls <code class="computeroutput">BZ2_bzCompress</code> to transfer
1147
data between them.</p>
1148
<p>Before each call to
1149
<code class="computeroutput">BZ2_bzCompress</code>,
1150
<code class="computeroutput">next_in</code> should point at the data
1151
to be compressed, and <code class="computeroutput">avail_in</code>
1152
should indicate how many bytes the library may read.
1153
<code class="computeroutput">BZ2_bzCompress</code> updates
1154
<code class="computeroutput">next_in</code>,
1155
<code class="computeroutput">avail_in</code> and
1156
<code class="computeroutput">total_in</code> to reflect the number
1157
of bytes it has read.</p>
1158
<p>Similarly, <code class="computeroutput">next_out</code> should
1159
point to a buffer in which the compressed data is to be placed,
1160
with <code class="computeroutput">avail_out</code> indicating how
1161
much output space is available.
1162
<code class="computeroutput">BZ2_bzCompress</code> updates
1163
<code class="computeroutput">next_out</code>,
1164
<code class="computeroutput">avail_out</code> and
1165
<code class="computeroutput">total_out</code> to reflect the number
1166
of bytes output.</p>
1167
<p>You may provide and remove as little or as much data as you
1168
like on each call of
1169
<code class="computeroutput">BZ2_bzCompress</code>. In the limit,
1170
it is acceptable to supply and remove data one byte at a time,
1171
although this would be terribly inefficient. You should always
1172
ensure that at least one byte of output space is available at
1174
<p>A second purpose of
1175
<code class="computeroutput">BZ2_bzCompress</code> is to request a
1176
change of mode of the compressed stream.</p>
1177
<p>Conceptually, a compressed stream can be in one of four
1178
states: IDLE, RUNNING, FLUSHING and FINISHING. Before
1180
(<code class="computeroutput">BZ2_bzCompressInit</code>) and after
1181
termination (<code class="computeroutput">BZ2_bzCompressEnd</code>),
1182
a stream is regarded as IDLE.</p>
1183
<p>Upon initialisation
1184
(<code class="computeroutput">BZ2_bzCompressInit</code>), the stream
1185
is placed in the RUNNING state. Subsequent calls to
1186
<code class="computeroutput">BZ2_bzCompress</code> should pass
1187
<code class="computeroutput">BZ_RUN</code> as the requested action;
1188
other actions are illegal and will result in
1189
<code class="computeroutput">BZ_SEQUENCE_ERROR</code>.</p>
1190
<p>At some point, the calling program will have provided all
1191
the input data it wants to. It will then want to finish up -- in
1192
effect, asking the library to process any data it might have
1193
buffered internally. In this state,
1194
<code class="computeroutput">BZ2_bzCompress</code> will no longer
1195
attempt to read data from
1196
<code class="computeroutput">next_in</code>, but it will want to
1197
write data to <code class="computeroutput">next_out</code>. Because
1198
the output buffer supplied by the user can be arbitrarily small,
1199
the finishing-up operation cannot necessarily be done with a
1201
<code class="computeroutput">BZ2_bzCompress</code>.</p>
1202
<p>Instead, the calling program passes
1203
<code class="computeroutput">BZ_FINISH</code> as an action to
1204
<code class="computeroutput">BZ2_bzCompress</code>. This changes
1205
the stream's state to FINISHING. Any remaining input (ie,
1206
<code class="computeroutput">next_in[0 .. avail_in-1]</code>) is
1207
compressed and transferred to the output buffer. To do this,
1208
<code class="computeroutput">BZ2_bzCompress</code> must be called
1209
repeatedly until all the output has been consumed. At that
1210
point, <code class="computeroutput">BZ2_bzCompress</code> returns
1211
<code class="computeroutput">BZ_STREAM_END</code>, and the stream's
1212
state is set back to IDLE.
1213
<code class="computeroutput">BZ2_bzCompressEnd</code> should then be
1215
<p>Just to make sure the calling program does not cheat, the
1216
library makes a note of <code class="computeroutput">avail_in</code>
1217
at the time of the first call to
1218
<code class="computeroutput">BZ2_bzCompress</code> which has
1219
<code class="computeroutput">BZ_FINISH</code> as an action (ie, at
1220
the time the program has announced its intention to not supply
1221
any more input). By comparing this value with that of
1222
<code class="computeroutput">avail_in</code> over subsequent calls
1223
to <code class="computeroutput">BZ2_bzCompress</code>, the library
1224
can detect any attempts to slip in more data to compress. Any
1225
calls for which this is detected will return
1226
<code class="computeroutput">BZ_SEQUENCE_ERROR</code>. This
1227
indicates a programming mistake which should be corrected.</p>
1228
<p>Instead of asking to finish, the calling program may ask
1229
<code class="computeroutput">BZ2_bzCompress</code> to take all the
1230
remaining input, compress it and terminate the current
1231
(Burrows-Wheeler) compression block. This could be useful for
1232
error control purposes. The mechanism is analogous to that for
1233
finishing: call <code class="computeroutput">BZ2_bzCompress</code>
1234
with an action of <code class="computeroutput">BZ_FLUSH</code>,
1235
remove output data, and persist with the
1236
<code class="computeroutput">BZ_FLUSH</code> action until the value
1237
<code class="computeroutput">BZ_RUN</code> is returned. As with
1238
finishing, <code class="computeroutput">BZ2_bzCompress</code>
1239
detects any attempt to provide more input data once the flush has
1241
<p>Once the flush is complete, the stream returns to the
1242
normal RUNNING state.</p>
1243
<p>This all sounds pretty complex, but isn't really. Here's a
1244
table which shows which actions are allowable in each state, what
1245
action will be taken, what the next state is, and what the
1246
non-error return values are. Note that you can't explicitly ask
1247
what state the stream is in, but nor do you need to -- it can be
1248
inferred from the values returned by
1249
<code class="computeroutput">BZ2_bzCompress</code>.</p>
1250
<pre class="programlisting">IDLE/any
1251
Illegal. IDLE state only exists after BZ2_bzCompressEnd or
1252
before BZ2_bzCompressInit.
1253
Return value = BZ_SEQUENCE_ERROR
1256
Compress from next_in to next_out as much as possible.
1257
Next state = RUNNING
1258
Return value = BZ_RUN_OK
1261
Remember current value of next_in. Compress from next_in
1262
to next_out as much as possible, but do not accept any more input.
1263
Next state = FLUSHING
1264
Return value = BZ_FLUSH_OK
1267
Remember current value of next_in. Compress from next_in
1268
to next_out as much as possible, but do not accept any more input.
1269
Next state = FINISHING
1270
Return value = BZ_FINISH_OK
1273
Compress from next_in to next_out as much as possible,
1274
but do not accept any more input.
1275
If all the existing input has been used up and all compressed
1276
output has been removed
1277
Next state = RUNNING; Return value = BZ_RUN_OK
1279
Next state = FLUSHING; Return value = BZ_FLUSH_OK
1283
Return value = BZ_SEQUENCE_ERROR
1286
Compress from next_in to next_out as much as possible,
1287
but to not accept any more input.
1288
If all the existing input has been used up and all compressed
1289
output has been removed
1290
Next state = IDLE; Return value = BZ_STREAM_END
1292
Next state = FINISHING; Return value = BZ_FINISH_OK
1296
Return value = BZ_SEQUENCE_ERROR</pre>
1297
<p>That still looks complicated? Well, fair enough. The
1298
usual sequence of calls for compressing a load of data is:</p>
1299
<div class="orderedlist"><ol class="orderedlist" type="1">
1300
<li class="listitem"><p>Get started with
1301
<code class="computeroutput">BZ2_bzCompressInit</code>.</p></li>
1302
<li class="listitem"><p>Shovel data in and shlurp out its compressed form
1303
using zero or more calls of
1304
<code class="computeroutput">BZ2_bzCompress</code> with action =
1305
<code class="computeroutput">BZ_RUN</code>.</p></li>
1306
<li class="listitem"><p>Finish up. Repeatedly call
1307
<code class="computeroutput">BZ2_bzCompress</code> with action =
1308
<code class="computeroutput">BZ_FINISH</code>, copying out the
1309
compressed output, until
1310
<code class="computeroutput">BZ_STREAM_END</code> is
1312
<li class="listitem"><p>Close up and go home. Call
1313
<code class="computeroutput">BZ2_bzCompressEnd</code>.</p></li>
1315
<p>If the data you want to compress fits into your input
1316
buffer all at once, you can skip the calls of
1317
<code class="computeroutput">BZ2_bzCompress ( ..., BZ_RUN )</code>
1318
and just do the <code class="computeroutput">BZ2_bzCompress ( ..., BZ_FINISH
1320
<p>All required memory is allocated by
1321
<code class="computeroutput">BZ2_bzCompressInit</code>. The
1322
compression library can accept any data at all (obviously). So
1323
you shouldn't get any error return values from the
1324
<code class="computeroutput">BZ2_bzCompress</code> calls. If you
1326
<code class="computeroutput">BZ_SEQUENCE_ERROR</code>, and indicate
1327
a bug in your programming.</p>
1328
<p>Trivial other possible return values:</p>
1329
<pre class="programlisting">BZ_PARAM_ERROR
1330
if strm is NULL, or strm->s is NULL</pre>
1332
<div class="sect2" title="3.3.3.�BZ2_bzCompressEnd">
1333
<div class="titlepage"><div><div><h3 class="title">
1334
<a name="bzCompress-end"></a>3.3.3.�BZ2_bzCompressEnd</h3></div></div></div>
1335
<pre class="programlisting">int BZ2_bzCompressEnd ( bz_stream *strm );</pre>
1336
<p>Releases all memory associated with a compression
1338
<p>Possible return values:</p>
1339
<pre class="programlisting">BZ_PARAM_ERROR if strm is NULL or strm->s is NULL
1340
BZ_OK otherwise</pre>
1342
<div class="sect2" title="3.3.4.�BZ2_bzDecompressInit">
1343
<div class="titlepage"><div><div><h3 class="title">
1344
<a name="bzDecompress-init"></a>3.3.4.�BZ2_bzDecompressInit</h3></div></div></div>
1345
<pre class="programlisting">int BZ2_bzDecompressInit ( bz_stream *strm, int verbosity, int small );</pre>
1346
<p>Prepares for decompression. As with
1347
<code class="computeroutput">BZ2_bzCompressInit</code>, a
1348
<code class="computeroutput">bz_stream</code> record should be
1349
allocated and initialised before the call. Fields
1350
<code class="computeroutput">bzalloc</code>,
1351
<code class="computeroutput">bzfree</code> and
1352
<code class="computeroutput">opaque</code> should be set if a custom
1353
memory allocator is required, or made
1354
<code class="computeroutput">NULL</code> for the normal
1355
<code class="computeroutput">malloc</code> /
1356
<code class="computeroutput">free</code> routines. Upon return, the
1357
internal state will have been initialised, and
1358
<code class="computeroutput">total_in</code> and
1359
<code class="computeroutput">total_out</code> will be zero.</p>
1360
<p>For the meaning of parameter
1361
<code class="computeroutput">verbosity</code>, see
1362
<code class="computeroutput">BZ2_bzCompressInit</code>.</p>
1363
<p>If <code class="computeroutput">small</code> is nonzero, the
1364
library will use an alternative decompression algorithm which
1365
uses less memory but at the cost of decompressing more slowly
1366
(roughly speaking, half the speed, but the maximum memory
1367
requirement drops to around 2300k). See <a class="xref" href="#using" title="2.�How to use bzip2">How to use bzip2</a>
1368
for more information on memory management.</p>
1369
<p>Note that the amount of memory needed to decompress a
1370
stream cannot be determined until the stream's header has been
1372
<code class="computeroutput">BZ2_bzDecompressInit</code> succeeds, a
1373
subsequent <code class="computeroutput">BZ2_bzDecompress</code>
1375
<code class="computeroutput">BZ_MEM_ERROR</code>.</p>
1376
<p>Possible return values:</p>
1377
<pre class="programlisting">BZ_CONFIG_ERROR
1378
if the library has been mis-compiled
1380
if ( small != 0 && small != 1 )
1381
or (verbosity <; 0 || verbosity > 4)
1383
if insufficient memory is available</pre>
1384
<p>Allowable next actions:</p>
1385
<pre class="programlisting">BZ2_bzDecompress
1386
if BZ_OK was returned
1387
no specific action required in case of error</pre>
1389
<div class="sect2" title="3.3.5.�BZ2_bzDecompress">
1390
<div class="titlepage"><div><div><h3 class="title">
1391
<a name="bzDecompress"></a>3.3.5.�BZ2_bzDecompress</h3></div></div></div>
1392
<pre class="programlisting">int BZ2_bzDecompress ( bz_stream *strm );</pre>
1393
<p>Provides more input and/out output buffer space for the
1394
library. The caller maintains input and output buffers, and uses
1395
<code class="computeroutput">BZ2_bzDecompress</code> to transfer
1396
data between them.</p>
1397
<p>Before each call to
1398
<code class="computeroutput">BZ2_bzDecompress</code>,
1399
<code class="computeroutput">next_in</code> should point at the
1400
compressed data, and <code class="computeroutput">avail_in</code>
1401
should indicate how many bytes the library may read.
1402
<code class="computeroutput">BZ2_bzDecompress</code> updates
1403
<code class="computeroutput">next_in</code>,
1404
<code class="computeroutput">avail_in</code> and
1405
<code class="computeroutput">total_in</code> to reflect the number
1406
of bytes it has read.</p>
1407
<p>Similarly, <code class="computeroutput">next_out</code> should
1408
point to a buffer in which the uncompressed output is to be
1409
placed, with <code class="computeroutput">avail_out</code>
1410
indicating how much output space is available.
1411
<code class="computeroutput">BZ2_bzCompress</code> updates
1412
<code class="computeroutput">next_out</code>,
1413
<code class="computeroutput">avail_out</code> and
1414
<code class="computeroutput">total_out</code> to reflect the number
1415
of bytes output.</p>
1416
<p>You may provide and remove as little or as much data as you
1417
like on each call of
1418
<code class="computeroutput">BZ2_bzDecompress</code>. In the limit,
1419
it is acceptable to supply and remove data one byte at a time,
1420
although this would be terribly inefficient. You should always
1421
ensure that at least one byte of output space is available at
1423
<p>Use of <code class="computeroutput">BZ2_bzDecompress</code> is
1425
<code class="computeroutput">BZ2_bzCompress</code>.</p>
1426
<p>You should provide input and remove output as described
1427
above, and repeatedly call
1428
<code class="computeroutput">BZ2_bzDecompress</code> until
1429
<code class="computeroutput">BZ_STREAM_END</code> is returned.
1430
Appearance of <code class="computeroutput">BZ_STREAM_END</code>
1431
denotes that <code class="computeroutput">BZ2_bzDecompress</code>
1432
has detected the logical end of the compressed stream.
1433
<code class="computeroutput">BZ2_bzDecompress</code> will not
1434
produce <code class="computeroutput">BZ_STREAM_END</code> until all
1435
output data has been placed into the output buffer, so once
1436
<code class="computeroutput">BZ_STREAM_END</code> appears, you are
1437
guaranteed to have available all the decompressed output, and
1438
<code class="computeroutput">BZ2_bzDecompressEnd</code> can safely
1440
<p>If case of an error return value, you should call
1441
<code class="computeroutput">BZ2_bzDecompressEnd</code> to clean up
1442
and release memory.</p>
1443
<p>Possible return values:</p>
1444
<pre class="programlisting">BZ_PARAM_ERROR
1445
if strm is NULL or strm->s is NULL
1446
or strm->avail_out < 1
1448
if a data integrity error is detected in the compressed stream
1450
if the compressed stream doesn't begin with the right magic bytes
1452
if there wasn't enough memory available
1454
if the logical end of the data stream was detected and all
1455
output in has been consumed, eg s-->avail_out > 0
1458
<p>Allowable next actions:</p>
1459
<pre class="programlisting">BZ2_bzDecompress
1460
if BZ_OK was returned
1464
<div class="sect2" title="3.3.6.�BZ2_bzDecompressEnd">
1465
<div class="titlepage"><div><div><h3 class="title">
1466
<a name="bzDecompress-end"></a>3.3.6.�BZ2_bzDecompressEnd</h3></div></div></div>
1467
<pre class="programlisting">int BZ2_bzDecompressEnd ( bz_stream *strm );</pre>
1468
<p>Releases all memory associated with a decompression
1470
<p>Possible return values:</p>
1471
<pre class="programlisting">BZ_PARAM_ERROR
1472
if strm is NULL or strm->s is NULL
1475
<p>Allowable next actions:</p>
1476
<pre class="programlisting"> None.</pre>
1479
<div class="sect1" title="3.4.�High-level interface">
1480
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
1481
<a name="hl-interface"></a>3.4.�High-level interface</h2></div></div></div>
1482
<p>This interface provides functions for reading and writing
1483
<code class="computeroutput">bzip2</code> format files. First, some
1485
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
1486
<li class="listitem" style="list-style-type: disc"><p>All of the functions take an
1487
<code class="computeroutput">int*</code> first argument,
1488
<code class="computeroutput">bzerror</code>. After each call,
1489
<code class="computeroutput">bzerror</code> should be consulted
1490
first to determine the outcome of the call. If
1491
<code class="computeroutput">bzerror</code> is
1492
<code class="computeroutput">BZ_OK</code>, the call completed
1493
successfully, and only then should the return value of the
1494
function (if any) be consulted. If
1495
<code class="computeroutput">bzerror</code> is
1496
<code class="computeroutput">BZ_IO_ERROR</code>, there was an
1497
error reading/writing the underlying compressed file, and you
1498
should then consult <code class="computeroutput">errno</code> /
1499
<code class="computeroutput">perror</code> to determine the cause
1500
of the difficulty. <code class="computeroutput">bzerror</code>
1501
may also be set to various other values; precise details are
1502
given on a per-function basis below.</p></li>
1503
<li class="listitem" style="list-style-type: disc"><p>If <code class="computeroutput">bzerror</code> indicates
1504
an error (ie, anything except
1505
<code class="computeroutput">BZ_OK</code> and
1506
<code class="computeroutput">BZ_STREAM_END</code>), you should
1508
<code class="computeroutput">BZ2_bzReadClose</code> (or
1509
<code class="computeroutput">BZ2_bzWriteClose</code>, depending on
1510
whether you are attempting to read or to write) to free up all
1511
resources associated with the stream. Once an error has been
1512
indicated, behaviour of all calls except
1513
<code class="computeroutput">BZ2_bzReadClose</code>
1514
(<code class="computeroutput">BZ2_bzWriteClose</code>) is
1515
undefined. The implication is that (1)
1516
<code class="computeroutput">bzerror</code> should be checked
1517
after each call, and (2) if
1518
<code class="computeroutput">bzerror</code> indicates an error,
1519
<code class="computeroutput">BZ2_bzReadClose</code>
1520
(<code class="computeroutput">BZ2_bzWriteClose</code>) should then
1521
be called to clean up.</p></li>
1522
<li class="listitem" style="list-style-type: disc"><p>The <code class="computeroutput">FILE*</code> arguments
1523
passed to <code class="computeroutput">BZ2_bzReadOpen</code> /
1524
<code class="computeroutput">BZ2_bzWriteOpen</code> should be set
1525
to binary mode. Most Unix systems will do this by default, but
1526
other platforms, including Windows and Mac, will not. If you
1527
omit this, you may encounter problems when moving code to new
1529
<li class="listitem" style="list-style-type: disc"><p>Memory allocation requests are handled by
1530
<code class="computeroutput">malloc</code> /
1531
<code class="computeroutput">free</code>. At present there is no
1532
facility for user-defined memory allocators in the file I/O
1533
functions (could easily be added, though).</p></li>
1535
<div class="sect2" title="3.4.1.�BZ2_bzReadOpen">
1536
<div class="titlepage"><div><div><h3 class="title">
1537
<a name="bzreadopen"></a>3.4.1.�BZ2_bzReadOpen</h3></div></div></div>
1538
<pre class="programlisting">typedef void BZFILE;
1540
BZFILE *BZ2_bzReadOpen( int *bzerror, FILE *f,
1541
int verbosity, int small,
1542
void *unused, int nUnused );</pre>
1543
<p>Prepare to read compressed data from file handle
1544
<code class="computeroutput">f</code>.
1545
<code class="computeroutput">f</code> should refer to a file which
1546
has been opened for reading, and for which the error indicator
1547
(<code class="computeroutput">ferror(f)</code>)is not set. If
1548
<code class="computeroutput">small</code> is 1, the library will try
1549
to decompress using less memory, at the expense of speed.</p>
1550
<p>For reasons explained below,
1551
<code class="computeroutput">BZ2_bzRead</code> will decompress the
1552
<code class="computeroutput">nUnused</code> bytes starting at
1553
<code class="computeroutput">unused</code>, before starting to read
1554
from the file <code class="computeroutput">f</code>. At most
1555
<code class="computeroutput">BZ_MAX_UNUSED</code> bytes may be
1556
supplied like this. If this facility is not required, you should
1557
pass <code class="computeroutput">NULL</code> and
1558
<code class="computeroutput">0</code> for
1559
<code class="computeroutput">unused</code> and
1560
n<code class="computeroutput">Unused</code> respectively.</p>
1561
<p>For the meaning of parameters
1562
<code class="computeroutput">small</code> and
1563
<code class="computeroutput">verbosity</code>, see
1564
<code class="computeroutput">BZ2_bzDecompressInit</code>.</p>
1565
<p>The amount of memory needed to decompress a file cannot be
1566
determined until the file's header has been read. So it is
1567
possible that <code class="computeroutput">BZ2_bzReadOpen</code>
1568
returns <code class="computeroutput">BZ_OK</code> but a subsequent
1569
call of <code class="computeroutput">BZ2_bzRead</code> will return
1570
<code class="computeroutput">BZ_MEM_ERROR</code>.</p>
1571
<p>Possible assignments to
1572
<code class="computeroutput">bzerror</code>:</p>
1573
<pre class="programlisting">BZ_CONFIG_ERROR
1574
if the library has been mis-compiled
1577
or small is neither 0 nor 1
1578
or ( unused == NULL && nUnused != 0 )
1579
or ( unused != NULL && !(0 <= nUnused <= BZ_MAX_UNUSED) )
1581
if ferror(f) is nonzero
1583
if insufficient memory is available
1586
<p>Possible return values:</p>
1587
<pre class="programlisting">Pointer to an abstract BZFILE
1591
<p>Allowable next actions:</p>
1592
<pre class="programlisting">BZ2_bzRead
1597
<div class="sect2" title="3.4.2.�BZ2_bzRead">
1598
<div class="titlepage"><div><div><h3 class="title">
1599
<a name="bzread"></a>3.4.2.�BZ2_bzRead</h3></div></div></div>
1600
<pre class="programlisting">int BZ2_bzRead ( int *bzerror, BZFILE *b, void *buf, int len );</pre>
1601
<p>Reads up to <code class="computeroutput">len</code>
1602
(uncompressed) bytes from the compressed file
1603
<code class="computeroutput">b</code> into the buffer
1604
<code class="computeroutput">buf</code>. If the read was
1605
successful, <code class="computeroutput">bzerror</code> is set to
1606
<code class="computeroutput">BZ_OK</code> and the number of bytes
1607
read is returned. If the logical end-of-stream was detected,
1608
<code class="computeroutput">bzerror</code> will be set to
1609
<code class="computeroutput">BZ_STREAM_END</code>, and the number of
1610
bytes read is returned. All other
1611
<code class="computeroutput">bzerror</code> values denote an
1613
<p><code class="computeroutput">BZ2_bzRead</code> will supply
1614
<code class="computeroutput">len</code> bytes, unless the logical
1615
stream end is detected or an error occurs. Because of this, it
1616
is possible to detect the stream end by observing when the number
1617
of bytes returned is less than the number requested.
1618
Nevertheless, this is regarded as inadvisable; you should instead
1619
check <code class="computeroutput">bzerror</code> after every call
1621
<code class="computeroutput">BZ_STREAM_END</code>.</p>
1622
<p>Internally, <code class="computeroutput">BZ2_bzRead</code>
1623
copies data from the compressed file in chunks of size
1624
<code class="computeroutput">BZ_MAX_UNUSED</code> bytes before
1625
decompressing it. If the file contains more bytes than strictly
1626
needed to reach the logical end-of-stream,
1627
<code class="computeroutput">BZ2_bzRead</code> will almost certainly
1628
read some of the trailing data before signalling
1629
<code class="computeroutput">BZ_SEQUENCE_END</code>. To collect the
1630
read but unused data once
1631
<code class="computeroutput">BZ_SEQUENCE_END</code> has appeared,
1632
call <code class="computeroutput">BZ2_bzReadGetUnused</code>
1634
<code class="computeroutput">BZ2_bzReadClose</code>.</p>
1635
<p>Possible assignments to
1636
<code class="computeroutput">bzerror</code>:</p>
1637
<pre class="programlisting">BZ_PARAM_ERROR
1638
if b is NULL or buf is NULL or len < 0
1640
if b was opened with BZ2_bzWriteOpen
1642
if there is an error reading from the compressed file
1644
if the compressed file ended before
1645
the logical end-of-stream was detected
1647
if a data integrity error was detected in the compressed stream
1649
if the stream does not begin with the requisite header bytes
1650
(ie, is not a bzip2 data file). This is really
1651
a special case of BZ_DATA_ERROR.
1653
if insufficient memory was available
1655
if the logical end of stream was detected.
1658
<p>Possible return values:</p>
1659
<pre class="programlisting">number of bytes read
1660
if bzerror is BZ_OK or BZ_STREAM_END
1663
<p>Allowable next actions:</p>
1664
<pre class="programlisting">collect data from buf, then BZ2_bzRead or BZ2_bzReadClose
1666
collect data from buf, then BZ2_bzReadClose or BZ2_bzReadGetUnused
1667
if bzerror is BZ_SEQUENCE_END
1671
<div class="sect2" title="3.4.3.�BZ2_bzReadGetUnused">
1672
<div class="titlepage"><div><div><h3 class="title">
1673
<a name="bzreadgetunused"></a>3.4.3.�BZ2_bzReadGetUnused</h3></div></div></div>
1674
<pre class="programlisting">void BZ2_bzReadGetUnused( int* bzerror, BZFILE *b,
1675
void** unused, int* nUnused );</pre>
1676
<p>Returns data which was read from the compressed file but
1677
was not needed to get to the logical end-of-stream.
1678
<code class="computeroutput">*unused</code> is set to the address of
1679
the data, and <code class="computeroutput">*nUnused</code> to the
1680
number of bytes. <code class="computeroutput">*nUnused</code> will
1681
be set to a value between <code class="computeroutput">0</code> and
1682
<code class="computeroutput">BZ_MAX_UNUSED</code> inclusive.</p>
1683
<p>This function may only be called once
1684
<code class="computeroutput">BZ2_bzRead</code> has signalled
1685
<code class="computeroutput">BZ_STREAM_END</code> but before
1686
<code class="computeroutput">BZ2_bzReadClose</code>.</p>
1687
<p>Possible assignments to
1688
<code class="computeroutput">bzerror</code>:</p>
1689
<pre class="programlisting">BZ_PARAM_ERROR
1691
or unused is NULL or nUnused is NULL
1693
if BZ_STREAM_END has not been signalled
1694
or if b was opened with BZ2_bzWriteOpen
1697
<p>Allowable next actions:</p>
1698
<pre class="programlisting">BZ2_bzReadClose</pre>
1700
<div class="sect2" title="3.4.4.�BZ2_bzReadClose">
1701
<div class="titlepage"><div><div><h3 class="title">
1702
<a name="bzreadclose"></a>3.4.4.�BZ2_bzReadClose</h3></div></div></div>
1703
<pre class="programlisting">void BZ2_bzReadClose ( int *bzerror, BZFILE *b );</pre>
1704
<p>Releases all memory pertaining to the compressed file
1705
<code class="computeroutput">b</code>.
1706
<code class="computeroutput">BZ2_bzReadClose</code> does not call
1707
<code class="computeroutput">fclose</code> on the underlying file
1708
handle, so you should do that yourself if appropriate.
1709
<code class="computeroutput">BZ2_bzReadClose</code> should be called
1710
to clean up after all error situations.</p>
1711
<p>Possible assignments to
1712
<code class="computeroutput">bzerror</code>:</p>
1713
<pre class="programlisting">BZ_SEQUENCE_ERROR
1714
if b was opened with BZ2_bzOpenWrite
1717
<p>Allowable next actions:</p>
1718
<pre class="programlisting">none</pre>
1720
<div class="sect2" title="3.4.5.�BZ2_bzWriteOpen">
1721
<div class="titlepage"><div><div><h3 class="title">
1722
<a name="bzwriteopen"></a>3.4.5.�BZ2_bzWriteOpen</h3></div></div></div>
1723
<pre class="programlisting">BZFILE *BZ2_bzWriteOpen( int *bzerror, FILE *f,
1724
int blockSize100k, int verbosity,
1725
int workFactor );</pre>
1726
<p>Prepare to write compressed data to file handle
1727
<code class="computeroutput">f</code>.
1728
<code class="computeroutput">f</code> should refer to a file which
1729
has been opened for writing, and for which the error indicator
1730
(<code class="computeroutput">ferror(f)</code>)is not set.</p>
1731
<p>For the meaning of parameters
1732
<code class="computeroutput">blockSize100k</code>,
1733
<code class="computeroutput">verbosity</code> and
1734
<code class="computeroutput">workFactor</code>, see
1735
<code class="computeroutput">BZ2_bzCompressInit</code>.</p>
1736
<p>All required memory is allocated at this stage, so if the
1737
call completes successfully,
1738
<code class="computeroutput">BZ_MEM_ERROR</code> cannot be signalled
1739
by a subsequent call to
1740
<code class="computeroutput">BZ2_bzWrite</code>.</p>
1741
<p>Possible assignments to
1742
<code class="computeroutput">bzerror</code>:</p>
1743
<pre class="programlisting">BZ_CONFIG_ERROR
1744
if the library has been mis-compiled
1747
or blockSize100k < 1 or blockSize100k > 9
1749
if ferror(f) is nonzero
1751
if insufficient memory is available
1754
<p>Possible return values:</p>
1755
<pre class="programlisting">Pointer to an abstract BZFILE
1759
<p>Allowable next actions:</p>
1760
<pre class="programlisting">BZ2_bzWrite
1762
(you could go directly to BZ2_bzWriteClose, but this would be pretty pointless)
1766
<div class="sect2" title="3.4.6.�BZ2_bzWrite">
1767
<div class="titlepage"><div><div><h3 class="title">
1768
<a name="bzwrite"></a>3.4.6.�BZ2_bzWrite</h3></div></div></div>
1769
<pre class="programlisting">void BZ2_bzWrite ( int *bzerror, BZFILE *b, void *buf, int len );</pre>
1770
<p>Absorbs <code class="computeroutput">len</code> bytes from the
1771
buffer <code class="computeroutput">buf</code>, eventually to be
1772
compressed and written to the file.</p>
1773
<p>Possible assignments to
1774
<code class="computeroutput">bzerror</code>:</p>
1775
<pre class="programlisting">BZ_PARAM_ERROR
1776
if b is NULL or buf is NULL or len < 0
1778
if b was opened with BZ2_bzReadOpen
1780
if there is an error writing the compressed file.
1784
<div class="sect2" title="3.4.7.�BZ2_bzWriteClose">
1785
<div class="titlepage"><div><div><h3 class="title">
1786
<a name="bzwriteclose"></a>3.4.7.�BZ2_bzWriteClose</h3></div></div></div>
1787
<pre class="programlisting">void BZ2_bzWriteClose( int *bzerror, BZFILE* f,
1789
unsigned int* nbytes_in,
1790
unsigned int* nbytes_out );
1792
void BZ2_bzWriteClose64( int *bzerror, BZFILE* f,
1794
unsigned int* nbytes_in_lo32,
1795
unsigned int* nbytes_in_hi32,
1796
unsigned int* nbytes_out_lo32,
1797
unsigned int* nbytes_out_hi32 );</pre>
1798
<p>Compresses and flushes to the compressed file all data so
1799
far supplied by <code class="computeroutput">BZ2_bzWrite</code>.
1800
The logical end-of-stream markers are also written, so subsequent
1801
calls to <code class="computeroutput">BZ2_bzWrite</code> are
1802
illegal. All memory associated with the compressed file
1803
<code class="computeroutput">b</code> is released.
1804
<code class="computeroutput">fflush</code> is called on the
1805
compressed file, but it is not
1806
<code class="computeroutput">fclose</code>'d.</p>
1807
<p>If <code class="computeroutput">BZ2_bzWriteClose</code> is
1808
called to clean up after an error, the only action is to release
1809
the memory. The library records the error codes issued by
1810
previous calls, so this situation will be detected automatically.
1811
There is no attempt to complete the compression operation, nor to
1812
<code class="computeroutput">fflush</code> the compressed file. You
1813
can force this behaviour to happen even in the case of no error,
1814
by passing a nonzero value to
1815
<code class="computeroutput">abandon</code>.</p>
1816
<p>If <code class="computeroutput">nbytes_in</code> is non-null,
1817
<code class="computeroutput">*nbytes_in</code> will be set to be the
1818
total volume of uncompressed data handled. Similarly,
1819
<code class="computeroutput">nbytes_out</code> will be set to the
1820
total volume of compressed data written. For compatibility with
1821
older versions of the library,
1822
<code class="computeroutput">BZ2_bzWriteClose</code> only yields the
1823
lower 32 bits of these counts. Use
1824
<code class="computeroutput">BZ2_bzWriteClose64</code> if you want
1825
the full 64 bit counts. These two functions are otherwise
1826
absolutely identical.</p>
1827
<p>Possible assignments to
1828
<code class="computeroutput">bzerror</code>:</p>
1829
<pre class="programlisting">BZ_SEQUENCE_ERROR
1830
if b was opened with BZ2_bzReadOpen
1832
if there is an error writing the compressed file
1836
<div class="sect2" title="3.4.8.�Handling embedded compressed data streams">
1837
<div class="titlepage"><div><div><h3 class="title">
1838
<a name="embed"></a>3.4.8.�Handling embedded compressed data streams</h3></div></div></div>
1839
<p>The high-level library facilitates use of
1840
<code class="computeroutput">bzip2</code> data streams which form
1841
some part of a surrounding, larger data stream.</p>
1842
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
1843
<li class="listitem" style="list-style-type: disc"><p>For writing, the library takes an open file handle,
1844
writes compressed data to it,
1845
<code class="computeroutput">fflush</code>es it but does not
1846
<code class="computeroutput">fclose</code> it. The calling
1847
application can write its own data before and after the
1848
compressed data stream, using that same file handle.</p></li>
1849
<li class="listitem" style="list-style-type: disc"><p>Reading is more complex, and the facilities are not as
1850
general as they could be since generality is hard to reconcile
1851
with efficiency. <code class="computeroutput">BZ2_bzRead</code>
1852
reads from the compressed file in blocks of size
1853
<code class="computeroutput">BZ_MAX_UNUSED</code> bytes, and in
1854
doing so probably will overshoot the logical end of compressed
1855
stream. To recover this data once decompression has ended,
1856
call <code class="computeroutput">BZ2_bzReadGetUnused</code> after
1857
the last call of <code class="computeroutput">BZ2_bzRead</code>
1859
<code class="computeroutput">BZ_STREAM_END</code>) but before
1861
<code class="computeroutput">BZ2_bzReadClose</code>.</p></li>
1863
<p>This mechanism makes it easy to decompress multiple
1864
<code class="computeroutput">bzip2</code> streams placed end-to-end.
1865
As the end of one stream, when
1866
<code class="computeroutput">BZ2_bzRead</code> returns
1867
<code class="computeroutput">BZ_STREAM_END</code>, call
1868
<code class="computeroutput">BZ2_bzReadGetUnused</code> to collect
1869
the unused data (copy it into your own buffer somewhere). That
1870
data forms the start of the next compressed stream. To start
1871
uncompressing that next stream, call
1872
<code class="computeroutput">BZ2_bzReadOpen</code> again, feeding in
1873
the unused data via the <code class="computeroutput">unused</code> /
1874
<code class="computeroutput">nUnused</code> parameters. Keep doing
1875
this until <code class="computeroutput">BZ_STREAM_END</code> return
1876
coincides with the physical end of file
1877
(<code class="computeroutput">feof(f)</code>). In this situation
1878
<code class="computeroutput">BZ2_bzReadGetUnused</code> will of
1879
course return no data.</p>
1880
<p>This should give some feel for how the high-level interface
1881
can be used. If you require extra flexibility, you'll have to
1882
bite the bullet and get to grips with the low-level
1885
<div class="sect2" title="3.4.9.�Standard file-reading/writing code">
1886
<div class="titlepage"><div><div><h3 class="title">
1887
<a name="std-rdwr"></a>3.4.9.�Standard file-reading/writing code</h3></div></div></div>
1888
<p>Here's how you'd write data to a compressed file:</p>
1889
<pre class="programlisting">FILE* f;
1892
char buf[ /* whatever size you like */ ];
1896
f = fopen ( "myfile.bz2", "w" );
1900
b = BZ2_bzWriteOpen( &bzerror, f, 9 );
1901
if (bzerror != BZ_OK) {
1902
BZ2_bzWriteClose ( b );
1906
while ( /* condition */ ) {
1907
/* get data to write into buf, and set nBuf appropriately */
1908
nWritten = BZ2_bzWrite ( &bzerror, b, buf, nBuf );
1909
if (bzerror == BZ_IO_ERROR) {
1910
BZ2_bzWriteClose ( &bzerror, b );
1915
BZ2_bzWriteClose( &bzerror, b );
1916
if (bzerror == BZ_IO_ERROR) {
1919
<p>And to read from a compressed file:</p>
1920
<pre class="programlisting">FILE* f;
1923
char buf[ /* whatever size you like */ ];
1927
f = fopen ( "myfile.bz2", "r" );
1931
b = BZ2_bzReadOpen ( &bzerror, f, 0, NULL, 0 );
1932
if ( bzerror != BZ_OK ) {
1933
BZ2_bzReadClose ( &bzerror, b );
1938
while ( bzerror == BZ_OK && /* arbitrary other conditions */) {
1939
nBuf = BZ2_bzRead ( &bzerror, b, buf, /* size of buf */ );
1940
if ( bzerror == BZ_OK ) {
1941
/* do something with buf[0 .. nBuf-1] */
1944
if ( bzerror != BZ_STREAM_END ) {
1945
BZ2_bzReadClose ( &bzerror, b );
1948
BZ2_bzReadClose ( &bzerror, b );
1952
<div class="sect1" title="3.5.�Utility functions">
1953
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
1954
<a name="util-fns"></a>3.5.�Utility functions</h2></div></div></div>
1955
<div class="sect2" title="3.5.1.�BZ2_bzBuffToBuffCompress">
1956
<div class="titlepage"><div><div><h3 class="title">
1957
<a name="bzbufftobuffcompress"></a>3.5.1.�BZ2_bzBuffToBuffCompress</h3></div></div></div>
1958
<pre class="programlisting">int BZ2_bzBuffToBuffCompress( char* dest,
1959
unsigned int* destLen,
1961
unsigned int sourceLen,
1964
int workFactor );</pre>
1965
<p>Attempts to compress the data in <code class="computeroutput">source[0
1966
.. sourceLen-1]</code> into the destination buffer,
1967
<code class="computeroutput">dest[0 .. *destLen-1]</code>. If the
1968
destination buffer is big enough,
1969
<code class="computeroutput">*destLen</code> is set to the size of
1970
the compressed data, and <code class="computeroutput">BZ_OK</code>
1971
is returned. If the compressed data won't fit,
1972
<code class="computeroutput">*destLen</code> is unchanged, and
1973
<code class="computeroutput">BZ_OUTBUFF_FULL</code> is
1975
<p>Compression in this manner is a one-shot event, done with a
1976
single call to this function. The resulting compressed data is a
1977
complete <code class="computeroutput">bzip2</code> format data
1978
stream. There is no mechanism for making additional calls to
1979
provide extra input data. If you want that kind of mechanism,
1980
use the low-level interface.</p>
1981
<p>For the meaning of parameters
1982
<code class="computeroutput">blockSize100k</code>,
1983
<code class="computeroutput">verbosity</code> and
1984
<code class="computeroutput">workFactor</code>, see
1985
<code class="computeroutput">BZ2_bzCompressInit</code>.</p>
1986
<p>To guarantee that the compressed data will fit in its
1987
buffer, allocate an output buffer of size 1% larger than the
1988
uncompressed data, plus six hundred extra bytes.</p>
1989
<p><code class="computeroutput">BZ2_bzBuffToBuffDecompress</code>
1990
will not write data at or beyond
1991
<code class="computeroutput">dest[*destLen]</code>, even in case of
1992
buffer overflow.</p>
1993
<p>Possible return values:</p>
1994
<pre class="programlisting">BZ_CONFIG_ERROR
1995
if the library has been mis-compiled
1997
if dest is NULL or destLen is NULL
1998
or blockSize100k < 1 or blockSize100k > 9
1999
or verbosity < 0 or verbosity > 4
2000
or workFactor < 0 or workFactor > 250
2002
if insufficient memory is available
2004
if the size of the compressed data exceeds *destLen
2008
<div class="sect2" title="3.5.2.�BZ2_bzBuffToBuffDecompress">
2009
<div class="titlepage"><div><div><h3 class="title">
2010
<a name="bzbufftobuffdecompress"></a>3.5.2.�BZ2_bzBuffToBuffDecompress</h3></div></div></div>
2011
<pre class="programlisting">int BZ2_bzBuffToBuffDecompress( char* dest,
2012
unsigned int* destLen,
2014
unsigned int sourceLen,
2016
int verbosity );</pre>
2017
<p>Attempts to decompress the data in <code class="computeroutput">source[0
2018
.. sourceLen-1]</code> into the destination buffer,
2019
<code class="computeroutput">dest[0 .. *destLen-1]</code>. If the
2020
destination buffer is big enough,
2021
<code class="computeroutput">*destLen</code> is set to the size of
2022
the uncompressed data, and <code class="computeroutput">BZ_OK</code>
2023
is returned. If the compressed data won't fit,
2024
<code class="computeroutput">*destLen</code> is unchanged, and
2025
<code class="computeroutput">BZ_OUTBUFF_FULL</code> is
2027
<p><code class="computeroutput">source</code> is assumed to hold
2028
a complete <code class="computeroutput">bzip2</code> format data
2030
<code class="computeroutput">BZ2_bzBuffToBuffDecompress</code> tries
2031
to decompress the entirety of the stream into the output
2033
<p>For the meaning of parameters
2034
<code class="computeroutput">small</code> and
2035
<code class="computeroutput">verbosity</code>, see
2036
<code class="computeroutput">BZ2_bzDecompressInit</code>.</p>
2037
<p>Because the compression ratio of the compressed data cannot
2038
be known in advance, there is no easy way to guarantee that the
2039
output buffer will be big enough. You may of course make
2040
arrangements in your code to record the size of the uncompressed
2041
data, but such a mechanism is beyond the scope of this
2043
<p><code class="computeroutput">BZ2_bzBuffToBuffDecompress</code>
2044
will not write data at or beyond
2045
<code class="computeroutput">dest[*destLen]</code>, even in case of
2046
buffer overflow.</p>
2047
<p>Possible return values:</p>
2048
<pre class="programlisting">BZ_CONFIG_ERROR
2049
if the library has been mis-compiled
2051
if dest is NULL or destLen is NULL
2052
or small != 0 && small != 1
2053
or verbosity < 0 or verbosity > 4
2055
if insufficient memory is available
2057
if the size of the compressed data exceeds *destLen
2059
if a data integrity error was detected in the compressed data
2061
if the compressed data doesn't begin with the right magic bytes
2063
if the compressed data ends unexpectedly
2068
<div class="sect1" title="3.6.�zlib compatibility functions">
2069
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2070
<a name="zlib-compat"></a>3.6.�zlib compatibility functions</h2></div></div></div>
2071
<p>Yoshioka Tsuneo has contributed some functions to give
2072
better <code class="computeroutput">zlib</code> compatibility.
2073
These functions are <code class="computeroutput">BZ2_bzopen</code>,
2074
<code class="computeroutput">BZ2_bzread</code>,
2075
<code class="computeroutput">BZ2_bzwrite</code>,
2076
<code class="computeroutput">BZ2_bzflush</code>,
2077
<code class="computeroutput">BZ2_bzclose</code>,
2078
<code class="computeroutput">BZ2_bzerror</code> and
2079
<code class="computeroutput">BZ2_bzlibVersion</code>. These
2080
functions are not (yet) officially part of the library. If they
2081
break, you get to keep all the pieces. Nevertheless, I think
2083
<pre class="programlisting">typedef void BZFILE;
2085
const char * BZ2_bzlibVersion ( void );</pre>
2086
<p>Returns a string indicating the library version.</p>
2087
<pre class="programlisting">BZFILE * BZ2_bzopen ( const char *path, const char *mode );
2088
BZFILE * BZ2_bzdopen ( int fd, const char *mode );</pre>
2089
<p>Opens a <code class="computeroutput">.bz2</code> file for
2090
reading or writing, using either its name or a pre-existing file
2091
descriptor. Analogous to <code class="computeroutput">fopen</code>
2092
and <code class="computeroutput">fdopen</code>.</p>
2093
<pre class="programlisting">int BZ2_bzread ( BZFILE* b, void* buf, int len );
2094
int BZ2_bzwrite ( BZFILE* b, void* buf, int len );</pre>
2095
<p>Reads/writes data from/to a previously opened
2096
<code class="computeroutput">BZFILE</code>. Analogous to
2097
<code class="computeroutput">fread</code> and
2098
<code class="computeroutput">fwrite</code>.</p>
2099
<pre class="programlisting">int BZ2_bzflush ( BZFILE* b );
2100
void BZ2_bzclose ( BZFILE* b );</pre>
2101
<p>Flushes/closes a <code class="computeroutput">BZFILE</code>.
2102
<code class="computeroutput">BZ2_bzflush</code> doesn't actually do
2103
anything. Analogous to <code class="computeroutput">fflush</code>
2104
and <code class="computeroutput">fclose</code>.</p>
2105
<pre class="programlisting">const char * BZ2_bzerror ( BZFILE *b, int *errnum )</pre>
2106
<p>Returns a string describing the more recent error status of
2107
<code class="computeroutput">b</code>, and also sets
2108
<code class="computeroutput">*errnum</code> to its numerical
2111
<div class="sect1" title="3.7.�Using the library in a stdio-free environment">
2112
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2113
<a name="stdio-free"></a>3.7.�Using the library in a stdio-free environment</h2></div></div></div>
2114
<div class="sect2" title="3.7.1.�Getting rid of stdio">
2115
<div class="titlepage"><div><div><h3 class="title">
2116
<a name="stdio-bye"></a>3.7.1.�Getting rid of stdio</h3></div></div></div>
2117
<p>In a deeply embedded application, you might want to use
2118
just the memory-to-memory functions. You can do this
2119
conveniently by compiling the library with preprocessor symbol
2120
<code class="computeroutput">BZ_NO_STDIO</code> defined. Doing this
2121
gives you a library containing only the following eight
2123
<p><code class="computeroutput">BZ2_bzCompressInit</code>,
2124
<code class="computeroutput">BZ2_bzCompress</code>,
2125
<code class="computeroutput">BZ2_bzCompressEnd</code>
2126
<code class="computeroutput">BZ2_bzDecompressInit</code>,
2127
<code class="computeroutput">BZ2_bzDecompress</code>,
2128
<code class="computeroutput">BZ2_bzDecompressEnd</code>
2129
<code class="computeroutput">BZ2_bzBuffToBuffCompress</code>,
2130
<code class="computeroutput">BZ2_bzBuffToBuffDecompress</code></p>
2131
<p>When compiled like this, all functions will ignore
2132
<code class="computeroutput">verbosity</code> settings.</p>
2134
<div class="sect2" title="3.7.2.�Critical error handling">
2135
<div class="titlepage"><div><div><h3 class="title">
2136
<a name="critical-error"></a>3.7.2.�Critical error handling</h3></div></div></div>
2137
<p><code class="computeroutput">libbzip2</code> contains a number
2138
of internal assertion checks which should, needless to say, never
2139
be activated. Nevertheless, if an assertion should fail,
2140
behaviour depends on whether or not the library was compiled with
2141
<code class="computeroutput">BZ_NO_STDIO</code> set.</p>
2142
<p>For a normal compile, an assertion failure yields the
2144
<div class="blockquote"><blockquote class="blockquote">
2145
<p>bzip2/libbzip2: internal error number N.</p>
2146
<p>This is a bug in bzip2/libbzip2, 1.0.6 of 6 September 2010.
2147
Please report it to me at: jseward@bzip.org. If this happened
2148
when you were using some program which uses libbzip2 as a
2149
component, you should also report this bug to the author(s)
2150
of that program. Please make an effort to report this bug;
2151
timely and accurate bug reports eventually lead to higher
2152
quality software. Thanks. Julian Seward, 6 September 2010.
2155
<p>where <code class="computeroutput">N</code> is some error code
2156
number. If <code class="computeroutput">N == 1007</code>, it also
2157
prints some extra text advising the reader that unreliable memory
2158
is often associated with internal error 1007. (This is a
2159
frequently-observed-phenomenon with versions 1.0.0/1.0.1).</p>
2160
<p><code class="computeroutput">exit(3)</code> is then
2162
<p>For a <code class="computeroutput">stdio</code>-free library,
2163
assertion failures result in a call to a function declared
2165
<pre class="programlisting">extern void bz_internal_error ( int errcode );</pre>
2166
<p>The relevant code is passed as a parameter. You should
2167
supply such a function.</p>
2168
<p>In either case, once an assertion failure has occurred, any
2169
<code class="computeroutput">bz_stream</code> records involved can
2170
be regarded as invalid. You should not attempt to resume normal
2171
operation with them.</p>
2172
<p>You may, of course, change critical error handling to suit
2173
your needs. As I said above, critical errors indicate bugs in
2174
the library and should not occur. All "normal" error situations
2175
are indicated via error return codes from functions, and can be
2179
<div class="sect1" title="3.8.�Making a Windows DLL">
2180
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2181
<a name="win-dll"></a>3.8.�Making a Windows DLL</h2></div></div></div>
2182
<p>Everything related to Windows has been contributed by
2184
(<code class="computeroutput">tsuneo@rr.iij4u.or.jp</code>), so
2185
you should send your queries to him (but perhaps Cc: me,
2186
<code class="computeroutput">jseward@bzip.org</code>).</p>
2187
<p>My vague understanding of what to do is: using Visual C++
2188
5.0, open the project file
2189
<code class="computeroutput">libbz2.dsp</code>, and build. That's
2191
<p>If you can't open the project file for some reason, make a
2192
new one, naming these files:
2193
<code class="computeroutput">blocksort.c</code>,
2194
<code class="computeroutput">bzlib.c</code>,
2195
<code class="computeroutput">compress.c</code>,
2196
<code class="computeroutput">crctable.c</code>,
2197
<code class="computeroutput">decompress.c</code>,
2198
<code class="computeroutput">huffman.c</code>,
2199
<code class="computeroutput">randtable.c</code> and
2200
<code class="computeroutput">libbz2.def</code>. You will also need
2201
to name the header files <code class="computeroutput">bzlib.h</code>
2202
and <code class="computeroutput">bzlib_private.h</code>.</p>
2203
<p>If you don't use VC++, you may need to define the
2205
<code class="computeroutput">_WIN32</code>.</p>
2206
<p>Finally, <code class="computeroutput">dlltest.c</code> is a
2207
sample program using the DLL. It has a project file,
2208
<code class="computeroutput">dlltest.dsp</code>.</p>
2209
<p>If you just want a makefile for Visual C, have a look at
2210
<code class="computeroutput">makefile.msc</code>.</p>
2211
<p>Be aware that if you compile
2212
<code class="computeroutput">bzip2</code> itself on Win32, you must
2213
set <code class="computeroutput">BZ_UNIX</code> to 0 and
2214
<code class="computeroutput">BZ_LCCWIN32</code> to 1, in the file
2215
<code class="computeroutput">bzip2.c</code>, before compiling.
2216
Otherwise the resulting binary won't work correctly.</p>
2217
<p>I haven't tried any of this stuff myself, but it all looks
2221
<div class="chapter" title="4.�Miscellanea">
2222
<div class="titlepage"><div><div><h2 class="title">
2223
<a name="misc"></a>4.�Miscellanea</h2></div></div></div>
2225
<p><b>Table of Contents</b></p>
2227
<dt><span class="sect1"><a href="#limits">4.1. Limitations of the compressed file format</a></span></dt>
2228
<dt><span class="sect1"><a href="#port-issues">4.2. Portability issues</a></span></dt>
2229
<dt><span class="sect1"><a href="#bugs">4.3. Reporting bugs</a></span></dt>
2230
<dt><span class="sect1"><a href="#package">4.4. Did you get the right package?</a></span></dt>
2231
<dt><span class="sect1"><a href="#reading">4.5. Further Reading</a></span></dt>
2234
<p>These are just some random thoughts of mine. Your mileage
2236
<div class="sect1" title="4.1.�Limitations of the compressed file format">
2237
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2238
<a name="limits"></a>4.1.�Limitations of the compressed file format</h2></div></div></div>
2239
<p><code class="computeroutput">bzip2-1.0.X</code>,
2240
<code class="computeroutput">0.9.5</code> and
2241
<code class="computeroutput">0.9.0</code> use exactly the same file
2242
format as the original version,
2243
<code class="computeroutput">bzip2-0.1</code>. This decision was
2244
made in the interests of stability. Creating yet another
2245
incompatible compressed file format would create further
2246
confusion and disruption for users.</p>
2247
<p>Nevertheless, this is not a painless decision. Development
2248
work since the release of
2249
<code class="computeroutput">bzip2-0.1</code> in August 1997 has
2250
shown complexities in the file format which slow down
2251
decompression and, in retrospect, are unnecessary. These
2253
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
2254
<li class="listitem" style="list-style-type: disc"><p>The run-length encoder, which is the first of the
2255
compression transformations, is entirely irrelevant. The
2256
original purpose was to protect the sorting algorithm from the
2257
very worst case input: a string of repeated symbols. But
2258
algorithm steps Q6a and Q6b in the original Burrows-Wheeler
2259
technical report (SRC-124) show how repeats can be handled
2260
without difficulty in block sorting.</p></li>
2261
<li class="listitem" style="list-style-type: disc">
2262
<p>The randomisation mechanism doesn't really need to be
2263
there. Udi Manber and Gene Myers published a suffix array
2264
construction algorithm a few years back, which can be employed
2265
to sort any block, no matter how repetitive, in O(N log N)
2266
time. Subsequent work by Kunihiko Sadakane has produced a
2267
derivative O(N (log N)^2) algorithm which usually outperforms
2268
the Manber-Myers algorithm.</p>
2269
<p>I could have changed to Sadakane's algorithm, but I find
2270
it to be slower than <code class="computeroutput">bzip2</code>'s
2271
existing algorithm for most inputs, and the randomisation
2272
mechanism protects adequately against bad cases. I didn't
2273
think it was a good tradeoff to make. Partly this is due to
2274
the fact that I was not flooded with email complaints about
2275
<code class="computeroutput">bzip2-0.1</code>'s performance on
2276
repetitive data, so perhaps it isn't a problem for real
2278
<p>Probably the best long-term solution, and the one I have
2279
incorporated into 0.9.5 and above, is to use the existing
2280
sorting algorithm initially, and fall back to a O(N (log N)^2)
2281
algorithm if the standard algorithm gets into
2284
<li class="listitem" style="list-style-type: disc"><p>The compressed file format was never designed to be
2285
handled by a library, and I have had to jump though some hoops
2286
to produce an efficient implementation of decompression. It's
2287
a bit hairy. Try passing
2288
<code class="computeroutput">decompress.c</code> through the C
2289
preprocessor and you'll see what I mean. Much of this
2290
complexity could have been avoided if the compressed size of
2291
each block of data was recorded in the data stream.</p></li>
2292
<li class="listitem" style="list-style-type: disc"><p>An Adler-32 checksum, rather than a CRC32 checksum,
2293
would be faster to compute.</p></li>
2295
<p>It would be fair to say that the
2296
<code class="computeroutput">bzip2</code> format was frozen before I
2297
properly and fully understood the performance consequences of
2299
<p>Improvements which I was able to incorporate into 0.9.0,
2300
despite using the same file format, are:</p>
2301
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
2302
<li class="listitem" style="list-style-type: disc"><p>Single array implementation of the inverse BWT. This
2303
significantly speeds up decompression, presumably because it
2304
reduces the number of cache misses.</p></li>
2305
<li class="listitem" style="list-style-type: disc"><p>Faster inverse MTF transform for large MTF values.
2306
The new implementation is based on the notion of sliding blocks
2308
<li class="listitem" style="list-style-type: disc"><p><code class="computeroutput">bzip2-0.9.0</code> now reads
2309
and writes files with <code class="computeroutput">fread</code>
2310
and <code class="computeroutput">fwrite</code>; version 0.1 used
2311
<code class="computeroutput">putc</code> and
2312
<code class="computeroutput">getc</code>. Duh! Well, you live
2315
<p>Further ahead, it would be nice to be able to do random
2316
access into files. This will require some careful design of
2317
compressed file formats.</p>
2319
<div class="sect1" title="4.2.�Portability issues">
2320
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2321
<a name="port-issues"></a>4.2.�Portability issues</h2></div></div></div>
2322
<p>After some consideration, I have decided not to use GNU
2323
<code class="computeroutput">autoconf</code> to configure 0.9.5 or
2325
<p><code class="computeroutput">autoconf</code>, admirable and
2326
wonderful though it is, mainly assists with portability problems
2327
between Unix-like platforms. But
2328
<code class="computeroutput">bzip2</code> doesn't have much in the
2329
way of portability problems on Unix; most of the difficulties
2330
appear when porting to the Mac, or to Microsoft's operating
2331
systems. <code class="computeroutput">autoconf</code> doesn't help
2332
in those cases, and brings in a whole load of new
2334
<p>Most people should be able to compile the library and
2335
program under Unix straight out-of-the-box, so to speak,
2336
especially if you have a version of GNU C available.</p>
2337
<p>There are a couple of
2338
<code class="computeroutput">__inline__</code> directives in the
2339
code. GNU C (<code class="computeroutput">gcc</code>) should be
2340
able to handle them. If you're not using GNU C, your C compiler
2341
shouldn't see them at all. If your compiler does, for some
2342
reason, see them and doesn't like them, just
2343
<code class="computeroutput">#define</code>
2344
<code class="computeroutput">__inline__</code> to be
2345
<code class="computeroutput">/* */</code>. One easy way to do this
2346
is to compile with the flag
2347
<code class="computeroutput">-D__inline__=</code>, which should be
2348
understood by most Unix compilers.</p>
2349
<p>If you still have difficulties, try compiling with the
2350
macro <code class="computeroutput">BZ_STRICT_ANSI</code> defined.
2351
This should enable you to build the library in a strictly ANSI
2352
compliant environment. Building the program itself like this is
2353
dangerous and not supported, since you remove
2354
<code class="computeroutput">bzip2</code>'s checks against
2355
compressing directories, symbolic links, devices, and other
2356
not-really-a-file entities. This could cause filesystem
2358
<p>One other thing: if you create a
2359
<code class="computeroutput">bzip2</code> binary for public distribution,
2360
please consider linking it statically (<code class="computeroutput">gcc
2361
-static</code>). This avoids all sorts of library-version
2362
issues that others may encounter later on.</p>
2363
<p>If you build <code class="computeroutput">bzip2</code> on
2364
Win32, you must set <code class="computeroutput">BZ_UNIX</code> to 0
2365
and <code class="computeroutput">BZ_LCCWIN32</code> to 1, in the
2366
file <code class="computeroutput">bzip2.c</code>, before compiling.
2367
Otherwise the resulting binary won't work correctly.</p>
2369
<div class="sect1" title="4.3.�Reporting bugs">
2370
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2371
<a name="bugs"></a>4.3.�Reporting bugs</h2></div></div></div>
2372
<p>I tried pretty hard to make sure
2373
<code class="computeroutput">bzip2</code> is bug free, both by
2374
design and by testing. Hopefully you'll never need to read this
2375
section for real.</p>
2376
<p>Nevertheless, if <code class="computeroutput">bzip2</code> dies
2377
with a segmentation fault, a bus error or an internal assertion
2378
failure, it will ask you to email me a bug report. Experience from
2379
years of feedback of bzip2 users indicates that almost all these
2380
problems can be traced to either compiler bugs or hardware
2382
<div class="itemizedlist"><ul class="itemizedlist" type="bullet">
2383
<li class="listitem" style="list-style-type: disc">
2384
<p>Recompile the program with no optimisation, and
2385
see if it works. And/or try a different compiler. I heard all
2386
sorts of stories about various flavours of GNU C (and other
2387
compilers) generating bad code for
2388
<code class="computeroutput">bzip2</code>, and I've run across two
2389
such examples myself.</p>
2390
<p>2.7.X versions of GNU C are known to generate bad code
2391
from time to time, at high optimisation levels. If you get
2392
problems, try using the flags
2393
<code class="computeroutput">-O2</code>
2394
<code class="computeroutput">-fomit-frame-pointer</code>
2395
<code class="computeroutput">-fno-strength-reduce</code>. You
2396
should specifically <span class="emphasis"><em>not</em></span> use
2397
<code class="computeroutput">-funroll-loops</code>.</p>
2398
<p>You may notice that the Makefile runs six tests as part
2399
of the build process. If the program passes all of these, it's
2400
a pretty good (but not 100%) indication that the compiler has
2401
done its job correctly.</p>
2403
<li class="listitem" style="list-style-type: disc">
2404
<p>If <code class="computeroutput">bzip2</code>
2405
crashes randomly, and the crashes are not repeatable, you may
2406
have a flaky memory subsystem.
2407
<code class="computeroutput">bzip2</code> really hammers your
2408
memory hierarchy, and if it's a bit marginal, you may get these
2409
problems. Ditto if your disk or I/O subsystem is slowly
2410
failing. Yup, this really does happen.</p>
2411
<p>Try using a different machine of the same type, and see
2412
if you can repeat the problem.</p>
2414
<li class="listitem" style="list-style-type: disc"><p>This isn't really a bug, but ... If
2415
<code class="computeroutput">bzip2</code> tells you your file is
2416
corrupted on decompression, and you obtained the file via FTP,
2417
there is a possibility that you forgot to tell FTP to do a
2418
binary mode transfer. That absolutely will cause the file to
2419
be non-decompressible. You'll have to transfer it
2422
<p>If you've incorporated
2423
<code class="computeroutput">libbzip2</code> into your own program
2424
and are getting problems, please, please, please, check that the
2425
parameters you are passing in calls to the library, are correct,
2426
and in accordance with what the documentation says is allowable.
2427
I have tried to make the library robust against such problems,
2428
but I'm sure I haven't succeeded.</p>
2429
<p>Finally, if the above comments don't help, you'll have to
2430
send me a bug report. Now, it's just amazing how many people
2431
will send me a bug report saying something like:</p>
2432
<pre class="programlisting">bzip2 crashed with segmentation fault on my machine</pre>
2433
<p>and absolutely nothing else. Needless to say, a such a
2434
report is <span class="emphasis"><em>totally, utterly, completely and
2435
comprehensively 100% useless; a waste of your time, my time, and
2436
net bandwidth</em></span>. With no details at all, there's no way
2437
I can possibly begin to figure out what the problem is.</p>
2438
<p>The rules of the game are: facts, facts, facts. Don't omit
2439
them because "oh, they won't be relevant". At the bare
2441
<pre class="programlisting">Machine type. Operating system version.
2442
Exact version of bzip2 (do bzip2 -V).
2443
Exact version of the compiler used.
2444
Flags passed to the compiler.</pre>
2445
<p>However, the most important single thing that will help me
2446
is the file that you were trying to compress or decompress at the
2447
time the problem happened. Without that, my ability to do
2448
anything more than speculate about the cause, is limited.</p>
2450
<div class="sect1" title="4.4.�Did you get the right package?">
2451
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2452
<a name="package"></a>4.4.�Did you get the right package?</h2></div></div></div>
2453
<p><code class="computeroutput">bzip2</code> is a resource hog.
2454
It soaks up large amounts of CPU cycles and memory. Also, it
2455
gives very large latencies. In the worst case, you can feed many
2456
megabytes of uncompressed data into the library before getting
2457
any compressed output, so this probably rules out applications
2458
requiring interactive behaviour.</p>
2459
<p>These aren't faults of my implementation, I hope, but more
2460
an intrinsic property of the Burrows-Wheeler transform
2461
(unfortunately). Maybe this isn't what you want.</p>
2462
<p>If you want a compressor and/or library which is faster,
2463
uses less memory but gets pretty good compression, and has
2464
minimal latency, consider Jean-loup Gailly's and Mark Adler's
2465
work, <code class="computeroutput">zlib-1.2.1</code> and
2466
<code class="computeroutput">gzip-1.2.4</code>. Look for them at
2467
<a class="ulink" href="http://www.zlib.org" target="_top">http://www.zlib.org</a> and
2468
<a class="ulink" href="http://www.gzip.org" target="_top">http://www.gzip.org</a>
2470
<p>For something faster and lighter still, you might try Markus F
2471
X J Oberhumer's <code class="computeroutput">LZO</code> real-time
2472
compression/decompression library, at
2473
<a class="ulink" href="http://www.oberhumer.com/opensource" target="_top">http://www.oberhumer.com/opensource</a>.</p>
2475
<div class="sect1" title="4.5.�Further Reading">
2476
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
2477
<a name="reading"></a>4.5.�Further Reading</h2></div></div></div>
2478
<p><code class="computeroutput">bzip2</code> is not research
2479
work, in the sense that it doesn't present any new ideas.
2480
Rather, it's an engineering exercise based on existing
2482
<p>Four documents describe essentially all the ideas behind
2483
<code class="computeroutput">bzip2</code>:</p>
2484
<div class="literallayout"><p>Michael�Burrows�and�D.�J.�Wheeler:<br>
2485
��"A�block-sorting�lossless�data�compression�algorithm"<br>
2486
���10th�May�1994.�<br>
2487
���Digital�SRC�Research�Report�124.<br>
2488
���ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.gz<br>
2489
���If�you�have�trouble�finding�it,�try�searching�at�the<br>
2490
���New�Zealand�Digital�Library,�http://www.nzdl.org.<br>
2492
Daniel�S.�Hirschberg�and�Debra�A.�LeLewer<br>
2493
��"Efficient�Decoding�of�Prefix�Codes"<br>
2494
���Communications�of�the�ACM,�April�1990,�Vol�33,�Number�4.<br>
2495
���You�might�be�able�to�get�an�electronic�copy�of�this<br>
2496
���from�the�ACM�Digital�Library.<br>
2498
David�J.�Wheeler<br>
2499
���Program�bred3.c�and�accompanying�document�bred3.ps.<br>
2500
���This�contains�the�idea�behind�the�multi-table�Huffman�coding�scheme.<br>
2501
���ftp://ftp.cl.cam.ac.uk/users/djw3/<br>
2503
Jon�L.�Bentley�and�Robert�Sedgewick<br>
2504
��"Fast�Algorithms�for�Sorting�and�Searching�Strings"<br>
2505
���Available�from�Sedgewick's�web�page,<br>
2506
���www.cs.princeton.edu/~rs<br>
2508
<p>The following paper gives valuable additional insights into
2509
the algorithm, but is not immediately the basis of any code used
2511
<div class="literallayout"><p>Peter�Fenwick:<br>
2512
���Block�Sorting�Text�Compression<br>
2513
���Proceedings�of�the�19th�Australasian�Computer�Science�Conference,<br>
2514
�����Melbourne,�Australia.��Jan�31�-�Feb�2,�1996.<br>
2515
���ftp://ftp.cs.auckland.ac.nz/pub/peter-f/ACSC96paper.ps</p></div>
2516
<p>Kunihiko Sadakane's sorting algorithm, mentioned above, is
2518
<div class="literallayout"><p>http://naomi.is.s.u-tokyo.ac.jp/~sada/papers/Sada98b.ps.gz<br>
2520
<p>The Manber-Myers suffix array construction algorithm is
2521
described in a paper available from:</p>
2522
<div class="literallayout"><p>http://www.cs.arizona.edu/people/gene/PAPERS/suffix.ps<br>
2524
<p>Finally, the following papers document some
2525
investigations I made into the performance of sorting
2526
and decompression algorithms:</p>
2527
<div class="literallayout"><p>Julian�Seward<br>
2528
���On�the�Performance�of�BWT�Sorting�Algorithms<br>
2529
���Proceedings�of�the�IEEE�Data�Compression�Conference�2000<br>
2530
�����Snowbird,�Utah.��28-30�March�2000.<br>
2533
���Space-time�Tradeoffs�in�the�Inverse�B-W�Transform<br>
2534
���Proceedings�of�the�IEEE�Data�Compression�Conference�2001<br>
2535
�����Snowbird,�Utah.��27-29�March�2001.<br>
added
removed
win32/3rdparty/bzip2/manual.html
