~akopytov/percona-xtrabackup/bug1166888-2.1

Viewing changes to src/libarchive/libarchive/test/test_write_format_tar_ustar.c

Committer: Alexey Kopytov
Date: 2012-02-10 20:05:56 UTC
mto: (391.1.5 staging)
mto: This revision was merged to the branch mainline in revision 390.
Revision ID: akopytov@gmail.com-20120210200556-6kx41z8wwrqfucro

Rebase of the parallel compression patch on new trunk + post-review
fixes.

Implementation of parallel compression and streaming for XtraBackup.

This revision implements the following changes:

* InnoDB files are now streamed by the xtrabackup binary rather than
innobackupex. As a result, integrity is now verified by xtrabackup and
thus tar4ibd is no longer needed, so it was removed.

* xtrabackup binary now accepts the new '--stream' option which has
exactly the same semantics as the '--stream' option in
innobackupex: it tells xtrabackup to stream all files to the standard
output in the specified format rather than storing them locally.

* The xtrabackup binary can now do parallel compression using the
quicklz library. Two new options were added to xtrabackup to support
this feature:

- '--compress' tells xtrabackup to compress all output data, including
the transaction log file and meta data files, using the specified
compression algorithm. The only currently supported algorithm is
'quicklz'. The resulting files have the qpress archive format,
i.e. every *.qp file produced by xtrabackup is essentially a one-file
qpress archive and can be extracted and uncompressed by the qpress
file archiver (http://www.quicklz.com/).

- '--compress-threads' specifies the number of worker threads used by
xtrabackup for parallel data compression. This option defaults to 1.

Parallel compression ('--compress-threads') can be used together with
parallel file copying ('--parallel'). For example, '--parallel=4
--compress --compress-threads=2' will create 4 IO threads that will
read the data and pipe it to 2 compression threads.

* To support simultaneous compression and streaming, a new custom
streaming format called 'xbstream' was introduced to XtraBackup in
addition to the 'tar' format. That was required to overcome some
limitations of traditional archive formats such as 'tar', 'cpio' and
others that do not allow streaming dynamically generated files, for
example dynamically compressed files. Other advantages of xbstream over
traditional streaming/archive formats include ability to stream multiple
files concurrently (so it is possible to use streaming in the xbstream
format together with the --parallel option) and more compact data
storage.

* To allow streaming and extracting files to/from the xbstream format
produced by xtrabackup, a new utility aptly called 'xbstream' was
added to the XtraBackup distribution. This utility has a tar-like
interface:

- with the '-x' option it extracts files from the stream read from its
standard input to the current directory unless specified otherwise
with the '-C' option.

- with the '-c' option it streams files specified on the command line
to its standard output.

The utility also tries to minimize its impact on the OS page cache by
using the appropriate posix_fadvise() calls when available.

files added:
src

src/common.h

src/compress.c

src/compress.h

src/datasink.h

src/libarchive

src/libarchive/CMakeLists.txt

src/libarchive/COPYING

src/libarchive/INSTALL

src/libarchive/Makefile.am

src/libarchive/NEWS

src/libarchive/README

src/libarchive/build

src/libarchive/build/autoconf

src/libarchive/build/autoconf/check_stdcall_func.m4

src/libarchive/build/autoconf/compile

src/libarchive/build/autoconf/config.guess

src/libarchive/build/autoconf/config.sub

src/libarchive/build/autoconf/depcomp

src/libarchive/build/autoconf/install-sh

src/libarchive/build/autoconf/la_uid_t.m4

src/libarchive/build/autoconf/ltmain.sh

src/libarchive/build/autoconf/missing

src/libarchive/build/autogen.sh

src/libarchive/build/bump-version.sh

src/libarchive/build/clean.sh

src/libarchive/build/cmake

src/libarchive/build/cmake/AddTest28.cmake

src/libarchive/build/cmake/CheckFileOffsetBits.c

src/libarchive/build/cmake/CheckFileOffsetBits.cmake

src/libarchive/build/cmake/CheckFuncs.cmake

src/libarchive/build/cmake/CheckFuncs_stub.c.in

src/libarchive/build/cmake/CheckHeaderDirent.cmake

src/libarchive/build/cmake/CheckStructMember.cmake

src/libarchive/build/cmake/CheckTypeExists.cmake

src/libarchive/build/cmake/FindLZMA.cmake

src/libarchive/build/cmake/config.h.in

src/libarchive/build/pkgconfig

src/libarchive/build/pkgconfig/libarchive.pc.in

src/libarchive/build/version

src/libarchive/build/windows

src/libarchive/build/windows/mvcpp.nt

src/libarchive/build/windows/vc71

src/libarchive/build/windows/vc71/libarchive.sln

src/libarchive/build/windows/vc71/libarchive.vcproj

src/libarchive/build/windows/vc80

src/libarchive/build/windows/vc80/libarchive.sln

src/libarchive/build/windows/vc80/libarchive.vcproj

src/libarchive/build/windows/vc80/libarchive_test

src/libarchive/build/windows/vc80/libarchive_test/libarchive_test.vcproj

src/libarchive/build/windows/vc90

src/libarchive/build/windows/vc90/libarchive.sln

src/libarchive/build/windows/vc90/libarchive.vcproj

src/libarchive/build/windows/vc90/libarchive_test

src/libarchive/build/windows/vc90/libarchive_test/libarchive_test.vcproj

src/libarchive/build/windows/wccpp.nt

src/libarchive/config.h.in

src/libarchive/configure.ac

src/libarchive/contrib

src/libarchive/contrib/README

src/libarchive/contrib/libarchive.1aix53.spec

src/libarchive/contrib/libarchive.spec

src/libarchive/contrib/libarchive_autodetect-st_lib_archive.m4

src/libarchive/contrib/psota-benchmark

src/libarchive/contrib/psota-benchmark/results.txt

src/libarchive/contrib/psota-benchmark/tcp.sh

src/libarchive/contrib/shar

src/libarchive/contrib/shar/shar.1

src/libarchive/contrib/shar/shar.c

src/libarchive/contrib/shar/tree.c

src/libarchive/contrib/shar/tree.h

src/libarchive/contrib/shar/tree_config.h

src/libarchive/contrib/untar.c

src/libarchive/cpio

src/libarchive/cpio/CMakeLists.txt

src/libarchive/cpio/bsdcpio.1

src/libarchive/cpio/cmdline.c

src/libarchive/cpio/config_freebsd.h

src/libarchive/cpio/cpio.c

src/libarchive/cpio/cpio.h

src/libarchive/cpio/cpio_platform.h

src/libarchive/cpio/cpio_windows.c

src/libarchive/cpio/cpio_windows.h

src/libarchive/cpio/test

src/libarchive/cpio/test/CMakeLists.txt

src/libarchive/cpio/test/main.c

src/libarchive/cpio/test/test.h

src/libarchive/cpio/test/test_0.c

src/libarchive/cpio/test/test_basic.c

src/libarchive/cpio/test/test_cmdline.c

src/libarchive/cpio/test/test_format_newc.c

src/libarchive/cpio/test/test_gcpio_compat.c

src/libarchive/cpio/test/test_gcpio_compat_ref.bin.uu

src/libarchive/cpio/test/test_gcpio_compat_ref.crc.uu

src/libarchive/cpio/test/test_gcpio_compat_ref.newc.uu

src/libarchive/cpio/test/test_gcpio_compat_ref.ustar.uu

src/libarchive/cpio/test/test_gcpio_compat_ref_nosym.bin.uu

src/libarchive/cpio/test/test_gcpio_compat_ref_nosym.crc.uu

src/libarchive/cpio/test/test_gcpio_compat_ref_nosym.newc.uu

src/libarchive/cpio/test/test_gcpio_compat_ref_nosym.ustar.uu

src/libarchive/cpio/test/test_option_B_upper.c

src/libarchive/cpio/test/test_option_C_upper.c

src/libarchive/cpio/test/test_option_J_upper.c

src/libarchive/cpio/test/test_option_L_upper.c

src/libarchive/cpio/test/test_option_Z_upper.c

src/libarchive/cpio/test/test_option_a.c

src/libarchive/cpio/test/test_option_c.c

src/libarchive/cpio/test/test_option_d.c

src/libarchive/cpio/test/test_option_f.c

src/libarchive/cpio/test/test_option_f.cpio.uu

src/libarchive/cpio/test/test_option_help.c

src/libarchive/cpio/test/test_option_l.c

src/libarchive/cpio/test/test_option_lzma.c

src/libarchive/cpio/test/test_option_m.c

src/libarchive/cpio/test/test_option_m.cpio.uu

src/libarchive/cpio/test/test_option_t.c

src/libarchive/cpio/test/test_option_t.cpio.uu

src/libarchive/cpio/test/test_option_t.stdout.uu

src/libarchive/cpio/test/test_option_tv.stdout.uu

src/libarchive/cpio/test/test_option_u.c

src/libarchive/cpio/test/test_option_version.c

src/libarchive/cpio/test/test_option_y.c

src/libarchive/cpio/test/test_option_z.c

src/libarchive/cpio/test/test_owner_parse.c

src/libarchive/cpio/test/test_passthrough_dotdot.c

src/libarchive/cpio/test/test_passthrough_reverse.c

src/libarchive/cpio/test/test_pathmatch.c

src/libarchive/doc

src/libarchive/doc/html

src/libarchive/doc/man

src/libarchive/doc/mdoc2man.awk

src/libarchive/doc/mdoc2wiki.awk

src/libarchive/doc/pdf

src/libarchive/doc/text

src/libarchive/doc/update.sh

src/libarchive/doc/wiki

src/libarchive/examples

src/libarchive/examples/minitar

src/libarchive/examples/minitar/README

src/libarchive/examples/minitar/minitar.c

src/libarchive/examples/minitar/tree.c

src/libarchive/examples/minitar/tree.h

src/libarchive/examples/tarfilter.c

src/libarchive/examples/untar.c

src/libarchive/libarchive

src/libarchive/libarchive/CMakeLists.txt

src/libarchive/libarchive/archive.h

src/libarchive/libarchive/archive_check_magic.c

src/libarchive/libarchive/archive_crc32.h

src/libarchive/libarchive/archive_endian.h

src/libarchive/libarchive/archive_entry.3

src/libarchive/libarchive/archive_entry.c

src/libarchive/libarchive/archive_entry.h

src/libarchive/libarchive/archive_entry_copy_bhfi.c

src/libarchive/libarchive/archive_entry_copy_stat.c

src/libarchive/libarchive/archive_entry_link_resolver.c

src/libarchive/libarchive/archive_entry_private.h

src/libarchive/libarchive/archive_entry_stat.c

src/libarchive/libarchive/archive_entry_strmode.c

src/libarchive/libarchive/archive_entry_xattr.c

src/libarchive/libarchive/archive_hash.h

src/libarchive/libarchive/archive_platform.h

src/libarchive/libarchive/archive_private.h

src/libarchive/libarchive/archive_read.3

src/libarchive/libarchive/archive_read.c

src/libarchive/libarchive/archive_read_data_into_fd.c

src/libarchive/libarchive/archive_read_disk.3

src/libarchive/libarchive/archive_read_disk.c

src/libarchive/libarchive/archive_read_disk_entry_from_file.c

src/libarchive/libarchive/archive_read_disk_private.h

src/libarchive/libarchive/archive_read_disk_set_standard_lookup.c

src/libarchive/libarchive/archive_read_extract.c

src/libarchive/libarchive/archive_read_open_fd.c

src/libarchive/libarchive/archive_read_open_file.c

src/libarchive/libarchive/archive_read_open_filename.c

src/libarchive/libarchive/archive_read_open_memory.c

src/libarchive/libarchive/archive_read_private.h

src/libarchive/libarchive/archive_read_support_compression_all.c

src/libarchive/libarchive/archive_read_support_compression_bzip2.c

src/libarchive/libarchive/archive_read_support_compression_compress.c

src/libarchive/libarchive/archive_read_support_compression_gzip.c

src/libarchive/libarchive/archive_read_support_compression_none.c

src/libarchive/libarchive/archive_read_support_compression_program.c

src/libarchive/libarchive/archive_read_support_compression_rpm.c

src/libarchive/libarchive/archive_read_support_compression_uu.c

src/libarchive/libarchive/archive_read_support_compression_xz.c

src/libarchive/libarchive/archive_read_support_format_all.c

src/libarchive/libarchive/archive_read_support_format_ar.c

src/libarchive/libarchive/archive_read_support_format_cpio.c

src/libarchive/libarchive/archive_read_support_format_empty.c

src/libarchive/libarchive/archive_read_support_format_iso9660.c

src/libarchive/libarchive/archive_read_support_format_mtree.c

src/libarchive/libarchive/archive_read_support_format_raw.c

src/libarchive/libarchive/archive_read_support_format_tar.c

src/libarchive/libarchive/archive_read_support_format_xar.c

src/libarchive/libarchive/archive_read_support_format_zip.c

src/libarchive/libarchive/archive_string.c

src/libarchive/libarchive/archive_string.h

src/libarchive/libarchive/archive_string_sprintf.c

src/libarchive/libarchive/archive_util.3

src/libarchive/libarchive/archive_util.c

src/libarchive/libarchive/archive_virtual.c

src/libarchive/libarchive/archive_windows.c

src/libarchive/libarchive/archive_windows.h

src/libarchive/libarchive/archive_write.3

src/libarchive/libarchive/archive_write.c

src/libarchive/libarchive/archive_write_disk.3

src/libarchive/libarchive/archive_write_disk.c

src/libarchive/libarchive/archive_write_disk_private.h

src/libarchive/libarchive/archive_write_disk_set_standard_lookup.c

src/libarchive/libarchive/archive_write_open_fd.c

src/libarchive/libarchive/archive_write_open_file.c

src/libarchive/libarchive/archive_write_open_filename.c

src/libarchive/libarchive/archive_write_open_memory.c

src/libarchive/libarchive/archive_write_private.h

src/libarchive/libarchive/archive_write_set_compression_bzip2.c

src/libarchive/libarchive/archive_write_set_compression_compress.c

src/libarchive/libarchive/archive_write_set_compression_gzip.c

src/libarchive/libarchive/archive_write_set_compression_none.c

src/libarchive/libarchive/archive_write_set_compression_program.c

src/libarchive/libarchive/archive_write_set_compression_xz.c

src/libarchive/libarchive/archive_write_set_format.c

src/libarchive/libarchive/archive_write_set_format_ar.c

src/libarchive/libarchive/archive_write_set_format_by_name.c

src/libarchive/libarchive/archive_write_set_format_cpio.c

src/libarchive/libarchive/archive_write_set_format_cpio_newc.c

src/libarchive/libarchive/archive_write_set_format_mtree.c

src/libarchive/libarchive/archive_write_set_format_pax.c

src/libarchive/libarchive/archive_write_set_format_shar.c

src/libarchive/libarchive/archive_write_set_format_ustar.c

src/libarchive/libarchive/archive_write_set_format_zip.c

src/libarchive/libarchive/config_freebsd.h

src/libarchive/libarchive/cpio.5

src/libarchive/libarchive/filter_fork.c

src/libarchive/libarchive/filter_fork.h

src/libarchive/libarchive/filter_fork_windows.c

src/libarchive/libarchive/libarchive-formats.5

src/libarchive/libarchive/libarchive.3

src/libarchive/libarchive/libarchive_internals.3

src/libarchive/libarchive/mtree.5

src/libarchive/libarchive/tar.5

src/libarchive/libarchive/test

src/libarchive/libarchive/test/CMakeLists.txt

src/libarchive/libarchive/test/README

src/libarchive/libarchive/test/main.c

src/libarchive/libarchive/test/read_open_memory.c

src/libarchive/libarchive/test/test.h

src/libarchive/libarchive/test/test_acl_basic.c

src/libarchive/libarchive/test/test_acl_freebsd.c

src/libarchive/libarchive/test/test_acl_pax.c

src/libarchive/libarchive/test/test_archive_api_feature.c

src/libarchive/libarchive/test/test_bad_fd.c

src/libarchive/libarchive/test/test_compat_bzip2.c

src/libarchive/libarchive/test/test_compat_bzip2_1.tbz.uu

src/libarchive/libarchive/test/test_compat_bzip2_2.tbz.uu

src/libarchive/libarchive/test/test_compat_cpio.c

src/libarchive/libarchive/test/test_compat_cpio_1.cpio.uu

src/libarchive/libarchive/test/test_compat_gtar.c

src/libarchive/libarchive/test/test_compat_gtar_1.tar.uu

src/libarchive/libarchive/test/test_compat_gzip.c

src/libarchive/libarchive/test/test_compat_gzip_1.tgz.uu

src/libarchive/libarchive/test/test_compat_gzip_2.tgz.uu

src/libarchive/libarchive/test/test_compat_lzma.c

src/libarchive/libarchive/test/test_compat_lzma_1.tlz.uu

src/libarchive/libarchive/test/test_compat_lzma_2.tlz.uu

src/libarchive/libarchive/test/test_compat_lzma_3.tlz.uu

src/libarchive/libarchive/test/test_compat_solaris_tar_acl.c

src/libarchive/libarchive/test/test_compat_solaris_tar_acl.tar.uu

src/libarchive/libarchive/test/test_compat_tar_hardlink.c

src/libarchive/libarchive/test/test_compat_tar_hardlink_1.tar.uu

src/libarchive/libarchive/test/test_compat_xz.c

src/libarchive/libarchive/test/test_compat_xz_1.txz.uu

src/libarchive/libarchive/test/test_compat_zip.c

src/libarchive/libarchive/test/test_compat_zip_1.zip.uu

src/libarchive/libarchive/test/test_empty_write.c

src/libarchive/libarchive/test/test_entry.c

src/libarchive/libarchive/test/test_entry_strmode.c

src/libarchive/libarchive/test/test_extattr_freebsd.c

src/libarchive/libarchive/test/test_fuzz.c

src/libarchive/libarchive/test/test_fuzz_1.iso.Z.uu

src/libarchive/libarchive/test/test_link_resolver.c

src/libarchive/libarchive/test/test_open_fd.c

src/libarchive/libarchive/test/test_open_file.c

src/libarchive/libarchive/test/test_open_filename.c

src/libarchive/libarchive/test/test_pax_filename_encoding.c

src/libarchive/libarchive/test/test_pax_filename_encoding.tar.uu

src/libarchive/libarchive/test/test_read_compress_program.c

src/libarchive/libarchive/test/test_read_data_large.c

src/libarchive/libarchive/test/test_read_disk.c

src/libarchive/libarchive/test/test_read_disk_entry_from_file.c

src/libarchive/libarchive/test/test_read_extract.c

src/libarchive/libarchive/test/test_read_file_nonexistent.c

src/libarchive/libarchive/test/test_read_format_ar.ar.uu

src/libarchive/libarchive/test/test_read_format_ar.c

src/libarchive/libarchive/test/test_read_format_cpio_bin.c

src/libarchive/libarchive/test/test_read_format_cpio_bin_Z.c

src/libarchive/libarchive/test/test_read_format_cpio_bin_be.c

src/libarchive/libarchive/test/test_read_format_cpio_bin_be.cpio.uu

src/libarchive/libarchive/test/test_read_format_cpio_bin_bz2.c

src/libarchive/libarchive/test/test_read_format_cpio_bin_gz.c

src/libarchive/libarchive/test/test_read_format_cpio_bin_lzma.c

src/libarchive/libarchive/test/test_read_format_cpio_bin_xz.c

src/libarchive/libarchive/test/test_read_format_cpio_odc.c

src/libarchive/libarchive/test/test_read_format_cpio_svr4_bzip2_rpm.c

src/libarchive/libarchive/test/test_read_format_cpio_svr4_bzip2_rpm.rpm.uu

src/libarchive/libarchive/test/test_read_format_cpio_svr4_gzip.c

src/libarchive/libarchive/test/test_read_format_cpio_svr4_gzip_rpm.c

src/libarchive/libarchive/test/test_read_format_cpio_svr4_gzip_rpm.rpm.uu

src/libarchive/libarchive/test/test_read_format_cpio_svr4c_Z.c

src/libarchive/libarchive/test/test_read_format_empty.c

src/libarchive/libarchive/test/test_read_format_gtar_gz.c

src/libarchive/libarchive/test/test_read_format_gtar_lzma.c

src/libarchive/libarchive/test/test_read_format_gtar_sparse.c

src/libarchive/libarchive/test/test_read_format_gtar_sparse_1_13.tar.uu

src/libarchive/libarchive/test/test_read_format_gtar_sparse_1_17.tar.uu

src/libarchive/libarchive/test/test_read_format_gtar_sparse_1_17_posix00.tar.uu

src/libarchive/libarchive/test/test_read_format_gtar_sparse_1_17_posix01.tar.uu

src/libarchive/libarchive/test/test_read_format_gtar_sparse_1_17_posix10.tar.uu

src/libarchive/libarchive/test/test_read_format_gtar_sparse_1_17_posix10_modified.tar.uu

src/libarchive/libarchive/test/test_read_format_iso.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_iso_gz.c

src/libarchive/libarchive/test/test_read_format_iso_joliet.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_iso_joliet_long.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_iso_joliet_rockridge.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_iso_multi_extent.c

src/libarchive/libarchive/test/test_read_format_iso_multi_extent.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_iso_rockridge.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_iso_rockridge_ce.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_iso_rockridge_new.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_iso_rockridge_rr_moved.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_iso_zisofs.iso.Z.uu

src/libarchive/libarchive/test/test_read_format_isojoliet_bz2.c

src/libarchive/libarchive/test/test_read_format_isojoliet_long.c

src/libarchive/libarchive/test/test_read_format_isojoliet_rr.c

src/libarchive/libarchive/test/test_read_format_isorr_bz2.c

src/libarchive/libarchive/test/test_read_format_isorr_ce.c

src/libarchive/libarchive/test/test_read_format_isorr_new_bz2.c

src/libarchive/libarchive/test/test_read_format_isorr_rr_moved.c

src/libarchive/libarchive/test/test_read_format_isozisofs_bz2.c

src/libarchive/libarchive/test/test_read_format_mtree.c

src/libarchive/libarchive/test/test_read_format_mtree.mtree.uu

src/libarchive/libarchive/test/test_read_format_pax_bz2.c

src/libarchive/libarchive/test/test_read_format_raw.c

src/libarchive/libarchive/test/test_read_format_raw.data.Z.uu

src/libarchive/libarchive/test/test_read_format_raw.data.uu

src/libarchive/libarchive/test/test_read_format_tar.c

src/libarchive/libarchive/test/test_read_format_tar_empty_filename.c

src/libarchive/libarchive/test/test_read_format_tar_empty_filename.tar.uu

src/libarchive/libarchive/test/test_read_format_tbz.c

src/libarchive/libarchive/test/test_read_format_tgz.c

src/libarchive/libarchive/test/test_read_format_tlz.c

src/libarchive/libarchive/test/test_read_format_txz.c

src/libarchive/libarchive/test/test_read_format_tz.c

src/libarchive/libarchive/test/test_read_format_xar.c

src/libarchive/libarchive/test/test_read_format_zip.c

src/libarchive/libarchive/test/test_read_format_zip.zip.uu

src/libarchive/libarchive/test/test_read_large.c

src/libarchive/libarchive/test/test_read_pax_truncated.c

src/libarchive/libarchive/test/test_read_position.c

src/libarchive/libarchive/test/test_read_truncated.c

src/libarchive/libarchive/test/test_read_uu.c

src/libarchive/libarchive/test/test_tar_filenames.c

src/libarchive/libarchive/test/test_tar_large.c

src/libarchive/libarchive/test/test_ustar_filenames.c

src/libarchive/libarchive/test/test_write_compress.c

src/libarchive/libarchive/test/test_write_compress_bzip2.c

src/libarchive/libarchive/test/test_write_compress_gzip.c

src/libarchive/libarchive/test/test_write_compress_lzma.c

src/libarchive/libarchive/test/test_write_compress_program.c

src/libarchive/libarchive/test/test_write_compress_xz.c

src/libarchive/libarchive/test/test_write_disk.c

src/libarchive/libarchive/test/test_write_disk_failures.c

src/libarchive/libarchive/test/test_write_disk_hardlink.c

src/libarchive/libarchive/test/test_write_disk_perms.c

src/libarchive/libarchive/test/test_write_disk_secure.c

src/libarchive/libarchive/test/test_write_disk_sparse.c

src/libarchive/libarchive/test/test_write_disk_symlink.c

src/libarchive/libarchive/test/test_write_disk_times.c

src/libarchive/libarchive/test/test_write_format_ar.c

src/libarchive/libarchive/test/test_write_format_cpio.c

src/libarchive/libarchive/test/test_write_format_cpio_empty.c

src/libarchive/libarchive/test/test_write_format_cpio_newc.c

src/libarchive/libarchive/test/test_write_format_cpio_odc.c

src/libarchive/libarchive/test/test_write_format_mtree.c

src/libarchive/libarchive/test/test_write_format_pax.c

src/libarchive/libarchive/test/test_write_format_shar_empty.c

src/libarchive/libarchive/test/test_write_format_tar.c

src/libarchive/libarchive/test/test_write_format_tar_empty.c

src/libarchive/libarchive/test/test_write_format_tar_ustar.c

src/libarchive/libarchive/test/test_write_format_zip.c

src/libarchive/libarchive/test/test_write_format_zip_empty.c

src/libarchive/libarchive/test/test_write_format_zip_no_compression.c

src/libarchive/libarchive/test/test_write_open_memory.c

src/libarchive/libarchive_fe

src/libarchive/libarchive_fe/err.c

src/libarchive/libarchive_fe/err.h

src/libarchive/libarchive_fe/lafe_platform.h

src/libarchive/libarchive_fe/line_reader.c

src/libarchive/libarchive_fe/line_reader.h

src/libarchive/libarchive_fe/matching.c

src/libarchive/libarchive_fe/matching.h

src/libarchive/libarchive_fe/pathmatch.c

src/libarchive/libarchive_fe/pathmatch.h

src/libarchive/tar

src/libarchive/tar/CMakeLists.txt

src/libarchive/tar/bsdtar.1

src/libarchive/tar/bsdtar.c

src/libarchive/tar/bsdtar.h

src/libarchive/tar/bsdtar_platform.h

src/libarchive/tar/bsdtar_windows.c

src/libarchive/tar/bsdtar_windows.h

src/libarchive/tar/cmdline.c

src/libarchive/tar/config_freebsd.h

src/libarchive/tar/getdate.c

src/libarchive/tar/read.c

src/libarchive/tar/subst.c

src/libarchive/tar/test

src/libarchive/tar/test/CMakeLists.txt

src/libarchive/tar/test/main.c

src/libarchive/tar/test/test.h

src/libarchive/tar/test/test_0.c

src/libarchive/tar/test/test_basic.c

src/libarchive/tar/test/test_copy.c

src/libarchive/tar/test/test_empty_mtree.c

src/libarchive/tar/test/test_getdate.c

src/libarchive/tar/test/test_help.c

src/libarchive/tar/test/test_option_T_upper.c

src/libarchive/tar/test/test_option_q.c

src/libarchive/tar/test/test_option_r.c

src/libarchive/tar/test/test_option_s.c

src/libarchive/tar/test/test_patterns.c

src/libarchive/tar/test/test_patterns_2.tar.uu

src/libarchive/tar/test/test_patterns_3.tar.uu

src/libarchive/tar/test/test_patterns_4.tar.uu

src/libarchive/tar/test/test_stdio.c

src/libarchive/tar/test/test_strip_components.c

src/libarchive/tar/test/test_symlink_dir.c

src/libarchive/tar/test/test_version.c

src/libarchive/tar/test/test_windows.c

src/libarchive/tar/tree.c

src/libarchive/tar/tree.h

src/libarchive/tar/util.c

src/libarchive/tar/write.c

src/local.c

src/local.h

src/quicklz

src/quicklz/quicklz.c

src/quicklz/quicklz.h

src/stream.c

src/stream.h

src/xbstream.c

src/xbstream.h

src/xbstream_read.c

src/xbstream_write.c

test/t/ib_stream_compress.sh

test/t/ib_stream_parallel.sh

test/t/ib_stream_tar.sh

test/t/ib_stream_xbstream.sh

files removed:
patches/tar4ibd_libtar-1.2.11.patch

files renamed:
Makefile => src/Makefile

xb_regex.h => src/xb_regex.h

xtrabackup.c => src/xtrabackup.c

test/t/xb_stream.sh => test/inc/ib_stream_common.sh

files modified:
innobackupex

test/inc/common.sh

test/run.sh

test/t/bug514068.sh

test/t/bug606981.sh *

test/t/bug759225.sh

test/t/tar4ibd_symlink.sh

test/t/xb_incremental_compressed.sh

utils/build.sh

Show diffs side-by-side

added added

removed removed

src/libarchive/libarchive/test/test_write_format_tar_ustar.c

/*-

* Redistribution and use in source and binary forms, with or without

* modification, are permitted provided that the following conditions

* are met:

* 1. Redistributions of source code must retain the above copyright

* notice, this list of conditions and the following disclaimer.

* 2. Redistributions in binary form must reproduce the above copyright

* notice, this list of conditions and the following disclaimer in the

* documentation and/or other materials provided with the distribution.

* THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR

* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES

* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.

* IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT,

* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT

* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,

* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY

* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT

* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF

* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

#include "test.h"

__FBSDID("$FreeBSD: head/lib/libarchive/test/test_write_format_tar_ustar.c 201247 2009-12-30 05:59:21Z kientzle $");

static int

is_null(const char *p, size_t l)

{

while (l > 0) {

if (*p != '\0')

return (0);

--l;

++p;

}

return (1);

}

/* Verify the contents, then erase them to NUL bytes. */

/* Tar requires all "unused" bytes be set to NUL; this allows us

* to easily verify that by invoking is_null() over the entire header

* after verifying each field. */

#define myAssertEqualMem(a,b,s) assertEqualMem(a, b, s); memset(a, 0, s)

* Detailed verification that 'ustar' archives are written with

* the correct format.

DEFINE_TEST(test_write_format_tar_ustar)

{

struct archive *a;

struct archive_entry *entry;

char *buff, *e;

size_t buffsize = 100000;

size_t used;

int i;

char f99[100];

char f100[101];

char f256[257];

for (i = 0; i < 99; ++i)

f99[i] = 'a' + i % 26;

f99[99] = '\0';

for (i = 0; i < 100; ++i)

f100[i] = 'A' + i % 26;

f100[100] = '\0';

for (i = 0; i < 256; ++i)

f256[i] = 'A' + i % 26;

f256[155] = '/';

f256[256] = '\0';

buff = malloc(buffsize);

/* Create a new archive in memory. */

assert((a = archive_write_new()) != NULL);

assertEqualIntA(a, 0, archive_write_set_format_ustar(a));

assertEqualIntA(a, 0, archive_write_set_compression_none(a));

assertEqualIntA(a, 0, archive_write_open_memory(a, buff, buffsize, &used));

* Add various files to it.

* TODO: Extend this to cover more filetypes.

/* "file" with 10 bytes of content */

assert((entry = archive_entry_new()) != NULL);

archive_entry_set_mtime(entry, 1, 10);

archive_entry_set_pathname(entry, "file");

archive_entry_set_mode(entry, S_IFREG | 0664);

archive_entry_set_size(entry, 10);

archive_entry_set_uid(entry, 80);

archive_entry_set_gid(entry, 90);

archive_entry_set_dev(entry, 12);

archive_entry_set_ino(entry, 89);

archive_entry_set_nlink(entry, 2);

assertEqualIntA(a, 0, archive_write_header(a, entry));

100

archive_entry_free(entry);

101

assertEqualIntA(a, 10, archive_write_data(a, "1234567890", 10));

102

103

/* Hardlink to "file" with 10 bytes of content */

104

assert((entry = archive_entry_new()) != NULL);

105

archive_entry_set_mtime(entry, 1, 10);

106

archive_entry_set_pathname(entry, "linkfile");

107

archive_entry_set_mode(entry, S_IFREG | 0664);

108

/* TODO: Put this back and fix the bug. */

109

/* archive_entry_set_size(entry, 10); */

110

archive_entry_set_uid(entry, 80);

111

archive_entry_set_gid(entry, 90);

112

archive_entry_set_dev(entry, 12);

113

archive_entry_set_ino(entry, 89);

114

archive_entry_set_nlink(entry, 2);

115

assertEqualIntA(a, 0, archive_write_header(a, entry));

116

archive_entry_free(entry);

117

/* Write of data to dir should fail == zero bytes get written. */

118

assertEqualIntA(a, 0, archive_write_data(a, "1234567890", 10));

119

120

/* "dir" */

121

assert((entry = archive_entry_new()) != NULL);

122

archive_entry_set_mtime(entry, 2, 20);

123

archive_entry_set_pathname(entry, "dir");

124

archive_entry_set_mode(entry, S_IFDIR | 0775);

125

archive_entry_set_size(entry, 10);

126

archive_entry_set_nlink(entry, 2);

127

assertEqualIntA(a, 0, archive_write_header(a, entry));

128

archive_entry_free(entry);

129

/* Write of data to dir should fail == zero bytes get written. */

130

assertEqualIntA(a, 0, archive_write_data(a, "1234567890", 10));

131

132

/* "symlink" pointing to "file" */

133

assert((entry = archive_entry_new()) != NULL);

134

archive_entry_set_mtime(entry, 3, 30);

135

archive_entry_set_pathname(entry, "symlink");

136

archive_entry_set_mode(entry, 0664);

137

archive_entry_set_filetype(entry, AE_IFLNK);

138

archive_entry_set_symlink(entry,"file");

139

archive_entry_set_size(entry, 0);

140

archive_entry_set_uid(entry, 88);

141

archive_entry_set_gid(entry, 98);

142

archive_entry_set_dev(entry, 12);

143

archive_entry_set_ino(entry, 90);

144

archive_entry_set_nlink(entry, 1);

145

assertEqualIntA(a, 0, archive_write_header(a, entry));

146

archive_entry_free(entry);

147

/* Write of data to symlink should fail == zero bytes get written. */

148

assertEqualIntA(a, 0, archive_write_data(a, "1234567890", 10));

149

150

/* file with 99-char filename. */

151

assert((entry = archive_entry_new()) != NULL);

152

archive_entry_set_mtime(entry, 1, 10);

153

archive_entry_set_pathname(entry, f99);

154

archive_entry_set_mode(entry, S_IFREG | 0664);

155

archive_entry_set_size(entry, 0);

156

archive_entry_set_uid(entry, 82);

157

archive_entry_set_gid(entry, 93);

158

archive_entry_set_dev(entry, 102);

159

archive_entry_set_ino(entry, 7);

160

archive_entry_set_nlink(entry, 1);

161

assertEqualIntA(a, 0, archive_write_header(a, entry));

162

archive_entry_free(entry);

163

164

/* file with 100-char filename. */

165

assert((entry = archive_entry_new()) != NULL);

166

archive_entry_set_mtime(entry, 1, 10);

167

archive_entry_set_pathname(entry, f100);

168

archive_entry_set_mode(entry, S_IFREG | 0664);

169

archive_entry_set_size(entry, 0);

170

archive_entry_set_uid(entry, 82);

171

archive_entry_set_gid(entry, 93);

172

archive_entry_set_dev(entry, 102);

173

archive_entry_set_ino(entry, 7);

174

archive_entry_set_nlink(entry, 1);

175

assertEqualIntA(a, 0, archive_write_header(a, entry));

176

archive_entry_free(entry);

177

178

/* file with 256-char filename. */

179

assert((entry = archive_entry_new()) != NULL);

180

archive_entry_set_mtime(entry, 1, 10);

181

archive_entry_set_pathname(entry, f256);

182

archive_entry_set_mode(entry, S_IFREG | 0664);

183

archive_entry_set_size(entry, 0);

184

archive_entry_set_uid(entry, 82);

185

archive_entry_set_gid(entry, 93);

186

archive_entry_set_dev(entry, 102);

187

archive_entry_set_ino(entry, 7);

188

archive_entry_set_nlink(entry, 1);

189

assertEqualIntA(a, 0, archive_write_header(a, entry));

190

archive_entry_free(entry);

191

192

#if ARCHIVE_VERSION_NUMBER < 2000000

193

archive_write_finish(a);

194

#else

195

assert(0 == archive_write_finish(a));

196

#endif

197

198

199

* Verify the archive format.

200

201

e = buff;

202

203

/* "file" */

204

myAssertEqualMem(e + 0, "file", 5); /* Filename */

205

myAssertEqualMem(e + 100, "000664 ", 8); /* mode */

206

myAssertEqualMem(e + 108, "000120 ", 8); /* uid */

207

myAssertEqualMem(e + 116, "000132 ", 8); /* gid */

208

myAssertEqualMem(e + 124, "00000000012 ", 12); /* size */

209

myAssertEqualMem(e + 136, "00000000001 ", 12); /* mtime */

210

myAssertEqualMem(e + 148, "010034\0 ", 8); /* checksum */

211

myAssertEqualMem(e + 156, "0", 1); /* linkflag */

212

myAssertEqualMem(e + 157, "", 1); /* linkname */

213

myAssertEqualMem(e + 257, "ustar\000000", 8); /* signature/version */

214

myAssertEqualMem(e + 265, "", 1); /* uname */

215

myAssertEqualMem(e + 297, "", 1); /* gname */

216

myAssertEqualMem(e + 329, "000000 ", 8); /* devmajor */

217

myAssertEqualMem(e + 337, "000000 ", 8); /* devminor */

218

myAssertEqualMem(e + 345, "", 1); /* prefix */

219

assert(is_null(e + 0, 512));

220

myAssertEqualMem(e + 512, "1234567890", 10);

221

assert(is_null(e + 512, 512));

222

e += 1024;

223

224

/* hardlink to "file" */

225

myAssertEqualMem(e + 0, "linkfile", 9); /* Filename */

226

myAssertEqualMem(e + 100, "000664 ", 8); /* mode */

227

myAssertEqualMem(e + 108, "000120 ", 8); /* uid */

228

myAssertEqualMem(e + 116, "000132 ", 8); /* gid */

229

myAssertEqualMem(e + 124, "00000000000 ", 12); /* size */

230

myAssertEqualMem(e + 136, "00000000001 ", 12); /* mtime */

231

myAssertEqualMem(e + 148, "010707\0 ", 8); /* checksum */

232

myAssertEqualMem(e + 156, "0", 1); /* linkflag */

233

myAssertEqualMem(e + 157, "", 1); /* linkname */

234

myAssertEqualMem(e + 257, "ustar\000000", 8); /* signature/version */

235

myAssertEqualMem(e + 265, "", 1); /* uname */

236

myAssertEqualMem(e + 297, "", 1); /* gname */

237

myAssertEqualMem(e + 329, "000000 ", 8); /* devmajor */

238

myAssertEqualMem(e + 337, "000000 ", 8); /* devminor */

239

myAssertEqualMem(e + 345, "", 1); /* prefix */

240

assert(is_null(e + 0, 512));

241

e += 512;

242

243

/* "dir" */

244

myAssertEqualMem(e + 0, "dir/", 4); /* Filename */

245

myAssertEqualMem(e + 100, "000775 ", 8); /* mode */

246

myAssertEqualMem(e + 108, "000000 ", 8); /* uid */

247

myAssertEqualMem(e + 116, "000000 ", 8); /* gid */

248

myAssertEqualMem(e + 124, "00000000000 ", 12); /* size */

249

myAssertEqualMem(e + 136, "00000000002 ", 12); /* mtime */

250

myAssertEqualMem(e + 148, "007747\0 ", 8); /* checksum */

251

myAssertEqualMem(e + 156, "5", 1); /* typeflag */

252

myAssertEqualMem(e + 157, "", 1); /* linkname */

253

myAssertEqualMem(e + 257, "ustar\000000", 8); /* signature/version */

254

myAssertEqualMem(e + 265, "", 1); /* uname */

255

myAssertEqualMem(e + 297, "", 1); /* gname */

256

myAssertEqualMem(e + 329, "000000 ", 8); /* devmajor */

257

myAssertEqualMem(e + 337, "000000 ", 8); /* devminor */

258

myAssertEqualMem(e + 345, "", 1); /* prefix */

259

assert(is_null(e + 0, 512));

260

e += 512;

261

262

/* "symlink" pointing to "file" */

263

myAssertEqualMem(e + 0, "symlink", 8); /* Filename */

264

myAssertEqualMem(e + 100, "000664 ", 8); /* mode */

265

myAssertEqualMem(e + 108, "000130 ", 8); /* uid */

266

myAssertEqualMem(e + 116, "000142 ", 8); /* gid */

267

myAssertEqualMem(e + 124, "00000000000 ", 12); /* size */

268

myAssertEqualMem(e + 136, "00000000003 ", 12); /* mtime */

269

myAssertEqualMem(e + 148, "011446\0 ", 8); /* checksum */

270

myAssertEqualMem(e + 156, "2", 1); /* linkflag */

271

myAssertEqualMem(e + 157, "file", 5); /* linkname */

272

myAssertEqualMem(e + 257, "ustar\000000", 8); /* signature/version */

273

myAssertEqualMem(e + 265, "", 1); /* uname */

274

myAssertEqualMem(e + 297, "", 1); /* gname */

275

myAssertEqualMem(e + 329, "000000 ", 8); /* devmajor */

276

myAssertEqualMem(e + 337, "000000 ", 8); /* devminor */

277

myAssertEqualMem(e + 345, "", 1); /* prefix */

278

assert(is_null(e + 0, 512));

279

e += 512;

280

281

/* File with 99-char filename */

282

myAssertEqualMem(e + 0, f99, 100); /* Filename */

283

myAssertEqualMem(e + 100, "000664 ", 8); /* mode */

284

myAssertEqualMem(e + 108, "000122 ", 8); /* uid */

285

myAssertEqualMem(e + 116, "000135 ", 8); /* gid */

286

myAssertEqualMem(e + 124, "00000000000 ", 12); /* size */

287

myAssertEqualMem(e + 136, "00000000001 ", 12); /* mtime */

288

myAssertEqualMem(e + 148, "034242\0 ", 8); /* checksum */

289

myAssertEqualMem(e + 156, "0", 1); /* linkflag */

290

myAssertEqualMem(e + 157, "", 1); /* linkname */

291

myAssertEqualMem(e + 257, "ustar\000000", 8); /* signature/version */

292

myAssertEqualMem(e + 265, "", 1); /* uname */

293

myAssertEqualMem(e + 297, "", 1); /* gname */

294

myAssertEqualMem(e + 329, "000000 ", 8); /* devmajor */

295

myAssertEqualMem(e + 337, "000000 ", 8); /* devminor */

296

myAssertEqualMem(e + 345, "", 1); /* prefix */

297

assert(is_null(e + 0, 512));

298

e += 512;

299

300

/* File with 100-char filename */

301

myAssertEqualMem(e + 0, f100, 100); /* Filename */

302

myAssertEqualMem(e + 100, "000664 ", 8); /* mode */

303

myAssertEqualMem(e + 108, "000122 ", 8); /* uid */

304

myAssertEqualMem(e + 116, "000135 ", 8); /* gid */

305

myAssertEqualMem(e + 124, "00000000000 ", 12); /* size */

306

myAssertEqualMem(e + 136, "00000000001 ", 12); /* mtime */

307

myAssertEqualMem(e + 148, "026230\0 ", 8); /* checksum */

308

myAssertEqualMem(e + 156, "0", 1); /* linkflag */

309

myAssertEqualMem(e + 157, "", 1); /* linkname */

310

myAssertEqualMem(e + 257, "ustar\000000", 8); /* signature/version */

311

myAssertEqualMem(e + 265, "", 1); /* uname */

312

myAssertEqualMem(e + 297, "", 1); /* gname */

313

myAssertEqualMem(e + 329, "000000 ", 8); /* devmajor */

314

myAssertEqualMem(e + 337, "000000 ", 8); /* devminor */

315

myAssertEqualMem(e + 345, "", 1); /* prefix */

316

assert(is_null(e + 0, 512));

317

e += 512;

318

319

/* File with 256-char filename */

320

myAssertEqualMem(e + 0, f256 + 156, 100); /* Filename */

321

myAssertEqualMem(e + 100, "000664 ", 8); /* mode */

322

myAssertEqualMem(e + 108, "000122 ", 8); /* uid */

323

myAssertEqualMem(e + 116, "000135 ", 8); /* gid */

324

myAssertEqualMem(e + 124, "00000000000 ", 12); /* size */

325

myAssertEqualMem(e + 136, "00000000001 ", 12); /* mtime */

326

myAssertEqualMem(e + 148, "055570\0 ", 8); /* checksum */

327

myAssertEqualMem(e + 156, "0", 1); /* linkflag */

328

myAssertEqualMem(e + 157, "", 1); /* linkname */

329

myAssertEqualMem(e + 257, "ustar\000000", 8); /* signature/version */

330

myAssertEqualMem(e + 265, "", 1); /* uname */

331

myAssertEqualMem(e + 297, "", 1); /* gname */

332

myAssertEqualMem(e + 329, "000000 ", 8); /* devmajor */

333

myAssertEqualMem(e + 337, "000000 ", 8); /* devminor */

334

myAssertEqualMem(e + 345, f256, 155); /* prefix */

335

assert(is_null(e + 0, 512));

336

e += 512;

337

338

/* TODO: Verify other types of entries. */

339

340

/* Last entry is end-of-archive marker. */

341

assert(is_null(e, 1024));

342

e += 1024;

343

344

assertEqualInt((int)used, e - buff);

345

346

free(buff);

347

}

Older »