-
Committer:
Laurynas Biveinis
-
Date:
2013-04-16 10:35:58 UTC
-
Revision ID:
laurynas.biveinis@percona.com-20130416103558-1y3853gd6akfdrei
Implement bitmap-based incremental backups.
Blueprints:
https://blueprints.launchpad.net/percona-xtrabackup/+spec/changed-page-bmp-inc-backups
and
https://blueprints.launchpad.net/percona-xtrabackup/+spec/multiple-bmp-file-inc-backups
- innobackupex: split the xtrabackup_suspended to three different
files depending on the context (1st suspend, 2nd suspend, log copy
end). If backing up, check if server supports changed page bitmaps
by querying for the presence of I_S.INNODB_CHANGED_PAGES plugin. If
found, add 1st suspend to xtrabackup invocation and issue FLUSH
CHANGED_PAGE_BITMAPS during it. Factor out waitpid/sync file
presence loop out of wait_for_ibbackup_suspend and resume_ibbackup
into new subroutine wait_for_ibbackup_file_create. Split the log
copying finish logic out of resume_ibbackup into a new subroutine
wait_for_ibbackup_log_copy_finish. Add a new command line option
--incremental-force-scan that passes the same option to xtrabackup,
and skips the first suspend and FLUSH CHANGED_PAGE_BITMAP calls.
- New source files changed_page_bitmap.h and changed_page_bitmap.cc
containing the types and functions for working with page bitmaps and
their iterators.
- New source files read_filt.h and read_filt.cc containing the data
file read filter abstraction. Provide two filters: rf_pass_through
for no filtering, and rf_bitmap that uses the changed page bitmaps.
- fil_cur.h, fil_cur.cc: xb_fil_cur_t: move offset field to
xb_read_filt_ctxt_t. Add fields for the read filter and its
context: read_filter and read_filter_ctxt. xb_fil_cur_open(): new
arg read_filter. Initialize the read filter context.
xb_fil_cur_read: use the read filter to get the next read offset and
length. xb_fil_cur_close(): deinitialize the read filter.
- innodb_int.h: include mysql_version.h to make version checks not
depend on mysql_version.h having been included before innodb_int.h.
- innodb_int.cc: fix an unrelated warning to make it easier to ensure
that the feature does not regress in warnings: xb_space_get_by_name:
mark fold as unused.
- xtrabackup.h: declare checkpoint_lsn_start and changed_page_bitmap.
- xtrabackup.cc: split xtrabackup_suspended to three different files
as in innobackupex. New functions xb_make_sync_file_name and
xtrabackup_suspend, rename xb_create_suspend_file to
xb_create_sync_file. Add new option --suspend-at-start to suspend
xtrabackup after the log copying thread has started.
xtrabackup_copy_datafile: set up a bitmap read filter if the bitmap
has been allocated, a pass through filter otherwise and pass it to
the file cursor.
xtrabackup_backup_func(): in case of incremental backups attempt to
read the changed page bitmap. Write a message if the bitmap was
found, otherwise that a full scan is going to be used. Free the
bitmap in the end.
New command-line option --incremental-force-scan to use the
full-scan algorithm even in the presence of bitmap data.
- test/inc/common.sh: init_server_variables(), reset_server_variables:
set up server error log location in SRV_MYSQLD_ERRFILE[].
switch_server(): set up MySQL error log.
check_full_scan_inc_backup(), check_bitmap_inc_backup(): new
functions for grepping XtraBackup output for the incremental backup
type used. wait_for_xb_suspend(), resume_suspended_xb(): new helper
functions for the testcases that use xtrabackup suspend.
- Rename test test/t/ib_incremental.sh to a test include file
test/inc/ib_incremental_common.sh. Pass additional args in
$mysqld_extra_args to mysqld.
- Rename test test/t/xb_incremental.sh to an include file
test/inc/xb_incremental_common.sh. Pass additional args in
$mysqld_extra_args to mysqld. Adjust server options to have a
minimum possible log file size (1M) and thread concurrency that can
support such size for 5.0. Change the schema of test table T2 to
have large rows. Insert enough rows into a database so that the
resulting bitmap file spans at least two bitmap pages for the T2
tablespace.
- Rename test test/t/ib_incremental.sh to an include file
test/inc/ib_incremental_common.sh. Pass additional args in
$mysqld_extra_args to mysqld. Tweak it to increase the testing
coverage. ADd three new testcases ib_incremental_bitmap.sh,
ib_incremental_full_scan.sh, and ib_incremental_force_full_scan.sh
that use it.
- Adjust test/t/xb_incremental_compressed.inc for bitmap backups. New
tests xb_incremental_compressed_bitmap_(1-16)kb.sh. Rename
the previous xb_incremental_compressed_*.sh tests to
xb_incremental_compressed_full_scan_*.sh. Remove the InnoDB version
check from xb_incremental_compressed.inc as it's always passing with
the currently supported versions.
- New test xb_incremental_bitmap_misc.sh to test diagnostics for
partially-missing bitmap data and --incremental-force-scan option.
- Temporarily disable all the bitmap tests on XtraDB Cluster
configuration until a supporting version is released.
- Adjust test/t/bug1007446.sh for changed suspend file names.
- Adjust test/t/xb_log_overwraph.sh to use
wait_for_xb_to_suspend/resume_suspended_xb.
- Makefile: update dependencies.
- Updated docs.