~laurynas-biveinis/percona-xtrabackup/xb-changed-page-bitmap

  • Committer: Laurynas Biveinis
  • Date: 2013-04-16 10:35:58 UTC
  • Revision ID: laurynas.biveinis@percona.com-20130416103558-1y3853gd6akfdrei
Implement bitmap-based incremental backups.
Blueprints:
https://blueprints.launchpad.net/percona-xtrabackup/+spec/changed-page-bmp-inc-backups
and
https://blueprints.launchpad.net/percona-xtrabackup/+spec/multiple-bmp-file-inc-backups

- innobackupex: split the xtrabackup_suspended to three different
  files depending on the context (1st suspend, 2nd suspend, log copy
  end).  If backing up, check if server supports changed page bitmaps
  by querying for the presence of I_S.INNODB_CHANGED_PAGES plugin.  If
  found, add 1st suspend to xtrabackup invocation and issue FLUSH
  CHANGED_PAGE_BITMAPS during it.  Factor out waitpid/sync file
  presence loop out of wait_for_ibbackup_suspend and resume_ibbackup
  into new subroutine wait_for_ibbackup_file_create.  Split the log
  copying finish logic out of resume_ibbackup into a new subroutine
  wait_for_ibbackup_log_copy_finish.  Add a new command line option
  --incremental-force-scan that passes the same option to xtrabackup,
  and skips the first suspend and FLUSH CHANGED_PAGE_BITMAP calls.

- New source files changed_page_bitmap.h and changed_page_bitmap.cc
  containing the types and functions for working with page bitmaps and
  their iterators.

- New source files read_filt.h and read_filt.cc containing the data
  file read filter abstraction.  Provide two filters: rf_pass_through
  for no filtering, and rf_bitmap that uses the changed page bitmaps.

- fil_cur.h, fil_cur.cc: xb_fil_cur_t: move offset field to
  xb_read_filt_ctxt_t.  Add fields for the read filter and its
  context: read_filter and read_filter_ctxt.  xb_fil_cur_open(): new
  arg read_filter.  Initialize the read filter context.
  xb_fil_cur_read: use the read filter to get the next read offset and
  length.  xb_fil_cur_close(): deinitialize the read filter.

- innodb_int.h: include mysql_version.h to make version checks not
  depend on mysql_version.h having been included before innodb_int.h.

- innodb_int.cc: fix an unrelated warning to make it easier to ensure
  that the feature does not regress in warnings: xb_space_get_by_name:
  mark fold as unused.

- xtrabackup.h: declare checkpoint_lsn_start and changed_page_bitmap.

- xtrabackup.cc: split xtrabackup_suspended to three different files
  as in innobackupex.  New functions xb_make_sync_file_name and
  xtrabackup_suspend, rename xb_create_suspend_file to
  xb_create_sync_file.  Add new option  --suspend-at-start to suspend
  xtrabackup after the log copying thread has started.
  xtrabackup_copy_datafile: set up a bitmap read filter if the bitmap
  has been allocated, a pass through filter otherwise and pass it to
  the file cursor. 
  xtrabackup_backup_func():  in case of incremental backups attempt to
  read the changed page bitmap.  Write a message if the bitmap was
  found, otherwise that a full scan is going to be used.  Free the
  bitmap in the end.
  New command-line option --incremental-force-scan to use the
  full-scan algorithm even in the presence of bitmap data.

- test/inc/common.sh: init_server_variables(), reset_server_variables:
  set up server error log location in SRV_MYSQLD_ERRFILE[].
  switch_server():  set up MySQL error log.
  check_full_scan_inc_backup(), check_bitmap_inc_backup(): new
  functions for grepping XtraBackup output for the incremental backup
  type used.  wait_for_xb_suspend(), resume_suspended_xb(): new helper
  functions for the testcases that use xtrabackup suspend.

- Rename test test/t/ib_incremental.sh to a test include file
  test/inc/ib_incremental_common.sh.  Pass additional args in
  $mysqld_extra_args to mysqld.

- Rename test test/t/xb_incremental.sh to an include file
  test/inc/xb_incremental_common.sh.  Pass additional args in
  $mysqld_extra_args to mysqld.  Adjust server options to have a
  minimum possible log file size (1M) and thread concurrency that can
  support such size for 5.0.  Change the schema of test table T2 to
  have large rows.  Insert enough rows into a database so that the 
  resulting bitmap file spans at least two bitmap pages for the T2
  tablespace.

- Rename test test/t/ib_incremental.sh to an include file
  test/inc/ib_incremental_common.sh.  Pass additional args in
  $mysqld_extra_args to mysqld.  Tweak it to increase the testing
  coverage.  ADd three new testcases ib_incremental_bitmap.sh,
  ib_incremental_full_scan.sh, and ib_incremental_force_full_scan.sh
  that use it.

- Adjust test/t/xb_incremental_compressed.inc for bitmap backups.  New
  tests xb_incremental_compressed_bitmap_(1-16)kb.sh.  Rename
  the previous xb_incremental_compressed_*.sh tests to
  xb_incremental_compressed_full_scan_*.sh.  Remove the InnoDB version
  check from xb_incremental_compressed.inc as it's always passing with
  the currently supported versions.

- New test xb_incremental_bitmap_misc.sh to test diagnostics for
  partially-missing bitmap data and --incremental-force-scan option.

- Temporarily disable all the bitmap tests on XtraDB Cluster
  configuration until a supporting version is released.

- Adjust test/t/bug1007446.sh for changed suspend file names.

- Adjust test/t/xb_log_overwraph.sh to use
  wait_for_xb_to_suspend/resume_suspended_xb.

- Makefile: update dependencies.

- Updated docs.
Filename Latest Rev Last Changed Committer Comment Size
..
contrib 241.3.1 13 years ago Daniël van Eeden Add contrib directory with an backup script exampl Diff
doc 213.3.1 13 years ago Aleksandr Kuzminsky Added XtraBackup manual page Diff
patches 228.1.2 13 years ago Alexey Kopytov Port XtraBackup to Percona Server 5.5.9-beta20.1 c Diff
src 385.1.1 12 years ago Alexey Kopytov Rebase of the parallel compression patch on new tr Diff
test 134.1.1 13 years ago Aleksandr Kuzminsky Added the XtraBackup test framework Diff
utils 30.1.2 15 years ago Vadim Tkachenko directory restructure Diff
.bzrignore 391.1.58 11 years ago Stewart Smith merge from 1.6: Bug #1021954: XtraBackup on MultiI 196 bytes Diff Download File
BUILD-WIN.txt 244.4.1 12 years ago Alexey Kopytov Added initial support for Windows builds: - remov 2.9 KB Diff Download File
BUILD.txt 522.6.1 11 years ago Laurynas Biveinis Merge build against MySQL 5.6 support from 2.0. A 2.2 KB Diff Download File
CMakeLists.txt 244.4.1 12 years ago Alexey Kopytov Added initial support for Windows builds: - remov 1.6 KB Diff Download File
COPYING 174.1.1 13 years ago Stewart Smith add COPYING file with GPLv2 license. xtrabackup is 18.6 KB Diff Download File
File innobackupex 535 10 years ago Laurynas Biveinis Implement bitmap-based incremental backups. Bluepr 101 KB Diff Download File
VERSION 533 10 years ago Alexey Kopytov Manual merge from 2.0. 31 bytes Diff Download File