Skip to content
Percona Software for MySQL Documentation

Rate this page
☆ ☆ ☆ ☆ ☆
Thanks for your feedback
Thank you! The feedback has been submitted.

Get free database assistance or contact our experts for personalized support.

Get help from Percona

Percona XtraBackup internals

Percona XtraBackup (PXB) runs three sequential phases: backup, prepare, and restore. This page describes the internal work each phase performs.

For commands and flag selection, see How-to: backup lifecycle operations. For compact tables of privileges and defaults, see Reference: backup lifecycle.

Three phases at a glance

The following table maps each operator phase to the InnoDB-internal work the phase performs:

Phase Internal work
Backup (hot copy) Copy data files and capture redo for the full backup window
Prepare Replay captured redo, then roll back transactions open at backup end
Restore Place the prepared files in the destination datadir

The prepare phase runs the same pipeline as InnoDB crash recovery, but offline on the backup tree.

Backup phase

The backup phase produces a directory that holds everything mysqld needs to start a fresh server. The output includes data files for every storage engine, a captured slice of the InnoDB redo log, and a small set of metadata files.

PXB queries performance_schema.log_status to stamp a start log sequence number (LSN), then runs two parallel streams of work:

  • A data-file stream copies on-disk files for every storage engine in use. InnoDB tablespaces (.ibd files), the shared tablespace (ibdata1), and undo tablespaces make up most of the volume. PXB also copies files for MyISAM, CSV, MyRocks, and other recognized engines.

  • A background redo-log thread tails the InnoDB redo log from the start LSN through the end of the backup window. The thread writes the capture into xtrabackup_logfile inside the backup directory.

PXB picks the backup end LSN during the final lock posture. While LOCK BINLOG FOR BACKUP is held, PXB reads performance_schema.log_status. The query records the current LSN, binary log file and position, and replica coordinates. The redo thread keeps draining records past that sync-point LSN, then stops. PXB records both LSNs in xtrabackup_checkpoints. to_lsn is the sync-point LSN. last_lsn is the highest LSN the redo thread actually copied. For the LSN fields recorded in incremental backups, see Incremental backups.

InnoDB incremental backups can skip the full data-file scan with page tracking. The mysqlbackup server component records the LSN of every page write. The --page-tracking flag lets PXB copy only the pages changed since the previous backup. For setup and limitations, see Take an incremental backup using page tracking.

MyRocks copies are always wholesale. The MyRocks storage layout does not expose per-page change tracking. PXB copies every MyRocks file on each backup, including incremental backups. For the full storage-engine support list, see About Percona XtraBackup.

The dual capture lets the prepare phase advance the copy to a single LSN. PXB also writes metadata files that prepare and restore consume, including xtrabackup_checkpoints, xtrabackup_info, xtrabackup_binlog_info, and backup-my.cnf. For the full list of artifacts, see Index of files created by Percona XtraBackup.

PXB stages locking across the backup so InnoDB data manipulation language (DML) keeps running through the bulk file copy. Two flags shape the trade-offs:

Flag Default Effect
--lock-ddl ON Controls when the backup lock takes effect
--register-redo-log-consumer OFF Holds redo files on the server until PXB finishes reading them

Redo capture and why the copy is not yet consistent

InnoDB keeps writing while PXB reads files, so the copied pages reflect mixed points in time. Pages copied early carry older LSNs than pages copied later.

To bridge that gap, PXB stamps a start LSN and copies InnoDB data files. A background thread streams the InnoDB redo log in parallel for the entire backup window.

The redo capture matters for two reasons:

  • The .ibd page reads span real clock time. Without a continuous redo log, prepare cannot advance the copy to a single LSN.

  • InnoDB redo files wrap and recycle. The redo thread must read every record before the server overwrites the underlying file.

Without redo capture, the directory holds physically mixed page images. The copy then has no path to run InnoDB recovery against a single LSN. The dataset therefore becomes consistent during prepare, not during the file copy.

Three lock postures

PXB stages locking across the backup so InnoDB DML keeps running through the bulk copy.

The following table lists the three sub-phases in order:

# Lock posture Internal work
1 No global lock Copy InnoDB data files and stream redo while DML continues
2 Backup lock Copy non-InnoDB files such as .frm, .MRG, .MYD, .MYI, .CSM, .CSV, .sdi, and .par. Data definition language (DDL) is restricted; DML continues, with details that depend on the server build.
3 LOCK BINLOG FOR BACKUP (short) Pin binary log or replica coordinates, read performance_schema.log_status, and release locks

For background on the underlying lock primitives, see Reduction in locks and the upstream MySQL documentation for LOCK INSTANCE FOR BACKUP.

--lock-ddl=ON compared with --lock-ddl=REDUCED

The --lock-ddl flag controls when PXB takes the backup lock that gates DDL.

The following table compares the two modes:

Value Backup lock taken DDL impact
ON (default) Backup start DDL is restricted for the full backup window
REDUCED After the InnoDB copy finishes DDL is restricted for a shorter window. DDL runs concurrently with the InnoDB copy, so PXB tracks tablespace changes from redo and recopies affected tablespaces when the lock is taken.

InnoDB DML continues through the main copy under both modes.

--register-redo-log-consumer: keep redo until PXB finishes

On a high-traffic host, the server can recycle redo files before PXB finishes reading them. A purged redo file makes the backup unusable.

The --register-redo-log-consumer flag registers PXB as a redo consumer. The server then retains a redo file until PXB has copied that file.

Review the trade-offs before enabling the flag:

  • Redo files stay on disk longer, which raises peak redo size during the run.

  • Free space can drop sharply on a long backup of a write-heavy server.

  • The server can briefly stall writes when the consumer advances.

Warning

Monitor free disk during the backup. Abort with Ctrl+C or SIGTERM to xtrabackup if disk space reaches a critical threshold. The consumer releases on abort, and the server can then purge redo.

The default value is OFF. Enable the flag only when spare disk space is available.

When PXB can skip backup locks

PXB skips backup locks only when every table in every schema, including the mysql system schema, uses InnoDB. Most installations still hold CSV or MyISAM tables in mysql, such as general_log, so PXB takes backup locks in practice.

Replication coordinate handling depends on the server build:

  • Percona Server for MySQL can fold relay coordinates into log_status, which reduces the locks needed for --slave-info.

  • Stock MySQL can still require FLUSH TABLES WITH READ LOCK for some relay-position scenarios.

Prepare phase: roll forward, then roll back

xtrabackup --prepare runs two coordinated passes against the backup directory.

Roll forward applies the captured redo to the copied tablespaces. The pass is physical: each redo record describes a page-level change. InnoDB reapplies that change to the page image on disk.

Roll back removes transactions that had not committed at the backup end LSN. Redo replay can have written those changes to pages, so prepare must reverse the changes at the logical level.

Roll back relies on two structures inside the tablespaces:

  • Undo records describe how to reverse the row-level effects of each open transaction.

  • Serialized Dictionary Information (SDI) carries table and index definitions inside each tablespace. SDI supplies the schema context that older .frm files once provided to rollback.

Non-InnoDB tables match the same point in time because PXB copied them under backup locks during the backup phase.

After prepare finishes, the dataset matches the backup end LSN with open transactions stripped.

Restore phase: file placement

The restore phase reads destination paths from the server configuration, including datadir, innodb_data_home_dir, innodb_data_file_path, and innodb_log_group_home_dir.

PXB places files in a fixed order:

  • MyISAM-family files first, including .MRG, .MYD, .MYI, .CSM, .CSV, .sdi, and .par.

  • InnoDB tablespaces and indexes second.

  • Log files last.

File attributes survive the copy. A successful backup writes binary log coordinates to standard error (STDERR). Redirect STDERR to a file when the workflow needs to capture those coordinates.

Cloud and streaming: same mechanics, longer windows

Streaming and xbcloud workflows run the same internal phases as a local backup. Slow networks extend the PXB final-sync phase, which holds the short backup locks, including the binary log lock, for longer.

For workflow procedures, see the following references:

See also