You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/backups-incremental.md
+67-59Lines changed: 67 additions & 59 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,19 +6,11 @@
6
6
7
7
This feature is in the tech preview stage. The behavior can change in future releases.
8
8
9
-
If your database grows quickly or sees heavy write traffic, you may want frequent backups. Taking a full backup every time costs more storage, takes longer to upload, and adds load to the cluster.
9
+
**Incremental** backups copy only what changed since the previous backup in the chain. They are smaller, faster to transfer, and cheaper to keep than taking a full backup every time.
10
10
11
-
**Incremental** backups copy only what changed since the previous backup in the chain. They are usually smaller, faster to transfer, and cheaper to keep in the backup storage.
11
+
## Backup chain
12
12
13
-
## What you need first
14
-
15
-
Every incremental backup belongs to a **chain** that starts with one **full** backup on the **same** storage. The Operator checks that a valid full backup exists before it starts an incremental one. Using the same storage for the whole chain enables the Operator to reuse the same credentials and paths, and simplifies restore.
16
-
17
-
You can use any [supported storage type](backups-storage.md).
18
-
19
-
## What the backup chain looks like
20
-
21
-
Incremental backups build on the full backup and on each other in order:
13
+
Every incremental backup belongs to a **chain** that starts with one **full** backup on the **same** storage. The Operator checks that a valid full backup exists before it starts an incremental one.
22
14
23
15
```mermaid
24
16
flowchart LR
@@ -28,81 +20,97 @@ flowchart LR
28
20
D --> IN["Increment N"]
29
21
```
30
22
31
-
By default, the Operator uses the **latest full backup** as the base for both **scheduled** and **on-demand** incremental backups. If you want more control, you can explicitly specify the base backup in the configuration for on-demand backups. The Operator validates the specified backup and derives the incremental chain from it.
23
+
By default, the Operator uses the **latest full backup** as the base. For on-demand backups, you can set a different base with `spec.incrementalBaseBackupName`.
24
+
25
+
You can use any [supported storage type](backups-storage.md).
32
26
33
27
## How an incremental backup runs
34
28
35
-
1. You create a `PerconaServerMySQLBackup` object with type **incremental**. Or, you configure the **backup schedule** in the cluster Custom Resource that creates incremental backups.
36
-
2. The Operator confirms that a **full** backup exists on the same storage and is valid. Unless you specify another backup, it uses the **most recent** full backup.
37
-
3. If the backup is valid, the Operator sets the `percona.com/base-backup-name` annotation on it so that it serves as the base for the incremental backup chain.
38
-
4. If there are already incremental backups derived from the base, the Operator takes the `to_lsn` value from the previous increment and sets it as the `from_lsn` value for the new increment so the chain stays continuous.
39
-
5. The Operator streams the new incremental to the same storage.
40
-
6. The Operator records the backup type on the backup object.
29
+
1. You [make an on-demand backup](backups-ondemand.md) with type **incremental**, or you [configure a schedule](backups-scheduled.md) that creates incremental backups.
30
+
2. The Operator confirms that a valid **full** backup exists on the same storage. Unless you specify another backup, it uses the most recent full backup.
31
+
3. The Operator marks that full backup as the chain base and streams the increment to the same storage.
32
+
4. Later increments use the previous increment’s `to_lsn` as `from_lsn` so the chain stays continuous.
41
33
42
34
!!! important
43
35
44
-
The `percona.com/base-backup-name` annotation is internal and serves to correctly link incremental backups to the base one. Removing or editing it will lead to unpredictable results and data corruption. Don't remove or edit this annotation.
36
+
The Operator sets the `percona.com/base-backup-name` annotation on the base backup to link the chain. Do not edit or remove it. Changing it can break the chain and lead to data corruption.
45
37
46
-
## How restore from an incremental backup works
38
+
## Restore from an incremental backup
47
39
48
-
The restore flow is unified for both full and incremental backups. The Operator identifies the backup type by name or destination. To identify the increments and reconstruct the chain, the backup destination has now the `.incr` path segment. The Operator downloads the full backup and all related increments and sorts them in the correct order. Then it restores the full backup first and applies each incremental backup.
40
+
The restore flow is the same for full and incremental backups. Follow [Restore the cluster from a previously saved backup](backups-restore.md) or [Restore from a backup to a new Kubernetes-based environment](backups-restore-to-new-cluster.md).
49
41
50
-
If you make a [point-in-time recovery](backups-pitr.md), it also applies binlogs on top, after restoring all backups. To learn more how it works, check [the point-in-time recovery workflow](backups-pitr.md#how-the-operator-performs-pointintime-recovery).
42
+
The Operator identifies the backup type by name or destination. Incremental destinations use an `.incr` path segment so the Operator can reconstruct the chain. It downloads the full backup and all related increments, restores the full backup first, then applies each increment.
51
43
52
-
Here's how it works in detail:
44
+
If you make a [point-in-time recovery](backups-pitr.md), the Operator applies binlogs after the chain. See [the point-in-time recovery workflow](backups-pitr.md#point-in-time-recovery-workflow).
53
45
54
-
1. You create a `PerconaServerMySQLRestore` object and reference the incremental backup with `backupName` (same cluster) or `backupSource` (remote path / another environment).
55
-
2. The Operator detects that the target is incremental and resolves the chain.
56
-
3. It connects to the storage and lists full backup plus all increments up to and including the one you chose, using paths that include the `.incr` segment so increments are unambiguous.
57
-
4. The Operator sorts increments in the right order.
58
-
5. The Operator pauses the cluster for the restore.
59
-
6. The restore Job downloads and applies the full backup, then each incremental one in order.
60
-
7. The cluster is unpaused when the restore completes.
46
+
Incremental objects in storage use an `.incr` path segment so the Operator can tell them apart from full backups:
## Speed up incremental backups with page tracking
63
58
64
-
Incremental destinations use an `.incr` segment in the path so the Operator can tell full and incremental artifacts apart. A typical pattern resembles `prefix/<cluster>-<timestamp>-full.incr/<cluster>-<timestamp>-incr`; exact layout follows your `prefix` and storage settings.
Page tracking lets Percona XtraBackup copy only the InnoDB pages that changed since the last backup, instead of scanning every data file. The Operator installs the `mysqlbackup` component that this feature needs. For details, see [Take an incremental backup using page tracking :octicons-link-external-16:](https://docs.percona.com/percona-xtrabackup/latest/page-tracking.html).
Enable page tracking by passing the `--page-tracking` flag in the `xtrabackup` container options. Set it on the **storage** so the whole chain uses it:
76
64
77
-
## Why you need incremental backups
65
+
```yaml title="deploy/cr.yaml"
66
+
spec:
67
+
backup:
68
+
storages:
69
+
s3-us-west:
70
+
containerOptions:
71
+
args:
72
+
xtrabackup:
73
+
- "--page-tracking"
74
+
```
78
75
79
-
With incremental backups, you gain the following benefits:
76
+
Override it for a single on-demand backup:
77
+
78
+
```yaml title="deploy/backup/backup.yaml"
79
+
spec:
80
+
clusterName: ps-cluster1
81
+
storageName: s3-us-west
82
+
type: incremental
83
+
containerOptions:
84
+
args:
85
+
xtrabackup:
86
+
- "--page-tracking"
87
+
```
80
88
81
-
* strengthen your backup strategy by creating multiple restore points
82
-
* increase storage efficiency by avoiding duplication of unchanged data
83
-
* lower system load, since smaller backups require fewer compute resources and reduce impact on your cluster
84
-
* reduce both storage and data‑transfer costs
89
+
Pass the `--page-tracking` flag on the **base** backup to start tracking for the whole chain. The next incremental backup then copies only changed pages. Keep the flag on later increments so tracking continues.
90
+
91
+
If you set the `--page-tracking` flag only on an incremental job, that job still scans all pages because the previous full backup did not start tracking.
92
+
93
+
For how cluster-level and per-job options interact, see [Fine-tuning backup and restore operations](backups-fine-tune.md).
85
94
86
95
## Implementation specifics and rules
87
96
88
97
### Backup chain rules
89
98
90
-
1. A full backup is required to start the incremental chain. If none exists, the incremental backup fails.
91
-
2.A base full backup and incremental backups derived from it must be **on the same storage**.
92
-
3. By default, the Operator uses the most recent full backup to start the incremental chain. You can explicitly specify the base full backup in the `spec.incrementalBaseBackupName` option in the backup configuration file. If the specified full backup is valid, the Operator starts the incremental backup chain from it.
93
-
4. If the base backup already has the incremental backup chain, the Operator uses the most recent increment to continue the chain.
94
-
5. Retention applies to the chain as a unit: deleting the **base** full backup removes the **entire** incremental chain that depends on it, so you do not leave orphaned increments. Specifying the retention policy for increments is not supported.
95
-
6. You cannot delete an increment in the middle of a chain as it would break its continuity. You can delete only the **last** increment in the chain or the base backup, which removes the whole chain.
96
-
7.The Operator prevents to run two concurrent incremental backups against the **same** chain to avoid ambiguous ordering. The Operator runs increments one by one.
99
+
1. A full backup is required to start the chain. If none exists, the incremental backup fails.
100
+
2. The base full backup and its increments must be on the **same** storage.
101
+
3. By default, the Operator uses the most recent full backup. You can set a different base with `spec.incrementalBaseBackupName`. If that backup is valid, the Operator starts the chain from it.
102
+
4. If the base backup already has increments, the Operator continues from the most recent increment.
103
+
5. Retention applies to the chain as a unit: deleting the **base** full backup removes the entirechain. You cannot set retention on increments alone.
104
+
6. You cannot delete an increment in the middle of a chain. You can delete only the **last** increment, or the base backup, which removes the whole chain.
105
+
7. The Operator does not run two incremental backups on the **same** chain at once. Increments run one by one.
97
106
98
107
### Restore rules
99
108
100
-
1.You can make either an in-place restore pointing at an incremental backup object in the `backupName` option, or make a cross-cluster restore specifying the incremental backup path for the `backupSource.destination` option.
101
-
2.Restores that use `backupSource` work across clusters and namespaces when the storage destination is reachable; incremental paths remain discoverable because of the `.incr` layout.
109
+
1. [Restore in place](backups-restore.md) with `backupName`, or [restore across clusters](backups-restore-to-new-cluster.md) with `backupSource.destination`.
110
+
2. `backupSource` restores work across clusters and namespaces when the storage path is reachable. The `.incr` layout keeps incremental paths discoverable.
102
111
3. Restore always needs the full chain: full backup first, then increments in order, up to the backup you selected.
103
112
104
-
105
113
## Known limitations
106
114
107
-
- If a backup in your chain fails but some data was already uploaded to storage, your restore still includes that failed backup, so the restore fails. Whenever any backup in the chain fails, start a new chain.
108
-
- If the checkpoint file is missing from a backup directory, your next incremental backup can hang. Start a new chain.
115
+
- If a backup in your chain fails but some data was already uploaded, restore still includes that failed backup and fails. Start a new chain.
116
+
- If the checkpoint file is missing from a backup directory, the next incremental backup can hang. Start a new chain.
0 commit comments