Skip to content

feat: add LVM metadata backup - #168

Open
Kuruyia wants to merge 3 commits into
3.2.12-8.3-develfrom
aso/lvm_metadata_backup
Open

Kuruyia wants to merge 3 commits into
3.2.12-8.3-develfrom
aso/lvm_metadata_backup

Conversation

@Kuruyia

@Kuruyia Kuruyia commented Sep 15, 2026 •

Copy link
Copy Markdown

This adds backing up the metadata of the LVM VGs that are managed by the SM. This is done by copying the metadata files that are generated by LVM commands in the /etc/lvm/backup/ directory to an SM-controlled directory. This minimizes the impact on performance that backing up metadata incurs.
The SM also manages the lifecycle of the backed up files by only keeping some amount of the most recent files. Backups are done before running LVM commands, since the /etc/lvm/backup/ directory is expected to contain the current state of the VGs.

Metadata backup only occurs when running commands that would modify the metadata of a VG/LV, such as creation, removal and resizing. It also occurs when changing some of a VG/LV attributes, but not all. For instance, volume (de)activation is a common attribute-changing operation that does not require metadata backup.

The backed-up LVM metadata files are removed from the SM-controlled directory when the associated SR is destroyed.

@Kuruyia Kuruyia self-assigned this Sep 15, 2026
@Kuruyia
Kuruyia marked this pull request as draft September 15, 2026 13:13
@Kuruyia

Kuruyia commented Sep 15, 2026

Copy link
Copy Markdown
Author

I'm opening this PR as draft for now, mainly so we can assess whether we want to continue going with this implementation for LVM metadata backups. I'm more interested in reviews on the idea rather than the code for now.

I also have a couple of question that currently prevent this PR from being marked as ready, on which I'd like some discussions. Opening those in separate comment threads below.

Comment thread drivers/LinstorSR.py
Comment thread drivers/lvmbackup.py
@Kuruyia
Kuruyia requested a review from a team September 15, 2026 13:17
@Kuruyia
Kuruyia force-pushed the aso/lvm_metadata_backup branch from 8bf6934 to 47233a0 Compare September 18, 2026 14:48

@Kuruyia Kuruyia left a comment •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with @Wescoeur @Nambrok, we decided to:

  • Remove the specific support for the LINSTOR, EXT and XFS SRs
  • Change the backup destination directory to a sub-directory of /etc/sm/
  • Keep the current backup retention policy and adjust it later on if needed
  • Delete the backup directory on SR destroy
  • Check if code can be refactored between this PR and the LINSTOR database backup code to share some of the implementation

@Kuruyia
Kuruyia force-pushed the aso/lvm_metadata_backup branch from 47233a0 to 359d024 Compare September 24, 2026 08:52
@Kuruyia
Kuruyia force-pushed the aso/lvm_metadata_backup branch from 359d024 to 64eb310 Compare September 24, 2026 14:16
@Kuruyia
Kuruyia force-pushed the aso/lvm_metadata_backup branch from 64eb310 to 2aa9f70 Compare September 25, 2026 08:27
@Kuruyia
Kuruyia force-pushed the aso/lvm_metadata_backup branch 3 times, most recently from 451556a to 5299f58 Compare September 30, 2026 08:53
This adds backing up the metadata of the LVM VGs that are managed by the
SM. This is done by copying the metadata files that are generated by LVM
commands in the `/etc/lvm/backup/` directory to an SM-controlled
directory. This minimizes the impact on performance that backing up
metadata incurs. The SM also manages the lifecycle of the backed up
files by only keeping some amount of the most recent files. Backups are
done *before* running LVM commands, since the `/etc/lvm/backup/`
directory is expected to contain the current state of the VGs.

Metadata backup only occurs when running commands that would modify the
metadata of a VG/LV, such as creation, removal and resizing. It also
occurs when changing some of a VG/LV attributes, but not all. For
instance, volume (de)activation is a common attribute-changing operation
that does not require metadata backup.

The backed-up LVM metadata files are removed from the SM-controlled
directory when the associated SR is destroyed.

Signed-off-by: Alexandre Sollier <alexandre.sollier@vates.tech>
@Kuruyia
Kuruyia force-pushed the aso/lvm_metadata_backup branch from 5299f58 to cd01c96 Compare October 5, 2026 13:30
This adds a new generic abstract base class, `BackupManager`, that can
be inherited to implement file-based backup operations.

This base class implements common logic for sorting backup files,
creating new ones, and applying a retention policy based on keeping a
certain amount of the newest backup files.

Inheritors of this class need to provide the concrete implementations
for backup file naming, backup file listing, and the backup operation.
They can also provide an implementation for a custom per-backup file
retention policy, which can be used to e.g. forcefully remove backup
files that are considered invalid.

This also updates the LVM metadata backup code to take advantage of this
new base class.

Signed-off-by: Alexandre Sollier <alexandre.sollier@vates.tech>
This ports the LINSTOR database backup code to the newly-introduced
`BackupManager` base class. Current behavior has been kept whenever
possible.

Signed-off-by: Alexandre Sollier <alexandre.sollier@vates.tech>
@Kuruyia
Kuruyia marked this pull request as ready for review October 7, 2026 11:02
@Kuruyia

Kuruyia commented Oct 7, 2026

Copy link
Copy Markdown
Author

Done applying all suggestions that were discussed a few weeks ago. Leaving this PR out of draft as I think it's ready for review.

I'm keeping the first commit for now since I'm waiting to hear whether we want to keep the refactor part or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant