fix(linstor): Database Backup when Linstor Controller is not the Pool Master. by Ythogtha · Pull Request #142 · xcp-ng/sm

Ythogtha · 2026-06-10T09:46:34Z

Added secondary backup location outside of Linstor's DRBD mounts.
Throttling is restricted to backup on the Linstor Controller (needs access to files).
check_sr operates a regular backup only if it runs on the Linstor Controller, to allow throttling.
Retention is enforced by check_sr on Controller, with access to actual files.
Retention checks all backup.zip files for consistency :
Valid zip with a non-empty linstordb.mv.db file inside,
Reporting errors, and keeping only valid files.

Millefeuille42

Only small changes / clarification. Not all comments are actually blocking though.

Wescoeur · 2026-06-18T13:28:09Z

+    def is_controller(self):
+        """Checks if the current host is the Linstor Controller.
+        This is done by checking that the Linstor database path is a mountpoint.
+        Which should only be the case on the Linstor Controller."""
+        return os.path.ismount(DATABASE_PATH)


Should be merged with existing code like cls._is_mounted(DATABASE_PATH).

So can you replace existing cls._is_mounted(DATABASE_PATH) calls with is_controller() ? Thx. :)

Wescoeur · 2026-06-18T13:33:23Z

I don't understand the choice of using a log file rather than directly listing the logs, either by using:

The file dates (not the most robust method if they are modified).
Or by using a clear and robust naming convention, in which case there are no problems.

Ythogtha · 2026-06-18T17:21:27Z

I don't understand the choice of using a log file rather than directly listing the logs, either by using:
* The file dates (not the most robust method if they are modified).

* Or by using a clear and robust naming convention, in which case there are no problems.

Because we can't use throttling with filenames on SR operations (not check_sr), because we don't know if we have access to the files.
So in order to have a usable delay parameter, another solutions needs to be found.
I used a logfile.
But that's a bad one, and instead I removed the delay parameter, since only check_sr needs it, and it can be run on the Controller, therefore have access to the files, and use them as reference.

Ythogtha · 2026-06-18T17:44:47Z

Also, I converted some method into @classmethod since only the actual backup requires a linstor connection, and an instance of LinstorVolumeManager.

Wescoeur

Ok for merge after rebase. Just one comment to add regarding the new LINSTOR_AUTO_BACKUP_DELAY constant and a remaining refactoring.

Wescoeur · 2026-09-02T19:57:31Z

 # on the master.
 NS_PREFIX_LVM: Final = "lvm-"
+
+LINSTOR_AUTO_BACKUP_DELAY = 3600


We should add a comment here to explain usage and unit used.
For example: "Delay expressed in seconds between two LINSTOR DB backups launched by check_sr helper.

Wescoeur · 2026-09-02T20:33:15Z

+    def is_controller(self):
+        """Checks if the current host is the Linstor Controller.
+        This is done by checking that the Linstor database path is a mountpoint.
+        Which should only be the case on the Linstor Controller."""
+        return os.path.ismount(DATABASE_PATH)


So can you replace existing cls._is_mounted(DATABASE_PATH) calls with is_controller() ? Thx. :)

- Retention of 10 backups from operations, and check_sr always saves on latest. - Provides a secondary backup location, with the same retention policy. Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

… Master. * Added secondary backup location outside of Linstor's DRBD mounts. * Throttling uses a logfile for backup operations. * sr_scan operates a regular backup only if it runs on the Linstor Controller. * Retention is enforced by sr_scan on controller, with access to actual files. * Exceptions are raised only on backup errors from sr_scan ; Other operations will only report in SMlog. * Retention checks all backup.zip files for consistency : Valid zip with a non-empty linstordb.mv.db file inside, Reporting errors, and keeping only valid files. Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

check_sr works only on the Controller to have access to the backup files. It applies Validation, Retention, and Throttling for the automatic backup. All three of which requires access to the backup files, on the Controller. Removal of the logfile to manage throttling for normal operation. Hence removal of the delay parameter. Restricted caught Exceptions in backup validation to (FileNotFoundError, zipfile.BadZipFile). Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Don't except/raise LinstorDatabaseBackupError in _check_database_backup(). Simpler glob search for backup files, as the date is properly validated by datetime. Fix bug in database_backup_age(). Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

database_backup_path instead of database_backup_file because it is a str|Path. Throttling delay converted to Constant. Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

…le None self._linstor Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Also some explanations on new LINSTOR_AUTO_BACKUP_DELAY in constants.py Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Ythogtha force-pushed the agf-linstor_backupdb_base branch from 0e5dcb8 to f5e5bfc Compare June 10, 2026 09:59

Ythogtha changed the base branch from 3.2.12-8.3 to 3.2.12-8.3-cowutil-qcow2 June 10, 2026 10:04

Wescoeur changed the base branch from 3.2.12-8.3-cowutil-qcow2 to 3.2.12-8.3-devel June 10, 2026 12:58

Ythogtha force-pushed the agf-linstor_backupdb_base branch from b18a3f2 to 46bc6e3 Compare June 16, 2026 10:05

Ythogtha changed the title ~~Linstor: (fix) Remove all operations on backup files…~~ fix(linstor): Database Backup when Linstor Controller is not the Pool Master. Jun 16, 2026

Wescoeur requested a review from a team June 16, 2026 15:15

Ythogtha mentioned this pull request Jun 17, 2026

Linstor backup database basic checks. #138

Closed

Millefeuille42 requested changes Jun 17, 2026

View reviewed changes

Comment thread drivers/LinstorSR.py Outdated

Comment thread drivers/LinstorSR.py Outdated

Comment thread drivers/linstorvolumemanager.py Outdated

Comment thread drivers/linstorvolumemanager.py Outdated

Millefeuille42 requested a review from a team June 17, 2026 08:54

Wescoeur requested changes Jun 18, 2026

View reviewed changes

Ythogtha requested review from Millefeuille42 and Wescoeur June 18, 2026 17:21

Ythogtha force-pushed the agf-linstor_backupdb_base branch 2 times, most recently from 9126138 to 6e1ab3d Compare June 18, 2026 17:43

Ythogtha force-pushed the agf-linstor_backupdb_base branch 2 times, most recently from b090447 to b6b7676 Compare June 19, 2026 07:45

Millefeuille42 requested changes Jun 24, 2026

View reviewed changes

Comment thread drivers/LinstorSR.py

Comment thread drivers/LinstorSR.py

Comment thread drivers/linstorvolumemanager.py

Comment thread drivers/linstorvolumemanager.py

Ythogtha force-pushed the agf-linstor_backupdb_base branch 2 times, most recently from 3dba5e3 to 09c9925 Compare June 25, 2026 12:04

Ythogtha requested a review from Millefeuille42 June 26, 2026 14:45

Millefeuille42 approved these changes Jun 26, 2026

View reviewed changes

Nambrok force-pushed the 3.2.12-8.3-devel branch from e712aa5 to fce2d13 Compare June 29, 2026 07:36

Ythogtha force-pushed the agf-linstor_backupdb_base branch from 09c9925 to 6c0d0a3 Compare June 29, 2026 14:02

Wescoeur requested a review from klmp200 July 2, 2026 14:18

Nambrok force-pushed the 3.2.12-8.3-devel branch from 6607beb to c4dce9e Compare July 2, 2026 14:25

Ythogtha force-pushed the agf-linstor_backupdb_base branch from 6c0d0a3 to b42bcd2 Compare July 7, 2026 08:42

klmp200 requested changes Jul 7, 2026

View reviewed changes

Comment thread drivers/LinstorSR.py Outdated

Comment thread drivers/LinstorSR.py

Comment thread drivers/LinstorSR.py

Comment thread drivers/LinstorSR.py

Comment thread drivers/linstorvolumemanager.py

Comment thread drivers/LinstorSR.py

Ythogtha force-pushed the agf-linstor_backupdb_base branch 7 times, most recently from 3ed3cbd to b2d00c7 Compare July 20, 2026 13:03

klmp200 approved these changes Jul 29, 2026

View reviewed changes

Wescoeur requested changes Jul 29, 2026

View reviewed changes

Ythogtha requested a review from Wescoeur July 30, 2026 15:37

Wescoeur requested changes Jul 30, 2026

View reviewed changes

Comment thread drivers/linstorvolumemanager.py Outdated

Comment thread drivers/LinstorSR.py Outdated

Ythogtha requested a review from Wescoeur July 31, 2026 15:23

Nambrok force-pushed the 3.2.12-8.3-devel branch 2 times, most recently from 1825cbe to 9978d7f Compare August 13, 2026 07:45

Ythogtha force-pushed the agf-linstor_backupdb_base branch from 14961f8 to 43e4e88 Compare August 13, 2026 09:21

Millefeuille42 self-requested a review August 19, 2026 13:39

Nambrok force-pushed the 3.2.12-8.3-devel branch from 85d4bc6 to 285450d Compare August 24, 2026 14:21

Wescoeur requested changes Sep 2, 2026

View reviewed changes

Ythogtha added 8 commits September 3, 2026 13:06

fix(linstor): Clarifications, documentations, and more explicit work.

a681f0c

Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Clarifying comments, type hint for database_backup#name parameter.

6a7983b

Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Better names, comments and a small but elegant optimization.

ad99004

database_backup_path instead of database_backup_file because it is a str|Path. Throttling delay converted to Constant. Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Better name for backup invalidation and mypy handling of a (im)possib…

d341622

…le None self._linstor Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Ythogtha force-pushed the agf-linstor_backupdb_base branch from 43e4e88 to d341622 Compare September 3, 2026 11:07

Replaced _is_mounted(DATABASE_PATH) by is_controller() when relevant.

f31152e

Also some explanations on new LINSTOR_AUTO_BACKUP_DELAY in constants.py Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Ythogtha requested a review from Wescoeur September 4, 2026 08:27

Fix: usage of typing:Literal on sm_typing is added for obsolete Python.

8b62522

Signed-off-by: Arnaud Garcia-Fernandez <arnaud.garcia-fernandez@vates.tech>

Uh oh!

Conversation

Ythogtha commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Millefeuille42 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Wescoeur Jun 18, 2026

Choose a reason for hiding this comment

Uh oh!

Wescoeur Sep 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Wescoeur commented Jun 18, 2026

Uh oh!

Ythogtha commented Jun 18, 2026

Uh oh!

Ythogtha commented Jun 18, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Wescoeur left a comment

Choose a reason for hiding this comment

Uh oh!

Wescoeur Sep 2, 2026

Choose a reason for hiding this comment

Uh oh!

Wescoeur Sep 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Ythogtha commented Jun 10, 2026 •

edited

Loading