Enhance block device management

[glehmann]

Some block devices may be used to build other devices (mdadm, lvm) or be accessible on multiple hosts, but only usable on a single one at a time (fc, iscsi).

This PR is part of a tree containing 19 PRs:

1. master
2. storage: avoid removing the xva/vdi before going in the debugger #436 → master
3. storage: test large volumes #437 → storage: avoid removing the xva/vdi before going in the debugger #436
4. host: include mdadm RAID devices in disk detection #447 → storage: test large volumes #437
5. Update randstream to 0.5.0 #446 → host: include mdadm RAID devices in disk detection #447
6. storage: Avoid writing the whole device in coalesce tests #449 → Update randstream to 0.5.0 #446
7. storage: avoid writing the whole device in migration tests #450 → storage: Avoid writing the whole device in coalesce tests #449
8. xva/vdi: only write a small data amount in large volumes for faster tests #452 → storage: avoid writing the whole device in migration tests #450
9. storage: test full device write #453 → xva/vdi: only write a small data amount in large volumes for faster tests #452
10. storage: test that we can't create a vdi over its max allowed size #454 → storage: test full device write #453
11. storage: add jobs for large volume tests #461 → storage: test that we can't create a vdi over its max allowed size #454
12. skip large volume tests for zvol and nfsv4 #464 → storage: add jobs for large volume tests #461
13. storage: add comprehensive tests for lvmohba storage repositories #470 → skip large volume tests for zvol and nfsv4 #464
14. storage: free space for XVA import by destroying source VM first #471 → storage: add comprehensive tests for lvmohba storage repositories #470
15. storage: limit data written per VDI with --write-volume-cap #481 → storage: free space for XVA import by destroying source VM first #471
16. Add Packer configuration to build a minimal Alpine 3.23 UEFI VM for XCP-ng tests #523 → storage: limit data written per VDI with --write-volume-cap #481
17. migration: create a xfs sr on the second host for intra/cross-pool migration #497 → storage: limit data written per VDI with --write-volume-cap #481
18. "Enhance block device management" (this PR) → migration: create a xfs sr on the second host for intra/cross-pool migration #497
19. Add VHD_MAX and QCOW2_MAX symbolic size constants #500 → Enhance block device management #498
20. partially_populate_device: align span positions to block size for better performance #509 → Add VHD_MAX and QCOW2_MAX symbolic size constants #500

[stormi]

Declare it before the method that uses it?

[stormi]

Comment about the commit message: I read it but it didn't explain what problem it solves. Only how it solves it.

[glehmann]

I've reworked the block device again to mark disks with unused partitions as available.

[vxgmichel]

One suggestion, LGTM otherwise 👍

[vxgmichel]

Maybe follow the convention for python test discovery by renaming tests/unit/rescan_block_devices_info.py to tests/unit/test_rescan_block_devices_info.py?

Suggested change

	- run: pytest tests/unit/*.py
	- run: pytest tests/unit

[glehmann]

I voluntarily kept them without the test_ prefix, so they are not run with the other tests, to not add more to the output in jenkins or in our terminal.

Do you think that's a problem?

[stormi]

Alternatively, we could put then under /unit so that they're easier to run from CLI directly without having to think about the *.py stuff.

[vxgmichel]

Apparently passing a test directory explicitly takes priority over --ignore, so another possible approach would be add the following.

diff --git a/pytest.ini b/pytest.ini
index 3deaa2c..bf08f83 100644
--- a/pytest.ini
+++ b/pytest.ini
@@ -1,5 +1,5 @@
 [pytest]
-addopts = -ra --maxfail=1 --ignore=contrib/
+addopts = -ra --maxfail=1 --ignore=contrib/ --ignore=tests/unit
 markers =
     # *** Markers that change test behaviour ***
     default_vm: mark a test with a default VM in case no --vm parameter was given.

This effectively disables the unit tests by default.

But @stormi suggestion of using a unit directory outside of tests is probably more explicit. It would require defining testpaths though:

diff --git a/pytest.ini b/pytest.ini
index 3deaa2c..a94601d 100644
--- a/pytest.ini
+++ b/pytest.ini
@@ -1,5 +1,6 @@
 [pytest]
-addopts = -ra --maxfail=1 --ignore=contrib/
+testpaths = tests
+addopts = -ra --maxfail=1
 markers =

[glehmann]

Apparently passing a test directory explicitly takes priority over --ignore

I did that. Running the unit tests is now done with just pytest tests/unit.