storage: limit data written per VDI with --write-volume-cap

[glehmann]

Refactor stream management to support arbitrary span distributions.
The new StreamSpan dataclass encapsulates stream metadata and operations,
while partially_populate_device() now supports configurable span counts
and skip patterns.

This required to upgrade randstream to the version 0.6.0 which supports
arbitrary position.

The default cap is 2GiB.

Signed-off-by: Gaëtan Lehmann gaetan.lehmann@vates.tech

This PR is part of a tree containing 19 PRs:

1. master
2. storage: avoid removing the xva/vdi before going in the debugger #436 → master
3. storage: test large volumes #437 → storage: avoid removing the xva/vdi before going in the debugger #436
4. host: include mdadm RAID devices in disk detection #447 → storage: test large volumes #437
5. Update randstream to 0.5.0 #446 → host: include mdadm RAID devices in disk detection #447
6. storage: Avoid writing the whole device in coalesce tests #449 → Update randstream to 0.5.0 #446
7. storage: avoid writing the whole device in migration tests #450 → storage: Avoid writing the whole device in coalesce tests #449
8. xva/vdi: only write a small data amount in large volumes for faster tests #452 → storage: avoid writing the whole device in migration tests #450
9. storage: test full device write #453 → xva/vdi: only write a small data amount in large volumes for faster tests #452
10. storage: test that we can't create a vdi over its max allowed size #454 → storage: test full device write #453
11. storage: add jobs for large volume tests #461 → storage: test that we can't create a vdi over its max allowed size #454
12. skip large volume tests for zvol and nfsv4 #464 → storage: add jobs for large volume tests #461
13. storage: add comprehensive tests for lvmohba storage repositories #470 → skip large volume tests for zvol and nfsv4 #464
14. storage: free space for XVA import by destroying source VM first #471 → storage: add comprehensive tests for lvmohba storage repositories #470
15. "storage: limit data written per VDI with --write-volume-cap" (this PR) → storage: free space for XVA import by destroying source VM first #471
16. Add Packer configuration to build a minimal Alpine 3.23 UEFI VM for XCP-ng tests #523 → storage: limit data written per VDI with --write-volume-cap #481
17. migration: create a xfs sr on the second host for intra/cross-pool migration #497 → storage: limit data written per VDI with --write-volume-cap #481
18. Enhance block device management #498 → migration: create a xfs sr on the second host for intra/cross-pool migration #497
19. Add VHD_MAX and QCOW2_MAX symbolic size constants #500 → Enhance block device management #498
20. partially_populate_device: align span positions to block size for better performance #509 → Add VHD_MAX and QCOW2_MAX symbolic size constants #500

[rzr]

is it twice volume size ?

[rzr]

Sorry if asked previously but why magic numbers are duplicated, and not a constant introduced?

[glehmann]

Because I didn't feel a need for it, I suppose.

Indeed, I think we've already had this discussion :)

[rzr]

What about using sets in place of list ?

[glehmann]

That's the type returned by the function that creates them.
And it's convenient to use a list as return type because they are not all equivalent when using skip_span.