Is your feature request related to a problem? Please describe?
As a end-user from Customer, it has been seen that NBD has known throughput limitations. SAN copy (as of today) limits to offline/no incremental copies. Running a hot add proxy (or multiple) in VMWare to VJB would allow a much higher throughput compared to NBD while remaining storage agnostic.
Detailed Description
When VJB runs the NBD copy, there's bottlenecks. VMware/Broadcom has documented this for years. NBD has always been a convenience setup compared to SAN copy or hot add as it has the fewest requirements.
HotAdd offers superior throughput by directly accessing datastores, ideal for high-speed backups, while NBD (Network Block Device) uses the management network, acting as a slower fallback, suitable when HotAdd fails due to storage access issues or configuration, with performance heavily reliant on network bandwidth (10GbE recommended)
Refer - https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-sdks-tools/9-0/virtual-disk-development-kit-programming-guide/virtual-disk-interfaces/virtual-disk-transport-methods/hotadd-transport.html
Anything else you would like to add?
Customer shared how they have used hot-add proxy transport in the past (for reference)
-Build fedora VM for source/destination
-Install VDDK toolkits, nbdkit/nbdcopy, dependencies
-Shutdown source VM (This is where disk lease management is needed to remain powered on)
-Take snapshot of source
-Add parent disk for source VM to source proxy
-Spin up nbdkit (nbdkit -fv --readonly file /dev/sdb)
-Run nbd copy from destination proxy in KVM (sudo qemu-nbd -d /dev/nbd0
nbdcopy nbd://10.100.50.170:10809 /dev/vdb --progress)
-Run ntfsfix then v2v in place (/usr/sbin/ntfsfix /dev/vdb2
/usr/sbin/virt-v2v-in-place -v -i disk /dev/vdb)
-Create VM
Is your feature request related to a problem? Please describe?
As a end-user from Customer, it has been seen that NBD has known throughput limitations. SAN copy (as of today) limits to offline/no incremental copies. Running a hot add proxy (or multiple) in VMWare to VJB would allow a much higher throughput compared to NBD while remaining storage agnostic.
Detailed Description
When VJB runs the NBD copy, there's bottlenecks. VMware/Broadcom has documented this for years. NBD has always been a convenience setup compared to SAN copy or hot add as it has the fewest requirements.
HotAdd offers superior throughput by directly accessing datastores, ideal for high-speed backups, while NBD (Network Block Device) uses the management network, acting as a slower fallback, suitable when HotAdd fails due to storage access issues or configuration, with performance heavily reliant on network bandwidth (10GbE recommended)
Refer - https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-sdks-tools/9-0/virtual-disk-development-kit-programming-guide/virtual-disk-interfaces/virtual-disk-transport-methods/hotadd-transport.html
Anything else you would like to add?
Customer shared how they have used hot-add proxy transport in the past (for reference)
-Build fedora VM for source/destination
-Install VDDK toolkits, nbdkit/nbdcopy, dependencies
-Shutdown source VM (This is where disk lease management is needed to remain powered on)
-Take snapshot of source
-Add parent disk for source VM to source proxy
-Spin up nbdkit (nbdkit -fv --readonly file /dev/sdb)
-Run nbd copy from destination proxy in KVM (sudo qemu-nbd -d /dev/nbd0
nbdcopy nbd://10.100.50.170:10809 /dev/vdb --progress)
-Run ntfsfix then v2v in place (/usr/sbin/ntfsfix /dev/vdb2
/usr/sbin/virt-v2v-in-place -v -i disk /dev/vdb)
-Create VM