Request: Persist writable VDEV properities across replacements

[tonyhutter]

Describe the feature you would like to see added to OpenZFS

ZFS does not currently preserve writable VDEV properties across replacements:

$ sudo ./zpool get failfast tank `pwd`/file2
NAME                    PROPERTY  VALUE     SOURCE
/home/hutter/zfs/file2  failfast  off       local

$ sudo ./zpool replace tank `pwd`/file2 `pwd`/file3
$ sudo ./zpool get failfast tank `pwd`/file3
NAME                    PROPERTY  VALUE     SOURCE
/home/hutter/zfs/file3  failfast  on        default

This makes pool management difficult, as it's a common to replace a drive with the exact same make/model of drive, and expect everything to work "the same", without having to manually re-set VDEV props on the new drive. This would be less of an issue if VDEV properties are inherited, but they currently are not inherited, short of some special cases like checksum_[n|t], io_[n|t], slow_io_[n|t].

How will this feature improve OpenZFS?

It would allow VDEV proprieties to persist across drive replacements. For example, you could set per-vdev queue settings on NVMe drives and have it persist across drive replacements (#17358). User-defined VDEV properties would also be retained.

Additional context

[amotin]

If user properties include serial numbers, etc, then retaining them may be confusing.