Partitions on Windows

[lundman]

So I finally understand what the issue is with OpenZFS partitioned disks, why Windows will "break" them and what we can do.

Unix uses gpt->NumPartitions == 9.
According to the UEFI specification (specifically Chapter 5 on the GUID Partition Table Format), the CRC32 checksum of the Partition Entry Array is calculated over the byte range defined by NumberOfPartitionEntries * SizeOfPartitionEntry.
9×128=1152 bytes.

Windows always calculates the Partition Entry Array based on the MAX_PARTITIONS==128. Even though they are all zero after partition 9, the CRC value will then differ. The GPT partition is considered invalid.

New versions of Windows 11 with helpfully then wipe the Partition Entry Array with all zeros, re-calculate the CRC and update the GPT header. The CRC will now match, so Windows considers it valid, if empty.

OpenZFS will now consider the disk empty, as it has no partitions.

If you now corrupt the GPT header, ie by writing _FI Part to it, OpenZFS will now consider the GPT partition label invalid.

OpenZFS will now automatically go read the Backup GPT partition at the end of the disk. The Backup CRC will be valid for 9 partitions (Windows did not change it), and it can read the Backup Partition Entry Array. Now it finds the OpenZFS partitions and can import it.

Technically, Windows has not implemented EFI GPT partitions correctly, but even if they fix it today, it doesn't solve the problem for a few years, if ever.

---

A)
Options then. We could replace the GPT labels with gpt->NumPartitions == 9 to be gpt->NumPartitions == 128 automatically, and Windows would be happy. Presumably Unix would be as well, but how sure are we on that. Unix has a bunch of variants, 32bit, endianess, toasters and so on. Are we sure we aren't breaking something? I dislike the idea of modifying the partition table just because the/a disk was too close to Windows, and now it is different (but same).

---

B)
I have been experimenting with a Shim driver in OpenZFS. That is, a barnacle inside the OpenZFS driver, which will check out any disks on the system (and new disks being attached), read in the partition label, and if it happens to be gpt->NumPartitions == 9. it goes and attaches a Shim driver on top of disk.sys for that specific disk (for each disk applicable).

The Shim will relay all the IRP communication as normal, except for IOCTL_DISK_GET_DRIVE_LAYOUT_EX. When it gets IOCTL_DISK_GET_DRIVE_LAYOUT_EX, which will be returned as "empty disk" as Windows as described above, it can read the EFI partition label itself, and modify the result on the fly.

We could even consider repairing a previous "Windows repair" by copying the Backup label back to Primary. As long as the Shim is active, Windows will not attempt to "repair" it again.

[lundman]

A successful run of the Shim:

ShimDiskInterfaceChange arrival: \??\USBSTOR#Disk&Ven_Logitec&Prod_&Rev_0213#2B345000000003C1&0#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}
SectorSize = 512
attempting to read offset 200 length 200
Read OK
Read Said 0. This disk has 9 partitions
attempting to read offset 3e00 length 600
Read OK
Sector Array is all zero, damnit Windows.
Trying backup label
attempting to read offset ee77a55e00 length 200
Read OK
attempting to read offset ee77a55800 length 600
Read OK
Sector Array found.
ShimDisk \??\USBSTOR#Disk&Ven_Logitec&Prod_&Rev_0213#2B345000000003C1&0#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}
ShimAttachWorker return 0

Got IOCTL_DISK_GET_DRIVE_LAYOUT_EX: added completion.
Changing partition label status 0 PartitionCount 4
Before FFFFB0086A15E000 of length 624
BuildGptDriveLayout: 9 real EFI, 2 non-empty populated
After FFFFB008728F4000 of length 624


$ diskpart

Microsoft DiskPart version 10.0.26100.1150

Copyright (C) Microsoft Corporation.
On computer: WIN11NEW

DISKPART> select disk 1

Disk 1 is now the selected disk.

DISKPART> list partition

  Partition ###  Type              Size     Offset
  -------------  ----------------  -------  -------
  Partition 1    Unknown            953 GB  1024 KB
  Partition 2    Unknown              8 MB   953 GB

[lundman]

OK so there is
https://github.com/openzfsonwindows/openzfs/releases/download/zfswin-2.3.1rc7/OpenZFSOnWindows-debug-2.3.1rc7shim.exe

Which has the new Shim layer, so we can explore if it is useful enough, actually works? Stable? Reliable? and so on. It does not fix the Primary Partition Entry Array yet, if Windows has blown it away, but it can read the Backup Partition Entry Array.

[dnelson-1901]

Are you seeing that 9 partitions value on disk, or is this just a logical value after something has processed the header? I checked SteamOS(Arch), Rocky, a Google cloud shell VM (Ubuntu), and FreeBSD, and assuming I'm printing the right bytes, they all seem to have it set to 128 (0x80 at offset 0x50) for their boot disks. Your 9-partition GPT header may be an outlier. Example:

(root@steamdeck) /home/deck># dd if=/dev/nvme0n1 count=1 skip=1 status=none | hexdump -C | grep 00000050
00000050  80 00 00 00 80 00 00 00  74 29 c3 38 00 00 00 00  |........t).8....|

That UEFI spec also says "A minimum of 16,384 bytes of space must be reserved for the GPT Partition Entry Array." It's weird to see NumPartitions less than 128 entries since you have to use at least 128*128 bytes on disk anyway.

[lundman]

Are you seeing that 9 partitions value on disk

hexdump -C dump.bin 
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 ff  |................|
000001c0  ff ff ee ff ff ff 01 00  00 00 ff ff 3f 01 00 00  |............?...|
000001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
00000200  45 46 49 20 50 41 52 54  00 00 01 00 5c 00 00 00  |EFI PART....\...|
00000210  0d 56 73 93 00 00 00 00  01 00 00 00 00 00 00 00  |.Vs.............|
00000220  ff ff 3f 01 00 00 00 00  22 00 00 00 00 00 00 00  |..?.....".......|
00000230  de ff 3f 01 00 00 00 00  de ea 8b b0 31 3b 11 f0  |..?.........1;..|
00000240  9f 82 7f 1a e2 ba 5d a6  02 00 00 00 00 00 00 00  |......].........|
00000250  09 00 00 00 80 00 00 00  ce 6d d7 70 00 00 00 00  |.........m.p....|
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

[dnelson-1901]

No, all of my disks have 0x80 at offset 0x250. In fact, I think one of my machines has a disk that was partitioned with openzfs-2.1.6 on Linux, and it has 0x80 also:

# zpool history | head -n2
History for 'u02':
2024-06-14.13:37:07 zpool create -O atime=off -O compress=on u02 sdb

# hexdump -C /dev/sdb | head -n 15
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 ff  |................|
000001c0  ff ff ee ff ff ff 01 00  00 00 ff ff 7f 02 00 00  |................|
000001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
00000200  45 46 49 20 50 41 52 54  00 00 01 00 5c 00 00 00  |EFI PART....\...|
00000210  7c d2 b3 97 00 00 00 00  01 00 00 00 00 00 00 00  ||...............|
00000220  ff ff 7f 02 00 00 00 00  22 00 00 00 00 00 00 00  |........".......|
00000230  de ff 7f 02 00 00 00 00  c5 5f bd 3d 29 61 42 fe  |........._.=)aB.|
00000240  a5 84 73 9d f2 79 f4 21  02 00 00 00 00 00 00 00  |..s..y.!........|
00000250  80 00 00 00 80 00 00 00  f7 23 93 a5 00 00 00 00  |.........#......|
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*

# gdisk -l /dev/sdb
GPT fdisk (gdisk) version 1.0.7

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdb: 41943040 sectors, 20.0 GiB
Model: Virtual disk    
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 3DBD5FC5-6129-FE42-A584-739DF279F421
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 41943006
Partitions will be aligned on 2048-sector boundaries
Total free space is 4029 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048        41924607   20.0 GiB    BF01  zfs-adc27b3f019238d4
   9        41924608        41940991   8.0 MiB     BF07  

[lundman]

Maybe they moved to 128 when I wasn't looking, let's take a look..

https://github.com/openzfs/zfs/blob/master/include/sys/efi_partition.h#L357-L361

So Linux did. Well then, we could as well. Just rewrite it if we come across it.

[sskras]

Oh, so it's a matter of just another defined(...) on #L357 then? :)

[lundman]

If we are ok replacing the label of foreign-OS disks yep.

[lundman]

Ah hmm it's not that useful unless I make it repair the Unix label too, so I'll make a new build

[lundman]

OK this is what we will do.

The Shim layer was fun code to write, and as a proof-of-concept does work, but it will be a pain to maintain, and make sure it works in all situations, like that of running other low-disk programs (DiskGenius, PartitionManager etc)

Linux already writes NumPart==128 by default, and they are the current biggest player.

OpenZFS on Windows writes with numPart==128 by default, now.

So, going forward, if I detect NumPart==9, which will involve deliberately reading the backup area, I will re-write it [1] with NumPart==128.

[1] That is to say, if you do an import scan, zpool import, it will read the backup area, and present any pools, but nothing more.

But if you import the pool, zpool import tank, then it will also re-label to disk, write a new label back. So we only modify the disk if we are told to .. modify.. the disk (by importing).

[lundman]

Ah Windows does something a tiny bit more bad.

It reads the GPT header, ignores the NumPartitions value, and assumes it is 128.
Calculates the CRC based on 128
The CRC will be wrong, since it should be calculated on basis of 9 partitions.
Windows assumes it is uninitialised, so it wipes the Partition Entry Array
Calculates the CRC based on 128 partitions.
Writes the new "correct" CRC to GPT header
Windows does NOT update NumPartitions to 128, but leaves it as 9.
Disk appears valid in Windows (it assumes 128)
Disk appears invalid on Unix (the CRC does not match when based on 9 partitions).

But I guess I can figure out when Windows has been here, since CRC will not match if I use
efi_gpt_NumberOfPartitionEntries (9) to calculate it, but will match if I use
efi_gpt_SizeOfPartitionEntry. (128)

[sskras]

Working around evil Windows bug is hard. Amazing research.
Does this happen on w10 too?

[lundman]

Sort of.. Windows USED to ask what to do, like when you open Partition Manager, it would point out X disks need to be initialised, and you can tick on/off, or just Cancel. So we lived with it without a care in the world.
But Win11, somewhat new, removed that optional part, and just BLASTS away! "In quiet solitude, or blasting across the alkali flats in a jet-powered, monkey-navigated…and it goes on like this"