Binary plists generated by plutil on Linux are regularly larger than plutil on macOS (26.4.1)

[nglevin]

I've noticed that binary plists generated by plutil on Linux are regularly larger than the binary plists generated by plutil on macOS 26.4.1.

The differences still exist when SWIFT_DETERMINISTIC_HASHING=1 is set in the environment, and they seem to happen more frequently depending on both the size of the strings within the plist and if the plist happens to have multiple keys.

I've tried to get this down to a minimal repro, as follows...

Repro steps:

• Given the following XML plist, where the first key has a value that's a few paragraphs of Loreum Ipsum and the second key has a one character value:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>A</key>
<string>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis finibus dapibus ligula, non vulputate neque vestibulum bibendum. Nulla interdum risus ut vehicula dapibus. Curabitur vestibulum efficitur tellus non iaculis. Maecenas eu viverra ligula, ultricies accumsan nunc. Cras id efficitur magna, vitae tincidunt eros. Fusce dictum eu nibh et rhoncus. Curabitur et libero eget risus bibendum molestie pulvinar sed ex.

Etiam viverra aliquam efficitur. Praesent porta orci dolor, ut hendrerit est posuere quis. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Duis pharetra ornare libero id elementum. Praesent vel nulla elit. In placerat velit at lorem laoreet, ut malesuada tellus rhoncus. Phasellus a maximus quam. Integer venenatis odio dui, quis cursus risus semper at. Mauris scelerisque sagittis felis a vulputate. Duis vitae aliquet risus. Pellentesque augue enim, rutrum non luctus maximus, convallis in augue. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Integer eget euismod urna. Sed ultrices felis in lacus rhoncus, id placerat neque hendrerit.

Aenean at sem ullamcorper, sodales dolor facilisis, finibus sem. Ut malesuada gravida fermentum. Nunc sagittis luctus mollis. Maecenas eget est non lacus dictum euismod. Curabitur lacinia ipsum non mi faucibus blandit. Nam ultrices suscipit erat et pharetra. Ut et leo rutrum, posuere neque sit amet, ullamcorper mauris. Sed fringilla facilisis eros, sed mattis mauris tincidunt malesuada. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Interdum et malesuada fames ac ante ipsum primis in faucibus.

Quisque condimentum malesuada neque at cursus. Aliquam elementum iaculis orci ut feugiat. Donec elementum mauris sit amet dapibus varius. Aliquam vitae eros cursus, eleifend augue eu, pulvinar ipsum. Nulla luctus pretium mauris, eget elementum eros interdum nec. Curabitur nec enim in neque sodales sagittis. Ut urna enim, consequat ut molestie in, ornare nec quam. Cras ut mi odio. Phasellus placerat turpis imperdiet nibh vestibulum, non porttitor nibh lacinia. Nam ac sodales massa, eget eleifend risus. Nullam ac nibh lobortis, elementum diam eu, lobortis tellus. Duis ut vestibulum est. Duis in dui nec nunc semper efficitur sit amet et augue.

Nam nunc neque, tincidunt sed finibus sed, laoreet vel neque. Nunc tempor ornare posuere. Praesent tristique, turpis in pulvinar dapibus, magna purus commodo magna, non facilisis eros massa eget tortor. Vivamus porta pretium quam, in hendrerit quam lacinia sit amet. Curabitur et est orci. Vivamus eu commodo risus. Nulla finibus, orci ut pharetra luctus, nulla nulla feugiat ante, id malesuada sapien lorem sit amet ante. Aenean at felis mi. Pellentesque fringilla vulputate bibendum.

</string>
<key>B</key>
<string>B</string>
</dict>
</plist>

• Save this XML plist as bug-repro-xml.plist

• Create two copies of this plist; bug-repro-xml-linux-snapshot.plist and bug-repro-xml-mac.plist

• Run plutil -convert binary1 bug-repro-xml-linux-snapshot.plist on Linux (Debian 13 in my case), using the release toolchain of Swift 6.3.0 installed via swiftly, to convert that copy of the XML plist into binary format.

• Run plutil -convert binary1 bug-repro-xml-mac.plist on macOS 26.4.1 to convert the other copy of the XML plist into binary format.

• Compare sizing data with ls -l.

Expected results:

• Sizing should be the same for these binary plists, even if the hashing leads to different contents between implementations.

Actual results:

• For the example above, I see:

$ ls -l bug-repro-xml-linux-snapshot.plist
-rwxrwxr-x 1 username primarygroup 2893 Apr 10 17:16 /path/to/bug-repro-xml-linux-snapshot.plist
$ ls -l bug-repro-xml-mac.plist
-rwxrwxr-x 1 username primarygroup 2889 Apr 10 17:16 /path/to/bug-repro-xml-mac.plist

[itingliu]

@parkera Do you happen to know what's going on since you recently made changes here?

[parkera]

Are we only looking at a deterministic size increase of 4 bytes?

[nglevin]

Unfortunately, it's not consistently that small...

Applying this to a set of plists (strings files) that I can't share right now, I've seen increases that can be 2645 -> 2954 bytes and that's about average given the size of most strings/plists files...

In one dramatic case, 563847 -> 628395. Another goes 669765 -> 755903, and a similar one goes 709440 -> 799941.

At first I was wondering if there was some manner of corruption, but as far as I've been able to determine, the outputs are still valid. They're just much larger than what the macOS infra produces with the same inputs.

I can give you more files to reproduce with when I find the time, if desired. Just let me know and I'll try to pull something together around Friday.

[nglevin]

Here's a set of inputs that show a larger size difference, same repro steps as above just replacing the XML plist with this... and this one surprised me with its result...

The original OpenStep plist
big.strings.txt

An XML plist based off the OpenStep plist (just to make sure I'm not running into odd legacy behavior)
big-xml.strings.txt

Convert one to binary1 format with the mac plutil on Tahoe 26.4.1 (25E253)... check sizing with ls -l on a Linux VM...

$ ls -l big-mac-binary.strings 
-rw-r--r-- 1 username username 383651 May  1 14:21 big-mac-binary.strings

Convert the other to binary1 format with Linux plutil on Debian 13, same setup as in the original report... check sizing with ls -l on a Linux VM...

$ ls -l big-linux-binary.strings
-rw-r--r-- 1 username username 948331 May  1 14:21 big-linux-binary.strings

Somehow the binary from Linux is even larger than the original OpenStep formatted plist... I reran these results a few times just to make sure I wasn't doing something absurd on the way.

[parkera]

Thanks for the additional data points

[finagolfin]

The OSS plutil used on linux is different than the one on macOS, so that probably causes the difference.

[parkera]

Actually they are the same now (as of #5172).

[parkera]

However, both depend on the implementation of PropertyListSerialization, which could potentially have differences.