Packages with multiple targets using a build plugin can cause superfluous rebuilds

[marcprux]

Clone https://github.com/apple/swift-openapi-generator.git and open Examples/streaming-chatgpt-proxy/Package.swift in Xcode 16.2 (16C5032a) and build the fosdem-2025-demo-Package for "My Mac" (authorizing the "swift-openapi-generator" plugin when prompted). After the initial build completes, try re-building (without cleaning or making any changes) a few times and you will find something curious: most re-builds take between 5-6 seconds (on an M1 MacBook Pro with macOS 15.3.2), but every once in a while a build takes only 1 second. See the timing summary at the bottom of these build logs:

What is the difference between these builds? Scroll to the top of each of these build logs, and you will see that for the slower builds, the "Prepare Build" phase has suspicious messages like:

Removed stale file '/Users/marc/Library/Developer/Xcode/DerivedData/streaming-chatgpt-proxy-himnvcsxyvjxreayeakwjymyjzho/Build/Intermediates.noindex/fosdem-2025-demo.build/Debug/ProxyServer.build/Script-817141908514328371.sh'
Removed stale file '/Users/marc/Library/Developer/Xcode/DerivedData/streaming-chatgpt-proxy-himnvcsxyvjxreayeakwjymyjzho/Build/Intermediates.noindex/fosdem-2025-demo.build/Debug/ClientCLI.build/Script-8973115602826654042.sh'
Removed stale file '/Users/marc/Library/Developer/Xcode/DerivedData/streaming-chatgpt-proxy-himnvcsxyvjxreayeakwjymyjzho/Build/Intermediates.noindex/fosdem-2025-demo.build/Debug/ChatGPT.build/Script-17548104271755262233.sh'

The fast build (on the right) contains no such warnings.

Note that this only happens when a package contains multiple targets that use a build plugin. For example, opening Examples/hello-world-urlsession-client-example/Package.swift and re-building repeatedly will always take <1s, and there are never any warnings about "Removed stale file…".

So why is the DerivedData/…/Target.build/Script-XYZ.sh file sometimes marked as stale, and other times not – and only for packages that contain multiple targets using a build plugin? Comparing the warning messages for the two slow builds in the left panes above, we see that the scripts have different filenames.

Build 1

Removed stale file '…/ProxyServer.build/Script-6658080100055822918.sh'
Removed stale file '…/ClientCLI.build/Script-14912566104187546689.sh'
Removed stale file '…/ChatGPT.build/Script-11658592523709397394.sh'

Build 2

Removed stale file `…/ProxyServer.build/Script-817141908514328371.sh'
Removed stale file '…/ClientCLI.build/Script-8973115602826654042.sh'
Removed stale file '…/ChatGPT.build/Script-17548104271755262233.sh'

So we might suspect that the removal of these files is somehow tainting the build and causing the project to re-execute the targets (and their plugins) rather than efficiently eliding the unnecessary re-builds. Keeping an eye on the ${HOME}/Library/Developer/Xcode/DerivedData/streaming-chatgpt-proxy-himnvcsxyvjxreayeakwjymyjzho/Build/Intermediates.noindex/fosdem-2025-demo.build/Debug/ProxyServer.build/ folder between builds shows that sometimes, but not always, the script file is re-generated with a new identifier. But the identifier isn't random, it appears to cycle through a narrow set of values.

Grabbing the Script-*.sh files for later analysis in between builds demonstrates this:

zap Debug/ProxyServer.build % mkdir /tmp/xcode-scripts/
zap Debug/ProxyServer.build % cp -av Script-*.sh /tmp/xcode-scripts/
Script-18433590652374256892.sh -> /tmp/xcode-scripts/Script-18433590652374256892.sh
zap Debug/ProxyServer.build % cp -av Script-*.sh /tmp/xcode-scripts/
Script-6658080100055822918.sh -> /tmp/xcode-scripts/Script-6658080100055822918.sh
zap Debug/ProxyServer.build % cp -av Script-*.sh /tmp/xcode-scripts/
Script-817141908514328371.sh -> /tmp/xcode-scripts/Script-817141908514328371.sh
zap Debug/ProxyServer.build % cp -av Script-*.sh /tmp/xcode-scripts/
Script-817141908514328371.sh -> /tmp/xcode-scripts/Script-817141908514328371.sh
zap Debug/ProxyServer.build % cp -av Script-*.sh /tmp/xcode-scripts/
Script-4219508519868706161.sh -> /tmp/xcode-scripts/Script-4219508519868706161.sh
zap Debug/ProxyServer.build % cp -av Script-*.sh /tmp/xcode-scripts/
Script-817141908514328371.sh -> /tmp/xcode-scripts/Script-817141908514328371.sh
zap Debug/ProxyServer.build % cp -av Script-*.sh /tmp/xcode-scripts/
Script-6658080100055822918.sh -> /tmp/xcode-scripts/Script-6658080100055822918.sh
zap Debug/ProxyServer.build % cp -av Script-*.sh /tmp/xcode-scripts/
Script-817141908514328371.sh -> /tmp/xcode-scripts/Script-817141908514328371.sh

These script files contain the generated plugin execution script with sandboxing. E.g.:

zap Debug/ProxyServer.build % cat /tmp/xcode-scripts/Script-18433590652374256892.sh
#!/bin/bash
/usr/bin/sandbox-exec -p "(version 1)
(deny default)
(import \"system.sb\")
(allow file-read*)
(allow process*)
(allow mach-lookup (global-name \"com.apple.lsd.mapdb\"))
(allow file-write*
    (subpath \"/private/tmp\")
    (subpath \"/private/var/folders/zl/wkdjv4s1271fbm6w0plzknkh0000gn/T\")
)
(deny file-write*
    (subpath \"/opt/src/github/marcprux/swift-openapi-generator/Examples/streaming-chatgpt-proxy\")
)
(allow file-write*
    (subpath \"/Users/marc/Library/Developer/Xcode/DerivedData/streaming-chatgpt-proxy-himnvcsxyvjxreayeakwjymyjzho/SourcePackages/plugins/streaming-chatgpt-proxy.output/ProxyServer/OpenAPIGenerator\")
    (subpath \"/private/var/folders/zl/wkdjv4s1271fbm6w0plzknkh0000gn/T/TemporaryItems/NSIRD_Xcode_qNrl14\")
    (subpath \"/private/var/folders/zl/wkdjv4s1271fbm6w0plzknkh0000gn/T/TemporaryItems\")
)
" "/${BUILD_DIR}/${CONFIGURATION}/swift-openapi-generator" generate /opt/src/github/marcprux/swift-openapi-generator/Examples/streaming-chatgpt-proxy/Sources/ProxyServer/openapi.yaml --config /opt/src/github/marcprux/swift-openapi-generator/Examples/streaming-chatgpt-proxy/Sources/ProxyServer/openapi-generator-config.yaml --output-directory /Users/marc/Library/Developer/Xcode/DerivedData/streaming-chatgpt-proxy-himnvcsxyvjxreayeakwjymyjzho/SourcePackages/plugins/streaming-chatgpt-proxy.output/ProxyServer/OpenAPIGenerator/GeneratedSources --plugin-source build

Diffing a couple of the files that we saved shows that the only difference is with the TemporaryItems/NSIRD_Xcode_XXXXXX path:

zap Debug/ProxyServer.build % diff /tmp/xcode-scripts/Script-18433590652374256892.sh /tmp/xcode-scripts/Script-6658080100055822918.sh 
17d16
<     (subpath \"/private/var/folders/zl/wkdjv4s1271fbm6w0plzknkh0000gn/T/TemporaryItems/NSIRD_Xcode_qNrl14\")
18a18
>     (subpath \"/private/var/folders/zl/wkdjv4s1271fbm6w0plzknkh0000gn/T/TemporaryItems/NSIRD_Xcode_FWBYcQ\")

So I suspect that the identifier of the script is based on some hash of the contents (Script-HASH.sh), and when the contents of the script change, the script hash identifier changes, the file is marked as stale, and the whole build is tainted, leading to the superfluous rebuild.

Why is the temporary folder sometimes different and other times the same? Sometimes the script will keep the same NSIRD_Xcode_XXXXXX identifier between rebuilds, and other times it changes. And, as previously mentioned, for a package that contains only a single target that uses a plugin, this identifier never changes. This leads me to suspect that these IDs are stable, but handed out on a timing basis, and that parallelization of the "Prepare Build" phase leads to an indeterminacy in the order in which the scripts are generated.

This should be fixed so the content of these scripts are always identical, which should lead to the file identifiers remaining stable, which should maintain determinism and eliminate the unnecessary rebuilds.

[marcprux]

I'll point out that while the timing difference in this reproducer (1 second vs. 5 seconds) isn't all that severe for a small project, in the case that triggered this investigation (skiptools/skip#372), we are seeing upwards of 1 minute rebuild times as a result of this bug.

[marcprux]

One other thing I wonder, is why does the script generate a sandbox config that authorizes file-write to both the temporary directory as well as a sub-directory of that same temporary directory?

(allow file-write*
    (subpath \"/Users/marc/Library/Developer/Xcode/DerivedData/streaming-chatgpt-proxy-himnvcsxyvjxreayeakwjymyjzho/SourcePackages/plugins/streaming-chatgpt-proxy.output/ProxyServer/OpenAPIGenerator\")
    (subpath \"/private/var/folders/zl/wkdjv4s1271fbm6w0plzknkh0000gn/T/TemporaryItems/NSIRD_Xcode_qNrl14\")
    (subpath \"/private/var/folders/zl/wkdjv4s1271fbm6w0plzknkh0000gn/T/TemporaryItems\")
)

Isn't the subfolder permission automatically granted by the permission to write to the parent folder? If the unnecessary NSIRD_Xcode_qNrl14 sub-folder wasn't there, then the script files would all have the same identifier hash, and this problem might just go away…

[neonichu]

I think those paths in the sandbox profile are generated by this code here: https://github.com/swiftlang/swift-package-manager/blob/bfb8857e68802979e935d8865d4071a2194a7a1f/Sources/Basics/Sandbox.swift#L223-L232

The comment makes me think we already made an attempt to get these paths more stable.

[neonichu]

It seems like swiftlang/swift-package-manager@5e22d1b is in Swift 6.1 only which is aligned with Xcode 16.3.

Have you tried whether this issue might have improved with the 16.3 betas?

[marcprux]

Have you tried whether this issue might have improved with the 16.3 betas?

I'll try it out and let you know…

[marcprux]

Yes, this is fixed in Xcode 16.3. Thanks!