Skip to content

Prune unused vendored protobuf sources#2066

Merged
thomasvl merged 4 commits into
apple:mainfrom
Qabbout:fix/reduce-vendored-protobuf-checkout
May 22, 2026
Merged

Prune unused vendored protobuf sources#2066
thomasvl merged 4 commits into
apple:mainfrom
Qabbout:fix/reduce-vendored-protobuf-checkout

Conversation

@Qabbout
Copy link
Copy Markdown
Contributor

@Qabbout Qabbout commented May 20, 2026

Summary

This PR reduces the amount of vendored upstream protobuf/abseil source that SwiftPM clients need to check out when depending on SwiftProtobuf.

It does this by pruning upstream files that are not used by SwiftProtobuf's package products:

  • upstream protoc built-in language generator implementations for C++, C#, Java, Kotlin, Objective-C, PHP, Python, Ruby, and Rust
  • upstream protobuf C++ test fixture directories that are not built by this SwiftPM package
  • upstream abseil/cctz timezone testdata used by upstream tests, not by SwiftProtobuf package products

It also updates scripts/UpdateProtobufSubtrees.py so future vendored protobuf/abseil refreshes continue to prune the same unused files instead of reintroducing them.

Motivation

Xcode/SwiftPM package resolution can spend a noticeable amount of time checking out SwiftProtobuf when it appears as a transitive dependency in iOS projects. A meaningful part of the package payload comes from vendored upstream protobuf/abseil files.

SwiftProtobuf's protoc executable target uses Sources/protobuf/main.cc, not upstream's full compiler/main.cc. That entry point enables external protoc-* plugins and does not register upstream's built-in language generators. The pruned generator trees therefore are not compiled or exposed by this package target.

The Swift package also has its own test/reference/fuzz/compile-test inputs under Tests/, Protos/, Reference/, FuzzTesting/, and CompileTests/; the removed upstream C++ test fixtures are not part of the SwiftPM build/test graph.

Local impact

Measured locally after this change:

  • Sources/protobuf size dropped from about 34M to 28M
  • tracked files under Sources/protobuf dropped from 2733 to 1787
  • total indexed tracked bytes dropped from about 57.2M to 53.7M

This is intentionally a conservative reduction. It does not change public Swift APIs, package products, target names, or plugin behavior.

Validation

Ran locally on macOS with Apple Swift 6.3.1:

swift package describe
swift build --target protoc
swift test

swift test result:

Executed 958 tests, with 0 failures (0 unexpected)

Notes and limitations

This reduces checkout payload for future commits/releases that include the change. It cannot change the checkout behavior of already-published tags.

A larger improvement would be to split runtime-only consumers from generator/tooling sources so iOS apps that only need the SwiftProtobuf runtime do not need to fetch protoc implementation sources at all. That would be a broader package-structure change; this PR keeps the current package structure and only removes files that are unused by the existing SwiftPM products.

@Qabbout Qabbout marked this pull request as draft May 20, 2026 18:34
@thomasvl thomasvl requested review from glbrntt and removed request for glbrntt May 20, 2026 18:39
@Qabbout Qabbout marked this pull request as ready for review May 20, 2026 18:41
@thomasvl thomasvl requested a review from glbrntt May 20, 2026 18:45
Copy link
Copy Markdown
Member

@FranzBusch FranzBusch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @thomasvl @allevato could you give this a look as well?

Copy link
Copy Markdown
Collaborator

@thomasvl thomasvl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable.

@thomasvl thomasvl added the 🔨 semver/patch No public API change. label May 21, 2026
@thomasvl
Copy link
Copy Markdown
Collaborator

@Qabbout - Can you also delete cpp, java, and ruby directories in Reference/upstream/google/protobuf/compiler/. That should fix the CI failures.

@Qabbout
Copy link
Copy Markdown
Contributor Author

Qabbout commented May 21, 2026

Done ✅ @thomasvl

@thomasvl
Copy link
Copy Markdown
Collaborator

thomasvl commented May 21, 2026

Shoot. Forgot, since we're removing Sources/protobuf/protobuf/src/google/protobuf/util/python/testdata/test_messages.proto, we also need to remove Reference/upstream/google/protobuf/util/python.

@thomasvl
Copy link
Copy Markdown
Collaborator

@Qabbout I think that will be the last thing needed. You should be able to do a make check to double check what was failing the CI.

@Qabbout
Copy link
Copy Markdown
Contributor Author

Qabbout commented May 22, 2026

Thanks for the tip @thomasvl we should be good to go.

@thomasvl
Copy link
Copy Markdown
Collaborator

Thanks again for the patience, merging now.

@thomasvl thomasvl merged commit b6b6433 into apple:main May 22, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🔨 semver/patch No public API change.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants