Update to the latest Protobuf version

[monicacecilia]

Task

Discuss and decide on updating to the latest Protobuf version.

Background

• Moving to the latest versions of Protocol Buffers will re-enable the use of the optional label to explicitly mark fields, addressing several open ticket requests.
• Investigation and testing are needed to determine whether adopting newer Protobuf features would provide meaningful benefits and whether any resulting changes would affect versioning or backward compatibility.
• Our repository is currently using Protobuf 'proto3' version 3.20.3. Compared to 'proto2', 'proto3' features the following:
  • Dropped required (only optional and repeated).
  • Every field has an implicit default (0 for ints, false for booleans, empty string for text)).
  • Simpler syntax: no defaults in .proto files (they’re implied).
  • proto 2's Extensions are mostly replaced with Any.
  • Better JSON support, better interoperability.
  • Diasmbiguate: the phenopacket-tools repository, claims in its pom.xml that we're using version, 3.21.7)
• As of May 2026, the latest stable release of the Protocol Buffers compiler (protoc) and core libraries is v35.0.
  • Because of how Google versions its language-specific packages across different package managers, you will see this release labeled slightly differently depending on the ecosystem you are targeting:
    • Java / C++ (Maven/RubyGems): 4.35.0
    • Python (PyPI): 7.35.0
    • Protoc / GitHub Releases: v35.0
• Once we make a decision to update, we must also update the Phenopackets reference implementation.

Thoughts on updating to Protobuf Editions

Historically, Protobuf required us to choose between two rigid syntax versions: proto2 and proto3. Each version had hardcoded behaviors—for example, how field presence was tracked, how default values were handled, or whether enums were open or closed. If you wanted a specific behavior from proto2, but your file was declared as proto3, you were generally out of luck. Protobuf Editions completely overhauls this model. They eliminate the rigid proto2 vs. proto3 binary and replace it with a Features-based system. An "Edition" is essentially just a packaged collection of default settings for these features. https://protobuf.dev/editions/overview/

1. Declaration Change: At the top of the.proto file, you no longer declare a syntax. Instead, you declare the year of the edition you are targeting.

Old way:
syntax = "proto3";

New Way
edition = "2023"; // or "2024", etc.

2. Features Define Behavior: Instead of the compiler deciding how your fields work based on the word "proto3", behavior is now controlled by explicit feature flags.

• For example, in proto3, the default behavior for tracking whether a field was actually set by a user (field presence) was "implicit." In Editions, you can control this using features.field_presence.
• If an Edition changes a default behavior, but you want to keep the old behavior, you simply override the feature setting. This allows you to mix and match proto2 and proto3 semantics within the exact same file.

3. Lexical Scoping (Granular Control): Under the old system, syntax = "proto3" applied to the entire file. Under the Editions model, you can define behaviors globally but override them at the individual field level using options.

edition = "2024";

// Set a file-wide default: all enums in this file are CLOSED (a proto2 behavior)
option features.enum_type = CLOSED;

message Player {
// Override field presence for just this specific field
int32 id = 1 [features.field_presence = IMPLICIT];

enum Status {
// Override the file-wide default for just this enum
option features.enum_type = OPEN;
STATUS_UNSPECIFIED = 0;
STATUS_ACTIVE = 1;
}
}

4. Incremental evolution:
The biggest problem with the jump from proto2 to proto3 was that it fractured the ecosystem. Moving a massive codebase from one to the other was very disruptive. Google designed Editions (heavily inspired by Rust editions) to fix this. New editions are planned to be released roughly once a year. When a new edition is released, it does not introduce entirely new binary behaviors; it simply changes the defaults of existing features. This means you can incrementally ratchet your codebase forward, updating one .proto file to a new edition at a time without breaking compatibility with other files using older editions.

Sources:
https://github.com/protocolbuffers/protobuf/blob/main/docs/design/editions/what-are-protobuf-editions.md
https://protobuf.dev/editions/overview/