Bazel //:protobuf_python does not bundle the upb extension, so bazel consumers silently fall back to the pure-Python backend

[sc0ttbeardsley]

What version of protobuf and what language are you using?

Version: main (protoc_version 37-dev); also present on all released versions since 4.21.0

Language: Python (specifically the Bazel build — depending on the //:protobuf_python target rather than the PyPI wheel)

What supported operating system version are you using (e.g. Linux, Windows)?

Linux (Ubuntu 24.04). Note: this is Bazel build-graph behavior and is OS-independent.

What supported runtime / compiler version are you using (e.g. python version, gcc version)?

Python 3.12, Bazel 8.6.0, rules_python 1.6.0. Version-independent — it reproduces on any Python/toolchain because the cause is in the BUILD dependency graph, not runtime code.

What did you do?

Steps to reproduce the behavior:

1. Create a py_binary/py_test in a Bazel workspace that depends on protobuf's //:protobuf_python target (as opposed to installing the PyPI wheel) — e.g. via a generated
   py_proto_library, whose runtime is //:protobuf_python.
2. Do not provide the upb extension through any other dependency (no protoc-gen-validate wheel or other package that vendors google/_upb).
3. Run the target and check the active backend:

   from google.protobuf.internal import api_implementation
   print(api_implementation.Type())

4. Observed: python (the pure-Python backend).
   Expected: upb — the documented default (python/README.md: "upb … is now the default … requires no special installation").

Root cause

//:protobuf_python (defined in python/build_targets.bzl) only ever bundles the cpp backend extensions, and only under --define=use_fast_cpp_protos=true:

data = select({
    "//conditions:default": [],
    ":use_fast_cpp_protos": [
        ":google/protobuf/internal/_api_implementation.so",
        ":google/protobuf/pyext/_message.so",
    ],
}),

It never depends on the upb extension //python:_message. So google._upb._message lands on a Bazel consumer's runfiles path only if some other dependency happens to ship it; otherwise
api_implementation (which prefers upb when importable) silently falls back to pure-Python. This makes the Bazel //:protobuf_python target diverge from the published wheel, where upb
is the default.

This appears to be an oversight rather than intent:

• protobuf_python was introduced in b3cbea18e ("[Bazel] Move Python rules to //python", 2022-05-12) with only the cpp backend in its data select — ~3 weeks after a27ce12d3
  (2022-04-22) made upb the top-priority auto-selected backend in api_implementation.py.
• google/_upb has never appeared in this target's data/deps in the file's history.
• The gap is already worked around in-tree: conformance/BUILD's conformance_python binary depends on //:protobuf_python and then adds //python:_message separately with the comment
  # Make upb visible if we need it. — a per-consumer patch for exactly this.

Why fix at the source vs. the per-consumer workaround?

The workaround (adding //python:_message to each consumer, as conformance_python does) works, but has to be repeated by every downstream target and is easy to miss — the failure is
silent (you get a working but slow pure-Python backend, no error), so it's typically only noticed via a perf regression. Fixing it once in protobuf_python:

• makes the Bazel target's default backend match the PyPI wheel and the documented default, so there are no surprises for Bazel consumers;
• removes a footgun where new py_proto consumers silently regress to pure-Python;
• lets the existing conformance/BUILD workaround be deleted rather than copied;
• is a no-op for the deliberate use_fast_cpp_protos (cpp) path and for @system_python//:none / :unsupported, so it doesn't change any currently-intended behavior.

Suggested fix

Add :_message to the default deps of protobuf_python so upb is importable (and thus auto-selected) out of the box, matching the wheel. It should be omitted under
use_fast_cpp_protos (the user explicitly chose cpp) and where python is unavailable (:_message is target_compatible_with-incompatible under @system_python//:none / :unsupported).
The conformance/BUILD workaround can then be removed.