Baseline data: block/buzz main is RED (14 flutter failures / 5 analyzer diagnostics) — relevant if fork-gates ever runs upstream suites

[mfethe1]

Hi — we maintain a separate long-running fork of block/buzz (mirror-main + product-main split) and
found your fork-contract work while surveying independent forks. The additive-only gate in
scripts/fork-gates/check-additive-only.sh + .github/workflows/fork-gates.yml (as of 468705310)
is the machine-enforced version of something we currently do by convention only, so we read it closely.

Sharing a measurement we have that seems directly relevant to anyone gating a fork against upstream:

Upstream block/buzz main is not green. On an untouched origin/main worktree we measured:

• flutter test (full mobile suite): 1647 pass / 14 fail on untouched upstream main.
  Failures cluster in channel_detail_page_test.dart (4), channels_page_test.dart (4),
  search_page_test.dart (3), remind_me_later_sheet_test.dart (3).
• flutter analyze: 5 diagnostics on untouched upstream main.
• Rust relay side also carries a stable set of pre-existing failures (media/admin auth-gate tests,
  plus 2 order-dependent flaky ones).

Measurement method, because this part bit us: Flutter's default text reporter truncates the failing
list (... and N more). Any grep-based comparison silently compares partial sets and produces false
greens. We now always run flutter test --reporter json, parse testDone events where
result != success && hidden != true, and diff sorted name sets in both directions. For the
analyzer we compare normalized signatures (severity|rule|path|line) rather than counts — counts hide
one-for-one substitutions. We re-baseline whenever upstream churns; an upstream change to
channel_detail_page_test.dart recently made an older baseline untrustworthy.

Practical implication for a fork gate: a red-on-upstream baseline means pass/fail or count-based
assertions can't distinguish "our change broke it" from "upstream is already broken", so the gate has
to diff failure name sets against a pinned upstream baseline rather than assert green.

Open question, genuinely curious rather than a suggestion: how do you intend fork-gates to treat
upstream test state? Your current gates assert structural properties (additive-only diff,
must-survive set) which sidestep this entirely — that may well be deliberate. If you ever extend the
gate to run upstream suites on contrib/* branches, does the CARRIED_PATCHES lane need a pinned
upstream baseline SHA to diff against, or would you keep verification purely structural?

Happy to share the exact JSON-reporter parsing snippet and our current baseline name set if useful.

---