M7: PLAN + RELEASE notes

- Renumber milestones: M7 = conformance breadth (this), M8 =
  native conv (lift Backend.conv off the BinaryBackend fallback),
  M9 = 1.0 release (was M7).
- M7 section documents the two new models and the rationale
  (vision, encoder-decoder, conv coverage) and records that
  MoE / Mixtral is deferred pending upstream Bumblebee support.
- RELEASE.md accumulates the added test suites, the shared
  ConformanceHelper, the test_helper exclude-list update, and
  the via_binary default-backend fix surfaced by ViT tiny-random.

Author: ausimian

PLAN.md

@@ -214,7 +214,47 @@ If MLX gains matmul-adjacent fusion (bias-fused matmul, attention
 fusion outside `fast::scaled_dot_product_attention`), re-run the bench
 and revisit.
 
-### M7 — 1.0 release
+### M7 — Bumblebee conformance breadth: ViT + Whisper
+
+DistilBERT (M3) and Qwen3 (M4) cover encoder-only and decoder-only
+transformers but leave three architectural shapes untested: 2-D
+convolution, encoder-decoder cross-attention, and the Bumblebee
+vision/audio pipelines. M7 closes the first two gaps.
+
+- **ViT** (`google/vit-base-patch16-224`): vision, encoder-only,
+  conv patch embedding, GELU FFN, classifier head. First suite to
+  exercise the `conv` fallback in anger (`lib/emily/backend.ex` still
+  routes `conv` through BinaryBackend as of M7 — correct but slow).
+- **Whisper** (`openai/whisper-tiny`): audio, encoder-decoder, 1-D
+  conv encoder frontend, sinusoidal position encodings, and
+  cross-attention KV-cache in the decoder.
+
+Each suite ships two tiers: a tiny-random tier that mirrors
+Bumblebee's own test (HuggingFace Transformers reference slices) and
+a full-checkpoint tier with deterministic synthetic inputs pinned
+against the real-weight forward pass on Emily. Both gated as in M3
+and M4: `:conformance` for tiny (opt in via `--only conformance`),
+per-model `:*_full` tag for full (opt in separately).
+
+Shared scaffolding (`test/support/conformance_helper.ex`) lifts the
+`setup_all` backend swap and `assert_all_close/3` out of each suite.
+
+**MoE / Mixtral deferred**: the pinned Bumblebee ref ships no
+Mixtral or MoE architecture. Track as a follow-up; revisit when
+upstream lands.
+
+**Exit:** ViT and Whisper each pass both tiers on Apple Silicon;
+`mix test --only conformance` aggregates 14 tiny-random tests across
+all four Bumblebee models.
+
+### M8 — Native conv
+
+Lift `Backend.conv` onto `Native.conv_general` (the NIF already
+exists; only the Backend callback still routes through the
+BinaryBackend fallback). Gated on the M7 ViT and Whisper suites
+staying green through the switchover.
+
+### M9 — 1.0 release
 
 - API docs, HexDocs, README with a worked Bumblebee example
 - Hex release (public), versioned per conventions (`@version` in mix.exs)

RELEASE.md

@@ -1,5 +1,72 @@
 # Release notes for next release
 
+## Added
+
+- M7 — Bumblebee conformance breadth. Two new models across four
+  new test suites extend M3 (DistilBERT) and M4 (Qwen3) beyond
+  encoder-only/decoder-only text into vision and audio.
+  - **`test/emily/conformance/vit_test.exs`**
+    (`@moduletag :conformance`) — ports `Bumblebee.Vision.VitTest`
+    verbatim: three tiny-random architectures (`:base`,
+    `:for_image_classification`, `:for_masked_image_modeling`)
+    driven with synthetic pixel input `Nx.broadcast(0.5, {1, 30, 30,
+    3})`, asserted against the same PyTorch-produced reference
+    slices Bumblebee's own suite pins. First conformance suite to
+    exercise the `conv` fallback path in anger.
+  - **`test/emily/conformance/vit_full_test.exs`**
+    (`@moduletag :vit_full`, excluded from `--only conformance`
+    because the checkpoint is ~330 MB) — loads
+    `google/vit-base-patch16-224`, runs a forward pass on a
+    deterministic constant-gray pixel tensor, asserts a pinned
+    leading-5 logits slice plus argmax == 763 (ImageNet class
+    "revolver"). Uses synthetic input rather than a checked-in JPEG
+    fixture so the repo stays free of binary assets and the
+    featurizer doesn't enter the assertion surface. Run with
+    `mix test --only vit_full`.
+  - **`test/emily/conformance/whisper_test.exs`**
+    (`@moduletag :conformance`) — ports `Bumblebee.Audio.WhisperTest`
+    verbatim: two tiny-random architectures (`:base`,
+    `:for_conditional_generation`) driven with the same
+    `Nx.sin(Nx.iota({1, 60, 80}))` mel features and decoder ids,
+    asserted against Bumblebee's reference slices. First
+    conformance suite to exercise encoder-decoder cross-attention on
+    Emily, and the first with strided 1-D conv in the encoder
+    frontend.
+  - **`test/emily/conformance/whisper_full_test.exs`**
+    (`@moduletag :whisper_full`, excluded from `--only conformance`
+    because the checkpoint is ~150 MB) — loads
+    `openai/whisper-tiny`, runs a forward pass on a synthetic 30-s
+    mel window (`sin(iota({1, 3000, 80}) * 0.01)`), asserts pinned
+    leading 3×3 logits slice + decoder-last-step argmax. Run with
+    `mix test --only whisper_full`.
+  - **`test/support/conformance_helper.ex`** — shared `use`-able
+    module lifting the `setup_all` backend-swap block and
+    `assert_all_close/3` out of DistilBERT, Qwen3, ViT, and Whisper
+    suites. Net change before the two new suites was ~zero LOC;
+    keeps future conformance additions terse.
+  - **`test_helper.exs`** — exclude list extended with `:vit_full`
+    and `:whisper_full`. Comment rewritten to document each
+    heavyweight tag and its cache footprint.
+  - **`PLAN.md`** — renumbered: M7 = conformance breadth (this),
+    M8 = native conv, M9 = 1.0 release (was M7). MoE / Mixtral
+    tracked as deferred pending upstream Bumblebee support.
+
+## Fixed
+
+- `Emily.Backend.via_binary/via_binary_tuple` — pin the default
+  backend to `Nx.BinaryBackend` for the duration of the fallback
+  `fun` call. Surfaced when ViT tiny-random exercised `conv`: the
+  helpers transferred input tensors correctly, but `Nx.conv`
+  constructs a scalar internally (`Nx.pad(t, 0, ...)` builds a
+  zero-pad tensor) and that scalar landed on whatever the current
+  global default was — `Emily.Backend`, because the conformance
+  `setup_all` installs it. BinaryBackend then saw a mixed-backend
+  operand list and crashed with a FunctionClauseError on `to_binary`.
+  Never surfaced before because `test/emily/backend_fallbacks_test.exs`
+  doesn't install Emily as the global default (tensors built with
+  `backend: Emily.Backend` opt-in) and every prior Bumblebee suite
+  had its hot-path ops off the fallback by M4.
+
 ## Changed
 
 - M6 — `mlx::core::compile` wrapping: **dropped** after Phase-1