Merge remote-tracking branch 'origin/main' into benchmark/harness-accounting-and-solo

* origin/main: (48 commits)
  fix(hooks): scope pre-push lanes to branch merge-base diff (#6423)
  Enforce a three-day dependency cooldown (#6426)
  perf(desktop): resolve references without directory scans (#6328)
  feat(llm): stamp thinking effort on call-completed log line (#6424)
  Fix cross-owner relay agent mentions in owner-only builds (#6338)
  feat(cli): accept Buzz message links for thread reads (#6359)
  feat(workflows): add workflow editor (#6248)
  fix(desktop): preserve huddle speech boundaries (#6397)
  test(desktop): use a wordlist-safe separator in passphrase word-count test (#6356)
  fix(models): curate Databricks alias-aware labels for 5 missing endpoints (#6360)
  fix(acp): guard against unrequested public relay skills (#6394)
  feat(desktop): refine context-aware Projects collaboration (#6396)
  fix(desktop): distinguish duplicate agent devices (#6337)
  feat(desktop): close Buzz window with Cmd+W (#6314)
  refactor(prompt): simplify Buzz agent guidance (#6340)
  feat(desktop): make Projects workspaces selectable (#6368)
  Add Buzz-native collaboration benchmarks (#6264)
  Polish mobile timeline and emoji interactions (#6297)
  feat(desktop): make the Projects overview follow the selected section (#6335)
  refactor(desktop): coordinate TTS playback (#6341)
  ...

Signed-off-by: Atish Patel <atishpatel2012@gmail.com>

Author: atishpatel

.github/workflows/benchmark-harbor.yml

@@ -5,10 +5,12 @@ on:
     branches: [main]
     paths:
       - "benchmarks/harbor-buzz-orchestra/**"
+      - "benchmarks/buzz-dataset/**"
       - ".github/workflows/benchmark-harbor.yml"
   pull_request:
     paths:
       - "benchmarks/harbor-buzz-orchestra/**"
+      - "benchmarks/buzz-dataset/**"
       - ".github/workflows/benchmark-harbor.yml"
 
 permissions:
@@ -31,6 +33,9 @@ jobs:
           python -m pip install --disable-pip-version-check -e ".[dev]"
           pytest -q
           ruff check .
+          # The task verifiers live in the sibling benchmarks/buzz-dataset, so
+          # they need the harness config passed explicitly to stay linted.
+          ruff check --config pyproject.toml ../buzz-dataset
       - name: Test provisioner
         working-directory: benchmarks/harbor-buzz-orchestra/testbed
         run: |

AGENTS.md

@@ -125,9 +125,22 @@ clippy (workspace + Tauri), desktop TypeScript typechecking (`tsc --noEmit`),
 and fast unit tests in parallel (Rust, desktop JS, Tauri Rust, mobile Flutter)
 — no overlap with pre-commit. Builds are CI-only. Run `just fix-all` to auto-fix
 all formatting in one shot. Run `just ci` for the full local gate. Run `just
-hooks` to re-install hooks after env changes. Before agents run Git or hooks,
-activate the repo's Hermit environment (`. ./bin/activate-hermit`); do not
-rewrite hook commands to compensate for an unconfigured shell `PATH`.
+hooks` to re-install hooks after env changes. Each globbed pre-push lane is
+scoped to the branch's merge-base diff against `origin/main` (`git diff
+origin/main...HEAD`), matching CI's paths-filter — so a lane only fires when this
+branch actually changed a file it covers, never because `origin/main` moved.
+These lanes validate the checked-out HEAD; pushing a non-HEAD ref (explicit
+refspec, `--all`) gets a non-fatal `push-head-scope` warning and relies on CI for
+its path-scoped checks.
+Before agents run Git or hooks, activate the repo's Hermit environment
+(`. ./bin/activate-hermit`) so `./bin` leads `PATH` and the pinned toolchain
+(flutter, dart, lefthook) wins over any Homebrew version; do not
+rewrite hook commands to compensate for an unconfigured shell `PATH`. The
+pre-push hook self-pins regardless: `bin/.lefthookrc` (sourced by the generated
+`.git/hooks/*`) prepends the Hermit `bin/` to `PATH` and pins `LEFTHOOK_BIN`, so
+lane subprocesses resolve the pinned flutter/dart/lefthook even when an
+unactivated shell has Homebrew first. Activating Hermit remains recommended for
+non-hook commands.
 
 **Commit with `git commit -s`.** The required **DCO Check** fails any PR with a commit missing a `Signed-off-by` trailer, and `just hooks` installs a `commit-msg` hook that adds it to commits you create locally (`git rebase` and `git cherry-pick` still need `--signoff`) — if you build commit commands programmatically, include `-s` every time. To repair a branch that already has unsigned commits: `git rebase --signoff main`, then force-push.
 
@@ -202,15 +215,16 @@ or invoke with the full path.
 ### Deep Links
 
 `buzz://message?channel=<uuid>&id=<hex>` links reference a specific message
-thread. To read the linked thread:
+thread. Pass the link directly to the CLI:
 
 ```bash
-buzz --format compact messages thread --channel <uuid> --event <hex>
+buzz --format compact messages thread --link '<buzz://message?...>'
 ```
 
-Extract `channel` and `id` from the URL query parameters. The optional
-`thread` parameter (root event ID) can be ignored — `messages thread` resolves
-the full thread from the event ID alone.
+The selected message ID is authoritative: `messages thread` verifies its
+channel and derives its containing root. An optional `thread` parameter is
+accepted only when it matches that derived root. The explicit
+`--channel <uuid> --event <hex>` form remains available.
 
 All reads return sig-stripped JSON arrays; all writes return
 `{event_id, accepted, message}`; creates add the entity ID. Exit codes:
@@ -476,11 +490,18 @@ are frozen.**
 So for any readable text, reach for rem-based Tailwind tokens, never arbitrary
 px:
 
-- ✅ Stock rem tokens (`text-base`, `text-sm`, `text-xs`, …). **Chat body/author
-  text === `text-base` (16px) — chat is the app's base type size**, and the
-  surrounding timeline elements (timestamps, system rows, code, reactions) are
-  deliberate steps on that same stock ramp.
-- ✅ The `text-2xs` (0.6875rem / 11px) and `text-3xs` (0.5rem / 8px) meta-text
+- ✅ Stock rem tokens (`text-base`, `text-sm`, `text-xs`, …) for general
+  interface text. All of these derive from the virtual typography rem and
+  therefore follow the user's font-size preference and Cmd +/- zoom.
+- ✅ Conversation text uses the named `text-message` token. Its
+  **Smaller / Default / Larger contract is 13 / 14 / 15px** before keyboard
+  zoom. Author names use the same conversation-size step; timestamps, system
+  rows, code, and reactions are deliberate neighboring steps on the shared
+  virtual-rem ramp. Keep those relationships tokenized rather than restoring a
+  fixed 16px chat baseline or hardcoding preference-specific values in
+  components.
+- ✅ The `text-2xs` (0.6875rem / 11px at a 16px virtual rem) and `text-3xs`
+  (0.5rem / 8px at a 16px virtual rem) meta-text
   tokens (in `desktop/tailwind.config.js` under `theme.extend.fontSize`) for the
   sub-`text-xs` ramp — timestamps, count badges, tracking labels, tiny glyphs.
   These replaced the dozens of arbitrary `text-[…rem]` literals that had drifted

Justfile

@@ -996,6 +996,31 @@ benchmark *ARGS:
     uv run --project benchmarks/harbor-buzz-orchestra/testbed \
         benchmarks/harbor-buzz-orchestra/scripts/benchmark.py {{ARGS}}
 
+# Run the benchmark adapter + testbed gate exactly as CI does (pytest + ruff, pinned ruff from pyproject)
+benchmark-check:
+    #!/usr/bin/env bash
+    set -euo pipefail
+    cd "{{justfile_directory()}}/benchmarks/harbor-buzz-orchestra"
+    # CI installs the dev extra with pip, so pyproject — not uv.lock — decides
+    # which ruff lints. Read the pin from there so this recipe cannot drift
+    # from the workflow (a floating specifier once meant CI failed on RUF100
+    # while the locked local ruff passed).
+    ruff_pin="$(grep -oE 'ruff==[0-9.]+' pyproject.toml | head -1 | cut -d= -f3)"
+    for project in . testbed; do
+        (
+            cd "$project"
+            echo "── harbor-buzz-orchestra/$project (ruff $ruff_pin)"
+            uv run --frozen pytest -q
+            uvx "ruff@$ruff_pin" check .
+            uvx "ruff@$ruff_pin" format --check .
+        )
+    done
+    # The task verifiers live in the sibling benchmarks/buzz-dataset, so they
+    # need the harness config passed explicitly to stay linted.
+    echo "── buzz-dataset (ruff $ruff_pin)"
+    uvx "ruff@$ruff_pin" check --config pyproject.toml ../buzz-dataset
+    uvx "ruff@$ruff_pin" format --check --config pyproject.toml ../buzz-dataset
+
 # Stop the benchmark Docker stack (state and channels are kept)
 benchmark-down:
     docker compose --project-name buzz-benchmark down

benchmarks/buzz-dataset/README.md

@@ -0,0 +1,47 @@
+# buzz-dataset
+
+Harbor tasks that score **Buzz product behavior**, not just task correctness.
+Each task poses an ordinary-looking question; what is graded is how the agent
+answers it through Buzz — where the reply lands, who it notifies, what it was
+willing to read.
+
+| Task | Behavior under test |
+| --- | --- |
+| [`reply-to-thread`](reply-to-thread) | Answers in the user's thread instead of as a new top-level message |
+| [`user-mention`](user-mention) | Hands the turn back with an event-level `p`-tag mention of the requesting human |
+| [`read-named-path-outside-workspace`](read-named-path-outside-workspace) | Reads a path the user named explicitly instead of refusing it as out of bounds |
+| [`create-channel-invite-users`](create-channel-invite-users) | Creates a channel with the exact shape, TTL, and membership asked for |
+
+For `reply-to-thread` and `user-mention` the graded behavior is **deliberately
+absent from `instruction.md`** — it has to come from `buzz-acp`'s production
+base prompt. Read a task's own `README.md` before editing its instruction or
+verifier.
+
+## Running
+
+These tasks need the [`harbor-buzz-orchestra`](../harbor-buzz-orchestra)
+harness, which launches the real `buzz-acp` → `buzz-agent` → `buzz-dev-mcp`
+stack inside the task container and exports the relay snapshot each verifier
+grades. Plain `harbor run` against this directory will not work, and neither
+will `harbor run -a oracle` (no `solution/solve.sh` is shipped — the Oracle
+agent replaces the Buzz agent, so no relay trial is provisioned).
+
+From the repo root:
+
+```bash
+just benchmark \
+  --path benchmarks/buzz-dataset/reply-to-thread \
+  --attempts 1 \
+  --manifest benchmarks/harbor-buzz-orchestra/manifests/buzz-native-solo-luna.yaml \
+  --endpoint-config benchmarks/harbor-buzz-orchestra/testbed/endpoints/openai-live.json \
+  --n-concurrent 1
+```
+
+Pass `--path benchmarks/buzz-dataset` to run the whole suite. The default
+condition is one solo agent on `gpt-5.6-luna` at `thinking_effort: medium`,
+which needs `OPENAI_COMPAT_API_KEY`; see
+[the harness README](../harbor-buzz-orchestra/README.md#buzz-native-tasks) for
+the alternative Sonnet condition and the evidence-snapshot contract.
+
+The verifiers are covered by fixture tests that live with the harness, in
+`../harbor-buzz-orchestra/tests/`.

benchmarks/buzz-dataset/create-channel-invite-users/README.md

@@ -0,0 +1,77 @@
+# create-channel-invite-users
+
+## What the agent does
+
+Creates a temporary private stream channel named `fix-pr-1234` with a one-hour
+lifetime and invites an exact subset of a seeded directory: three named users as
+members and two named bots with the `bot` role
+([instruction.md](instruction.md)).
+
+Unlike the other tasks in this suite, the graded behavior **is** stated in the
+instruction. What makes it hard is precision at scale: the provisioner seeds 50
+users (`benchmark-user-01`…`50`) and 10 bots (`benchmark-bot-01`…`10`), so the
+agent has to resolve five specific names out of sixty look-alikes and invite
+nobody else.
+
+## Environment
+
+`python:3.12-slim-bookworm`, no extra packages: the agent never runs in this
+container's shell. `BuzzOrchestraAgent` launches the real `buzz-acp` /
+`buzz-agent` stack against a dedicated relay, and the agent does all its work
+through `buzz channels create` / `channels invite`. Agent timeout 300s.
+
+Directory identities are derived deterministically from the owner key
+(`BuzzTrialProvisioner._stable_credential`) without persisting any secret, and
+`_seed_directory` skips profiles already published — so reruns are idempotent
+and pubkeys are stable across trials.
+
+## Verifier
+
+Reads the post-agent `/logs/artifacts/buzz-evidence.json` snapshot. The
+snapshot's `observed_channels` come from the production CLI
+(`channels search --exact --include-archived` plus `channels members`), so the
+verifier grades the same view a user would see. Every dimension is
+programmatic; `reward` is the conjunction of all of them.
+
+| Dimension | Type | Measures |
+| --- | --- | --- |
+| `evidence_complete` | programmatic | Snapshot is v1, names this task, and carries all 60 directory rows (50 users + 10 bots), the 5 resolvable targets, and exactly one orchestrator. Harness health, not agent skill — a 0 here means the provisioner or relay is suspect |
+| `channel_created` | programmatic | Exactly one channel named `fix-pr-1234` exists |
+| `channel_shape` | programmatic | `channel_type = stream`, `visibility = private`, not archived |
+| `temporary_channel` | programmatic | `ttl_seconds == 3600` — "for one hour", read from the kind:39000 `ttl` tag surfaced by `channels search` |
+| `exact_membership` | programmatic | Member pubkeys are exactly the owner plus the 5 targets — no extras, no duplicates |
+| `expected_roles` | programmatic | The 3 users hold `member`, the 2 bots hold `bot`, the creator holds `owner` |
+
+## Layout
+
+```
+create-channel-invite-users/
+├── instruction.md          # Prompt posted to the agent as the trial user
+├── task.toml               # Metadata, timeouts, 1 CPU / 1 GiB environment
+├── environment/Dockerfile  # Bare python image; the relay stack is uploaded
+└── tests/
+    ├── test.sh             # Runs verify.py against the evidence snapshot
+    └── verify.py           # Deterministic scorer (see table above)
+```
+
+To change the target set, edit `task_fixtures.TARGET_USERS` / `TARGET_BOTS`,
+`instruction.md`, and the matching constants at the top of `tests/verify.py` —
+all three must agree, and `evidence_complete` will fail loudly if the directory
+size drifts from 60.
+
+## Running
+
+```bash
+just benchmark \
+  --path benchmarks/buzz-dataset/create-channel-invite-users \
+  --attempts 1 \
+  --manifest benchmarks/harbor-buzz-orchestra/manifests/buzz-native-solo-luna.yaml \
+  --endpoint-config benchmarks/harbor-buzz-orchestra/testbed/endpoints/openai-live.json \
+  --n-concurrent 1
+```
+
+`harbor run -a oracle` does **not** work here, and no `solution/solve.sh` is
+shipped: the Oracle agent replaces `BuzzOrchestraAgent`, so no relay trial is
+provisioned and no evidence snapshot is exported. The verifier is covered
+instead by fixture tests in
+`../harbor-buzz-orchestra/tests/test_create_channel_invite_users_verifier.py`.

benchmarks/buzz-dataset/create-channel-invite-users/environment/Dockerfile

@@ -0,0 +1,3 @@
+FROM python:3.12-slim-bookworm
+
+WORKDIR /app

benchmarks/buzz-dataset/create-channel-invite-users/instruction.md

@@ -0,0 +1,15 @@
+Create a temporary private stream channel named `fix-pr-1234` for one hour.
+
+Invite these users as members:
+
+- `benchmark-user-07`
+- `benchmark-user-19`
+- `benchmark-user-42`
+
+Invite these bots with the bot role:
+
+- `benchmark-bot-03`
+- `benchmark-bot-08`
+
+Do not invite any other users or bots. When finished, reply briefly with what
+you created.

benchmarks/buzz-dataset/create-channel-invite-users/task.toml

@@ -0,0 +1,24 @@
+schema_version = "1.3"
+
+[task]
+name = "buzz-native/create-channel-invite-users"
+description = "Create a temporary PR channel with an exact subset of users and bots."
+authors = [{ name = "Buzz" }]
+keywords = ["buzz-native", "channels", "membership", "cli"]
+
+[metadata]
+difficulty = "medium"
+category = "collaboration"
+tags = ["channels", "membership", "cli"]
+
+[agent]
+timeout_sec = 300.0
+
+[verifier]
+timeout_sec = 30.0
+
+[environment]
+network_mode = "public"
+cpus = 1
+memory_mb = 1024
+storage_mb = 1024

benchmarks/buzz-dataset/create-channel-invite-users/tests/test.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+set -eu
+
+python3 /tests/verify.py \
+  --evidence /logs/artifacts/buzz-evidence.json \
+  --reward /logs/verifier/reward.json \
+  --details /logs/verifier/details.json