Skip to content

Commit 9cb5f33

Browse files
committed
Drop the side panel: host the conversation in the page, speech-only
The product is voice-in/voice-out; the panel existed mainly to give the speech APIs a document and the captions a home. Move the whole session into the content script instead: - The orchestrator, conversation machine, and STT now run in the puzzle page. The mic prompt/grant belongs to nytimes.com, and the session dies with the page on reload/navigation — both accepted by design. - TTS stays on chrome.tts (immune to page autoplay rules), which content scripts can't call, so speak/cancel are relayed to the service worker over the session port (new speech/remote-tts-port.js, unit-tested against a fake port). - No visual UI at all: captions are replaced by page-console diagnostics ([CrosswordChat] lines). REQ-SPCH-007/008 rewritten accordingly; no DOM injection, no server logging (REQ-NFR-001). - Icon click on an NYT page without the content script (article pages) now gets the "I don't see a crossword" line spoken by the service worker itself, since there is no panel to explain. - The selector probe stays reachable via a cc:probe message from the service-worker console (MT-01 updated with the one-liner). - Requirements, manual tests, architecture, feasibility, and README updated to match; sidepanel/ sources and manifest entries removed. Verified: 121 unit/integration tests + trace green (82/82 covered); Playwright smoke test of the built extension against the offline fixture exercises start → clue readout via the tts relay → listen → retry-once → session end with badge cleanup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QDWV85h1F7E7tosX5RAtCF
1 parent f6059dd commit 9cb5f33

15 files changed

Lines changed: 342 additions & 407 deletions

File tree

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,10 @@ Full analysis in [`docs/FEASIBILITY.md`](docs/FEASIBILITY.md):
1414
default speech recognition runs on Google's servers (part of the browser, not ours); recent
1515
Chrome can do it on-device. Enforced by an architecture test (no network primitives in source).
1616
2. **Speech APIs available to extensions?** Yes, both directions: `webkitSpeechRecognition`
17-
(with an n-best list we exploit for homophones) and `chrome.tts`/`speechSynthesis`. They need
18-
a document context, which is why the conversation lives in the side panel.
17+
(with an n-best list we exploit for homophones) and `chrome.tts`/`speechSynthesis`. Recognition
18+
needs a document context, which is why the conversation lives in the puzzle page itself
19+
(content script) — the extension is speech-only, with no visual UI. Speaking goes through the
20+
service worker's `chrome.tts`.
1921
3. **Can we type into the NYT grid?** Yes via simulated click+keydown per cell, verified by
2022
re-reading the DOM. One real risk (`isTrusted` filtering) with three documented fallbacks —
2123
validated early by manual test MT-02.
@@ -33,7 +35,7 @@ Full analysis in [`docs/FEASIBILITY.md`](docs/FEASIBILITY.md):
3335
| `extension/src/matching/` | Pure speech-to-answer matching: normalization, digits→words, homophone dictionary, length/collision verdicts, command lexicon. |
3436
| `extension/src/conversation/` | Pure dialog policy: state machine, next-clue strategies, all English strings + clue verbalizer. |
3537
| `extension/src/speech/` | TTS/STT wrappers over browser APIs, injectable for tests. |
36-
| `extension/src/{app,background,content,sidepanel}/` | Wiring: orchestrator, icon toggle, message routing, captions UI. |
38+
| `extension/src/{app,background,content}/` | Wiring: orchestrator, icon toggle, in-page session host, TTS relay. |
3739
| `tests/fixtures/fake-nyt/` | A faithful miniature of the NYT crossword page (same classes, same keyboard model) — integration-test target and offline demo stage. |
3840
| `tools/trace.mjs` | Requirements-coverage enforcer (`npm run trace`). |
3941
| [`docs/RELEASING.md`](docs/RELEASING.md) | Versioning (current: **0.9.0**), CI workflows, release process, Chrome Web Store deployment setup. |

docs/ARCHITECTURE.md

Lines changed: 42 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -9,28 +9,31 @@ plumbing are five different problems; none imports another's internals.
99
```
1010
┌─────────────────────────── Chrome ───────────────────────────────┐
1111
│ │
12-
│ Service worker (background/) Side panel (sidepanel/) │
13-
│ "switchboard" "brain + voice" │
14-
│ · icon click → open/close panel · orchestrator (app/) │
15-
│ · one-session bookkeeping, badge · conversation machine │
16-
│ · relays close requests · matching │
17-
│ ▲ │ · speech ports (TTS/STT) │
18-
│ │ port │ open/close · captions UI │
19-
│ ▼ ▼ │ │
20-
│ ┌──────────────────────────────┐ │ chrome.tabs │
21-
│ │ NYT crossword tab │◄────────────┘ .sendMessage │
22-
│ │ content script "hands" │ │
23-
│ │ · page-adapter (read/write) │ │
24-
│ │ · watcher (mutations) │ │
25-
│ │ main-world script (probe) │ │
26-
│ └──────────────────────────────┘ │
12+
│ Service worker (background/) "switchboard + mouth" │
13+
│ · icon click → start/stop the in-page session │
14+
│ · one-session bookkeeping, badge │
15+
│ · chrome.tts relay (speak/cancel) — content scripts can't │
16+
│ ▲ │
17+
│ │ port (session registration + speak/done/cancel) │
18+
│ ▼ │
19+
│ ┌──────────────────────────────────────────────┐ │
20+
│ │ NYT crossword tab — content script │ │
21+
│ │ "brain + ears + hands", speech-only UI │ │
22+
│ │ · orchestrator (app/) · conversation machine│ │
23+
│ │ · matching · STT port (mic, page origin) │ │
24+
│ │ · page-adapter (read/write/watch) │ │
25+
│ │ · console diagnostics (no visual UI) │ │
26+
│ │ main-world script (probe) │ │
27+
│ └──────────────────────────────────────────────┘ │
2728
└──────────────────────────────────────────────────────────────────┘
2829
```
2930

30-
- **Why the brain is in the side panel:** speech APIs need a document (not a service worker), the
31-
mic grant belongs to the extension origin, and captions need a UI surface. See FEASIBILITY §2.
32-
- **Why the content script is thin:** everything that touches the NYT DOM is a dumb, verifiable
33-
command (`snapshot`, `enter`, `select`, `clear`, `probe`, `watch`). No decisions are made there.
31+
- **Why the brain is in the page (content script):** the product is deliberately speech-only — no
32+
panel or popup to host or babysit. Speech recognition needs a document; the page is one. The mic
33+
grant belongs to nytimes.com (prompted once, on first session), and the session's lifetime is the
34+
page's — reload/navigate ends it, which is the wanted behavior. See FEASIBILITY §2.
35+
- **Why TTS stays in the service worker:** `chrome.tts` (immune to page autoplay rules) is not
36+
exposed to content scripts, so the session relays speak/cancel over its port.
3437

3538
## 2. Module map (source of truth for "what goes where")
3639

@@ -56,19 +59,19 @@ extension/src/
5659
strategies.js next-clue selection: list-order, most-filled
5760
phrases.js every English string; clue verbalizer (italics/brackets/?/blank/length)
5861
speech/ Browser speech plumbing. Injectable for tests.
59-
tts-port.js chrome.tts | speechSynthesis → speak()/cancel()
62+
tts-port.js chrome.tts | speechSynthesis → speak()/cancel() (used by the service worker)
63+
remote-tts-port.js same contract from the content script: relays speak/cancel over the port
6064
stt-port.js webkitSpeechRecognition → listenOnce() n-best, error taxonomy, mic preflight
6165
app/
6266
orchestrator.js executes machine actions via ports/pageClient; owns the event loop
6367
background/
64-
service-worker.js icon toggle, single-session bookkeeping, badge
68+
service-worker.js icon toggle, single-session bookkeeping, badge, chrome.tts relay
6569
content/
66-
content-script.js message router → page-adapter; inert until asked (REQ-NFR-004)
70+
content-script.js hosts the session: orchestrator + STT + page-adapter, console diagnostics;
71+
inert until asked (REQ-NFR-004)
6772
main-world.js optional in-page probe helper (window.gameData presence)
68-
sidepanel/
69-
panel.html/js/css boot, captions, probe button, port to service worker
7073
shared/
71-
messages.js message type constants shared by panel/content/background
74+
messages.js message type constants shared by content/background
7275
```
7376

7477
**Dependency direction (enforced by review + arch test):**
@@ -111,24 +114,27 @@ the verbalizer.
111114
dispatch(event):
112115
{state, actions} = machine.reduce(state, event)
113116
for a of actions:
114-
SAY → caption + tts.speak(phrases.render(a.say)) → dispatch(TTS_DONE)
117+
SAY → console line + tts.speak(phrases.render(a.say)) → dispatch(TTS_DONE)
115118
LISTEN → stt.listenOnce() → dispatch(HEARD | STT_ERROR)
116119
ENTER → pause watcher → pageClient.enterAnswer(...) → dispatch(ENTRY_RESULT)
117120
SELECT_CLUE → pageClient.selectClue(...)
118-
END → teardown (cancel tts/stt, stop watcher, close panel)
121+
END → teardown (cancel tts/stt, stop watcher, disconnect port)
119122
```
120123
Strictly sequential; no queues, no races. Watcher events are suppressed while we write (our own
121124
typing must not look like user activity — REQ-NAV-008's echo-loop clause).
122125

123-
## 5. Message protocol (panel ⇄ content ⇄ background)
126+
## 5. Message protocol (content ⇄ background)
124127

125-
- Panel → content (via `chrome.tabs.sendMessage(tabId, ...)`):
126-
`{type:'cc:snapshot'}` → Snapshot · `{type:'cc:enter', cells:[{index,letter}]}`
127-
`{ok,snapshot}` · `{type:'cc:select', clueId}` · `{type:'cc:clear', cellIndices}` ·
128-
`{type:'cc:probe'}` → report · `{type:'cc:watch'} / {type:'cc:unwatch'}`
129-
- Content → panel (broadcast): `{type:'cc:page-event', kind, snapshot, tabId}`
130-
- Panel ⇄ background (long-lived Port `cc-panel`): hello/`{tabId}` handshake; `close` request;
131-
disconnect = session over (badge cleared).
128+
Page operations need no messages anymore — the orchestrator calls the page adapter directly
129+
(same document). What's left is session control and the TTS relay:
130+
131+
- Background → content (via `chrome.tabs.sendMessage`): `{type:'cc:start'}` — begin a session.
132+
Debug-only, from the service-worker console: `{type:'cc:ping'}` · `{type:'cc:snapshot'}`
133+
Snapshot · `{type:'cc:probe'}` → report (MT-01).
134+
- Content ⇄ background (long-lived Port `cc-session`): connect = session registered (badge ON);
135+
`{type:'cc:speak', id, text}``chrome.tts``{type:'cc:speak-done', id}` ·
136+
`{type:'cc:tts-cancel'}` — immediate silence · `{type:'cc:close'}` (background → content: icon
137+
toggle / takeover) · disconnect = session over (badge cleared, in-flight speech cancelled).
132138

133139
## 6. Testing architecture (the executable-requirements machinery)
134140

@@ -150,7 +156,7 @@ NYT subscription.
150156

151157
| # | Decision | Why | Revisit when |
152158
|---|---|---|---|
153-
| D1 | MV3 + side panel hosts the conversation | Speech needs a document; panel persists; captions UI | Popup/offscreen if panel UX annoys |
159+
| D1 | MV3 + the content script hosts the conversation, speech-only (rev. 2 — originally the side panel) | Speech recognition needs a document and the page is one; no UI wanted (captions dropped for console diagnostics); session dying with the page is desired | Offscreen document if page-lifetime coupling or the nytimes.com mic grant annoys |
154160
| D2 | `chrome.tts` primary for output | Immune to page autoplay rules | speechSynthesis-only if voice quality disappoints |
155161
| D3 | Synthetic click+keydown for writing | Matches user behavior; per-cell addressing beats cursor settings | MT-02 fails → fallbacks in FEASIBILITY §3 |
156162
| D4 | Pure reducer for dialog policy | The spec is mostly dialog policy; reducers make it assertable ||

docs/FEASIBILITY.md

Lines changed: 22 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -43,23 +43,32 @@ Conclusion: the assumption in the brief is **correct**. The extension is a pure
4343
### Speech-to-text (available, with three practical caveats)
4444
- **`webkitSpeechRecognition`** (Web Speech API) is available in *document* contexts: content
4545
scripts, side panel, popup, offscreen documents. It is **not** available in the MV3 service
46-
worker — which is fine, because our conversation runs in the side panel (a document).
46+
worker — which is fine, because our conversation runs in the content script (the puzzle page is
47+
a document).
4748
- It returns an **n-best list** (`maxAlternatives`), which we exploit for homophone handling
4849
(REQ-ANS-004) — a genuine bonus for this use case.
4950
- Caveats and how the design absorbs them:
50-
1. **Mic permission context.** Permission is granted per-origin. Requesting from the side panel
51-
ties the grant to the extension origin — granted once, persists across NYT visits. The STT
52-
port pre-flights `navigator.permissions.query({name:'microphone'})` + a one-time
51+
1. **Mic permission context.** Permission is granted per-origin. Recognition runs in the content
52+
script, so the grant belongs to **nytimes.com** — the prompt reads as the site asking, and the
53+
grant persists across NYT visits. Accepted deliberately (rev. 2): it buys a zero-UI product.
54+
The STT port pre-flights `navigator.permissions.query({name:'microphone'})` + a one-time
5355
`getUserMedia` to surface the prompt cleanly (REQ-SPCH-003, MT-05).
54-
2. **No recognition in service workers.** Solved by architecture: speech lives in the side panel.
56+
2. **No recognition in service workers.** Solved by architecture: recognition lives in the page;
57+
TTS lives in the service worker (`chrome.tts`, relayed over the session port).
5558
3. **Cloud vs on-device** — see §1's asterisk.
5659

57-
### Why the conversation runs in the **side panel** (`chrome.sidePanel`, Chrome ≥ 114)
58-
- It's a persistent extension document: survives while you play, hosts STT/TTS legally, and gives
59-
us the caption/transcript UI (REQ-SPCH-007/008) for free.
60-
- A popup would close on any click into the page (killing the mic); a content script could host
61-
speech but ties mic permission and autoplay rules to nytimes.com and offers no UI surface.
62-
- The extension icon still toggles the whole session (open panel = start, click again = close/stop),
60+
### Why the conversation runs **in the page** (content script) — rev. 2; originally the side panel
61+
- The product is speech-only by decision: no captions, no panel, no visual surface (diagnostics go
62+
to the page console, REQ-SPCH-007/008). That removes the side panel's main reason to exist.
63+
- A content script is a document, so recognition works; the mic prompt coming from nytimes.com and
64+
the session dying on page reload/navigation are both accepted — the latter is even wanted
65+
(REQ-LIFE-008 for free).
66+
- TTS is *not* hosted in the page: `speechSynthesis` there is subject to nytimes.com's
67+
autoplay/user-activation rules, so speaking is relayed to the service worker's `chrome.tts`
68+
(immune to page policies) over the session port.
69+
- A popup would close on any click into the page (killing the mic); the side panel worked but was
70+
a permanent visual appendage the product doesn't need.
71+
- The extension icon still toggles the whole session (click = start, click again = stop),
6372
matching the brief's interaction model (REQ-LIFE-001/002).
6473

6574
## 3. Can we type answers into the NYT grid? — **Yes, with one risk to validate early.**
@@ -69,8 +78,8 @@ What the page gives us:
6978
- **Reading state** is straightforward: the grid is an SVG (`g.xwd__cell` cells with letter/number
7079
text), the clue lists are `<ol>`s with rich-text `<li>`s, selection is expressed via CSS classes,
7180
and solving triggers a congratulations modal. All selectors are quarantined in
72-
`page-adapter/selectors.js` with a one-click **probe** (REQ-PAGE-009) because NYT can rename
73-
classes any day.
81+
`page-adapter/selectors.js` with an on-demand **probe** (REQ-PAGE-009, run per MT-01) because
82+
NYT can rename classes any day.
7483
- **Writing** (primary approach): do what the user's fingers do —
7584
1. click the target cell (selects it),
7685
2. dispatch a `keydown` KeyboardEvent with the letter (NYT listens at document level),

0 commit comments

Comments
 (0)