feat: per-app writing styles for dictation output - #236
Open
Mr-Sunglasses wants to merge 15 commits into
Open
Conversation
Dictation is now shaped for the app that receives it. The same spoken words become `open config.json` in Cursor, `*bold*` in Slack, and a plain sentence in Messages. Seven built-in styles (Plain, Code, Terminal, Chat, Slack, Email, Notes) are bound to apps, seeded on first run for apps installed on this Mac. Plain reproduces the previous pipeline byte for byte, and one master toggle reverts everything. Deterministic and on-device — no LLM, no network call on the dictation path. - WritingStyleEngine: pipeline of filler trim, newline/list commands, symbol substitution, emphasis dialect, then capitalization/punctuation/spacing - SpokenSymbolTransformer: tiered by confidence. Tier A is context-locked (extension whitelist, spoken case commands, explicit bracket commands) and safe in prose; Tier B is heuristic (path slashes, identifier joiners) and limited to Code and Terminal. "literally" suppresses substitution. - FrontmostAppResolver: resolves the target at injection time, with a record-start snapshot as fallback when VocaMac's own window has focus. One NSWorkspace read per dictation, no polling. - AppIdentityMatching: bundle ID or process name, shared with auto-pause - Settings: new Writing Styles page with rule list, per-app rule editor, and a live preview; menu bar shows the active style and re-binds in one tap - capitalizeSentences now capitalizes the first letter rather than the first character, so markup prefixes do not swallow sentence case Catalog seeding runs off the launch path — a menu bar app should not stall its first paint on LaunchServices lookups.
✅ Deploy Preview for voca-mac canceled.
|
Engines emit different text for the same speech. Whisper converts spoken
symbols itself ("slash" becomes /), Parakeet emits the literal word, and
Apple Speech fuses it to the word after it — "edit source slashcomponents
slashbutton". Every rule in SpokenSymbolTransformer matches whole tokens, so
a fused symbol was invisible and Apple Speech output went unshaped. That
reads as "writing styles only work on Whisper", when in fact Whisper simply
leaves the transformer less to do.
A split pass now runs before every other rule, gated hard because it is a
guess about a single token:
- A "dot" split requires the remainder to be a known file extension
("dotjson" -> .json), which prose cannot produce by accident
- Other prefixes are Tier B and additionally need corroboration on the line
— a second symbol occurrence or a path cue — so an isolated "dashboard"
is never touched
- A denylist covers ordinary words beginning with a symbol word
(dashboard, slashed, dotted, underscored, …)
Also stops the slash rule absorbing a path cue as a path component:
"open slash utils" now yields `open /utils` rather than `open/utils`, since
a cue is a verb or preposition, not a path segment.
Adds a regression test pinning that formatting is identical across engines,
and clears the remaining shared preferences in the test factory — they leak
between test processes through UserDefaults and caused an intermittent
failure in testOutputPolishDefaults.
Resolve conflicts between per-app writing styles and the new snippets and About features: - SettingsPage / SettingsSearchIndex / SettingsView: keep both the Writing Styles and Snippets pages. - AppState: keep both injected dependencies, and run snippet expansion after WritingStyleEngine formatting (style first, expand second) on both the injection and Settings-preview paths. - Give the app-rules search entry the "mail app" keyword so the About page still owns the "email" query.
Member
|
/build |
|
⏳ PR Build started for Build signed & notarized DMG... this usually takes 10–20 minutes. |
|
❌ PR Build failed for
|
Bugs found while reviewing the feature end to end. Each has a regression test named after the failure it locks out. Text: - "literally" was consumed unconditionally, so "I literally cannot" lost the word in every style including Plain, the default. It now escapes only an actual command word. - Case commands swallowed symbol words: "camel case handle user input dot swift" produced handleUserInputDotSwift instead of handleUserInput.swift. - Filename, path, and identifier merging force-lowercased both sides, so "Info dot plist" became info.plist. Case is now preserved; only the extension is normalized. - Sentence case stopped at newlines, leaving every line after a "new paragraph" lowercase. - Sentence case renamed generated filenames (readme.md -> Readme.md). The symbol pass now masks what it builds, and capitalization also skips words that already read as identifiers, so a filename the speech engine emitted itself is safe too. - A bare spoken "backtick" became a literal backtick in prose. Bracket commands now all require an explicit open/close, as documented. - A line where no rule fires is returned untouched, so ordinary dictation keeps its own spacing. Durability: - Adding a field to WritingStyleRules would have wiped every app rule on upgrade: synthesized Codable throws on a missing key, and the store decoder turned any throw into "no bindings". Rules now decode field by field with defaults, and the store drops only the rule it cannot read. Resolution: - The menu bar style row and its one-tap bind read the frontmost app while the popover had focus — which is VocaMac — so the row always showed the default and binding silently did nothing. FrontmostAppResolver now tracks the last activated app, and a failed bind reports itself. Snippets: - Expansion runs before styling, masked as placeholder scalars. Triggers match the raw transcript (Code style's filler trimming no longer eats "so what"), and expansions are never re-cased, reshaped, or given a bolted-on period. Also: Code style no longer emits a trailing space into source files, the preview can target a saved rule's overrides rather than only a preset, bindings are cached instead of re-parsed on every SwiftUI read, rules can be exported/imported/cleared, apps can be picked from disk as well as from the running list, and the catalog covers ~25 more apps.
Four more bugs, three of them introduced by the previous pass. - "literally" was still deleted before words that are commands in one context and ordinary English in another: "I literally open the door" lost the word, as did "close", "go", "dash", "forward", "upper", "constant", "screaming" and "angle". No word list can decide this, so the transformer now answers the user's own question — would this word have been rewritten if I had not said it? — by re-running the line without the escape and checking whether the protected word survived untouched. One extra pass per line, and only when the word appears. - That check was first written as one full re-run per candidate, which took 13 seconds on a long utterance, on the main actor, mid-dictation. Tokens now carry the index of the spoken word they still are, so a single baseline pass answers it for every candidate at once: 13s -> 76ms. - Bracket literals were being masked along with identifiers, which hid the start of a sentence from the capitalization pass: "open paren value close paren is here" came out as "(value) is here". Only spans that contain letters or digits are masked now. - A trailing space was appended after a snippet expansion that already ended in one, because the mask hid that from the trailing-space rule. Restoring now swallows it back. Also dedupes the running-app list (helper processes can share a name and bundle ID, which made duplicate rows and ambiguous SwiftUI identities), opens the rule editor after picking an app from disk since the panel cannot ask which style it should use, and indexes the export/import actions for settings search.
|
| Filename | Overview |
|---|---|
| Sources/VocaMac/Models/AppState.swift | Integrates style selection into dictation and settings, while the previously reported concurrent-removal discovery behavior remains. |
| Sources/VocaMac/Services/TextPlaceholder.swift | Adds opaque snippet restoration, while the previously reported separator removal behavior remains. |
| Sources/VocaMac/Models/WritingStyle.swift | Defines the style policies, presets, and backward-compatible rule decoding. |
| Sources/VocaMac/Services/WritingStyleEngine.swift | Implements the deterministic formatting pipeline for resolved writing-style rules. |
| Sources/VocaMac/Models/WritingStyleCatalog.swift | Defines installed-app suggestions and absence-based merging used by the outstanding discovery race. |
Reviews (8): Last reviewed commit: "Merge branch 'main' into feat/writing-st..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Member
Author
|
/build |
|
⏳ PR Build started for Build signed & notarized DMG... this usually takes 10–20 minutes. |
|
❌ PR Build failed for
|
The symbol transformer is tiered, carries a prose negative corpus, and has an
escape word. The three rules outside it — newline commands, list markers, and
emphasis — had none of that, and fired on any matching word. Since a misfire
deletes the spoken command word, the user could not see what had happened.
Confirmed misfires, all now covered by tests:
draw a new line on the chart -> Draw a\nOn the chart
that new paragraph reads better -> That\n\nReads better.
bullet proof vest -> - Proof vest
list item pricing was wrong -> - Pricing was wrong
bold move by the team -> *Move by the team*
italic text is hard to read -> *Text is hard to read*.
strikethrough is a formatting option -> ~~Is a formatting option~~
Newline commands now check their immediate neighbours: a determiner in front
("a new line") or a subject-forming word after ("new line of thinking") means
prose. Ordinals are deliberately not determiners, so "line one new line line
two" still works.
List markers require structure rather than a keyword: two or more marked lines.
One leading "bullet" in a one-line utterance is not a list, and that is the
shape every false positive took. A single dictated bullet no longer gets its
dash — the deliberate cost of removing the class.
Implicit emphasis keeps its line-start anchor and adds a denylist for the word
after the command, splitting "command as subject" ("strikethrough is") and
"command as adjective" ("bold move") from the imperative. An explicit "end
bold" bypasses it, since a closing command is proof on its own.
`regexReplace` grew a variant that filters matches through a closure, which is
what lets each match be checked against its neighbours before it fires.
Writing styles no longer seed app rules at launch. Seeding writes rules, and a
rule changes the shape of an existing user's dictation on upgrade — no sentence
case in their editor, Slack markup in Slack — without them asking. The feature
now ships inert (Plain, no rules) and rules come from "Add Suggested Apps…" or
a menu bar binding. The Settings empty state says so.
`capitalizeSentences` gained a scope. Its style-aware upgrades — line starts and
identifier protection — were reaching users through passthrough rules, so the
master toggle and the Plain style were not the byte-for-byte old pipeline they
promise. Both now use the legacy pass; styles that shape output keep the new one.
Also: drop the dead `plainText` helper, correct the `AppIdentityMatching`
comment that claimed a suffix match where the code does equality, correct the
`ensureTerminalPeriod` doc's punctuation list, and mark `FrontmostAppResolving`
`@MainActor` to match how it is actually used.
Review flagged `MaskedText.restore` for dropping a separator when a snippet expansion that ends in whitespace sits mid-utterance. Checked against the real pipeline: the behavior is correct and the swallow is what makes it correct. "sig thanks" -> Jane Doe\nCEO\nthanks The expansion's own newline ends the line, so the next word starts the following one at column zero. Restoring the user's space would indent it by a stray column, exactly as an expansion ending in " " would otherwise produce a double space. Whitespace the expansion carries wins over the space beside the trigger. The coverage gap behind the report was real, though. The space-ending case had a test and the newline-ending case did not, so the file only pinned half the rule. Both are pinned now, plus back-to-back expansions. The comment above the swallow explained only the end-of-text case — the synthetic space from the trailing-space rule — while the code deliberately applies everywhere, which is what made the general behavior read as an oversight. It now states the mid-utterance case and why it is wanted.
Six review items, all reproduced first. **Terminal punctuation was ASCII-only.** `ensureTerminalPeriod` recognised `.!?:;` and Latin brackets, so a sentence a speech engine had already ended in its own script read as unpunctuated and got a second, wrong mark: धन्यवाद। -> धन्यवाद।. 谢谢。 -> 谢谢。. 已完成！ -> 已完成！. It now recognises terminal marks and closing quotes across Devanagari, Arabic, CJK, Armenian and the curly-quote forms. It also declines to invent a mark for a script whose full stop is not `.`: no language signal reaches this rule, so guessing that a Hindi sentence wants an ASCII period is a visible error, while leaving it as dictated is recoverable. Latin, Greek and Cyrillic still gain one. **Trailing periods survived a line break.** `stripTrailingPeriod` only fired when the period ended the whole string, so "run tests. new line" left Code style with "run tests.\n". It now splits trailing line endings the way `ensureTerminalPeriod` does and puts them back exactly. That shared helper had a bug of its own: it compared against "\n" and "\r" separately, but Swift treats CRLF as one Character, so a "\r\n" ending read as ordinary content. Both rules now use `isNewline`. **Main-actor mutation from a @sendable closure.** Mine, from the `@MainActor` annotation in the previous commit — the activation observer wrote `lastActive` from a closure the compiler could not see was main-actor. `MainActor.assumeIsolated` rather than a `Task` hop: the observer is registered on `.main`, so the isolation is real, and deferring the write would leave a dictation starting in the same turn reading a stale app. **A bundle identifier is now decisive.** `AppIdentityMatching.matches` fell through to executable-basename equality after a bundle-ID mismatch, so two apps shipping the same binary name — a fork, or Code vs Code Insiders on Electron — answered to each other's rules. Equal IDs mean yes, different IDs mean no; the basename fallback stays for CLI tools and hand-typed auto-pause entries, which have no ID. Auto-pause shares the function and inherits the fix. **The seeding lifecycle is gone.** Nothing had called `seedWritingStyleCatalogIfNeeded` since seeding stopped happening at launch, and the persisted marker, the in-flight flag and the bindings-revision guard existed only to protect it. "Add Suggested Apps" was meanwhile running the whole LaunchServices sweep synchronously from a Settings click. Discovery is now one async flow off the main actor with a progress view, and it re-reads bindings afterwards — so a rule the user adds while it runs survives, which is what the revision guard used to do, now a property of the merge. **The parity claim was overstated.** Expanding snippets before styling is a deliberate correction: the old pipeline polished first and appended its trailing space after an expansion that already ended in one, producing a double space or a stray column of indent. So Plain and the master toggle reproduce the old pipeline everywhere except a whitespace-ending snippet. Documented on `CapitalizationScope` and pinned by a test that asserts both the exception and the parity around it.
U+3001 separates clauses, so a sentence ending in one is unfinished — it does not belong beside the closing brackets and quotes that end a sentence. No behavior changes today, because a CJK sentence is left unpunctuated either way, but the set would have been wrong the moment the script rules were. Also drops a test doc comment still naming the seeding function that commit removed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dictation is now shaped for the app that receives it. The same spoken words become
open config.jsonin Cursor,*bold*in Slack, and a plain sentence in Messages.Deterministic and on-device — no LLM, no network call added to the dictation path.
Problem
A single global output pipeline cannot preserve code punctuation while also keeping messages and prose natural. Transcription engines also emit the same speech in different shapes, so output needs deterministic, app-aware normalization without sending text to a cloud service.
Styles
Seven built-in presets, bound to apps.
*bold*mrkdwn and bullet lists.**bold**, bullets, and filename shaping.The feature ships inert.
Plainis the default style and there are no app rules until you ask for them — "Add Suggested Apps…" in Settings sets up the editors, terminals, chat and mail apps you already have installed, and the menu bar binds the app in front in one tap. Creating rules changes the shape of an existing user's dictation, so an upgrade does not get to decide that for them. One master toggle switches the whole feature off.Discovery is a few dozen LaunchServices lookups, so it runs off the main actor behind a progress view and re-reads bindings afterwards — a rule added while it runs is preserved.
Examples
open config.jsonOpen config dot jsonOpen config dot jsonedit src/components/button.tsxhandleUserInput*Ship this today*Design decisions
Rules, not an LLM. VocaMac ships no LLM runtime, and a cloud rewrite would break the 100% local promise. The high-value cases — filenames, paths, identifier casing, emphasis dialect, punctuation policy — are mechanical, testable, instant, and reproducible from a bug report. On-device
FoundationModelsrefinement remains a possible opt-in follow-up.Every rule is context-locked, because every trigger word is also English. This is the whole risk of the feature, and it applies as much to "new line" and "bullet" as to "dot" and "slash". Each rule decides from the words immediately beside it and ignores evidence elsewhere in the utterance.
Symbol substitution is tiered by confidence:
dot comis never a filename), spoken case commands, and bracket commands carrying an explicit "open"/"close".The structural rules carry their own guards, and a misfire there is worse than a symbol misfire because it deletes the spoken command word, so the user cannot see what happened:
Negative cases are tested directly, for both halves: "the dot product", "slash and burn", "dash it all", "visit example dot com", "that was a bold move", "bullet proof vest", "list item pricing was wrong", "italic text is hard to read" and "that new paragraph reads better" all pass through untouched.
Filename lookback extends only across a small closed set of modifiers (
my,new,test, …), so "open my file dot md" givesmyfile.mdwhile "compare readme dot md" givesreadme.mdrather than gluing the verb on. Distinguishing a two-word filename from a verb cannot be done from grammar alone, so the rule prefers the single-word answer whenever it is unsure.The target app is resolved at injection time. Text lands wherever focus actually is when the paste or AX write happens, so that is what gets read — with the record-start snapshot as fallback for when VocaMac's own window has focus. One
NSWorkspaceread per dictation, no polling.App matching reuses the auto-pause contract.
AutoPauseMatching's rules moved to a sharedAppIdentityMatching, and auto-pause now delegates to it, so both features answer "is this the configured app" identically. A bundle identifier is decisive when both sides have one — equal means yes, different means no — so two apps shipping the same executable name (a fork, or Code vs Code Insiders on Electron) can no longer answer to each other's rules. The basename comparison remains the fallback for CLI tools and hand-typed auto-pause entries, which have no identifier.UI
Style: Code — Cursor) with a picker that re-binds the frontmost app in one tap. That persistent indicator is the discoverability mechanism, chosen over a one-shot notification since it explains itself every time the popover opens.Punctuation is script-aware
ensureTerminalPeriodrecognised only ASCII marks, so a sentence a speech engine had already ended in its own script read as unpunctuated and got a second, wrong one —धन्यवाद।becameधन्यवाद।.,谢谢。became谢谢。.. It now recognises terminal marks and closing quotes across Devanagari, Arabic, CJK, Armenian and the curly-quote forms, and declines to invent a mark for a script whose full stop is not.: no language signal reaches this rule, so guessing that a Hindi sentence wants an ASCII period is a visible error, while leaving it as dictated is recoverable.stripTrailingPeriodnow preserves trailing line endings exactly, mirroringensureTerminalPeriod, so Code style strips the period from "run tests.\n". Both useisNewlinerather than comparing against"\n"and"\r"separately — Swift treats CRLF as oneCharacter, which the two-way test missed.What the master toggle reverts, and the one thing it does not
DictationOutputFormatter.capitalizeSentencesgained aCapitalizationScope. The style-aware pass adds line starts, identifier protection (readme.mdmust not becomeReadme.md) and placeholder awareness — all improvements, but still changes, and passthrough rules were routing them to users who had not opted in. Passthrough now uses the pre-writing-styles pass, so both the master toggle and thePlainstyle reproduce the old pipeline.One deliberate exception. Expanding snippets before styling is itself a correction: the old pipeline polished first and appended its trailing space after an expansion that already ended in one, producing a double space or a stray column of indent. A whitespace-ending snippet is therefore the one place where output differs from the previous release, and the difference is the fix. A test pins both the exception and the parity around it.
Performance
Catalog discovery does one LaunchServices lookup per catalog entry, so it runs off the main actor behind a progress view rather than freezing the Settings window, and re-reads bindings afterwards so a concurrent edit is not clobbered. The formatting pipeline itself is string operations on utterance-length input.
Tests
667 tests executed locally: 664 passed, 3 platform/model-dependent skips, and 0 failures. The build is warning-clean.
SpokenSymbolTransformerTests— every Tier A and Tier B rule plus the prose negative corpusStructuralCommandProseTests— the counterpart corpus for newline commands, list markers, and emphasis: prose that must survive, command forms that must still fire, and an end-to-end pass through the real pipelineWritingStyleEngineTests— per-style pipeline behavior, idempotence, policy resolution, edge cases, and a fixture corpus that takes one line per new caseWritingStyleTests— presets, bindings, JSON envelope recovery from corrupt and future-schema payloads, resolution, catalog integrityTerminalPunctuationTests— Devanagari danda, Arabic, CJK, Armenian, curly quotes, the non-Latin no-invention rule, and trailing CR/LF handling for both punctuation rulesAppIdentityMatchingTests— bundle-ID authority, the executable-name fallback that CLI tools and hand-typed entries rely on, and auto-pause agreeing with writing stylesAppStateWritingStyleTests— resolution driving injection via a newMockFrontmostAppResolver, the feature being inert at launch, both passthrough paths matching the old pipeline, and a rule added during discovery surviving itThe test factory now also resets the output-polish preferences; they leak between test processes through
UserDefaults, which was a latent flake.One test is weaker than it looks and says so in its own doc comment:
testStartupLeavesTheFeatureInertruns withskipSystemIntegration, so it cannot catch discovery reintroduced inside that guard.Not included
On-device LLM refinement, user-authored named styles, and window-title matching for browser tabs (Slack-in-Chrome and Gmail-in-Chrome share one bundle ID — a documented limitation). Symbol vocabulary is English-only.