Annotate highlights - proposed implementation approach

[eyevancm]

This is a contribution proposal for the "Annotate highlights" roadmap item. I'd like to align on approach before writing code.

Background

I'm the author of Marginalia (https://github.com/eyevancm/marginalia), a Firefox extension for annotating web pages and exporting to Obsidian. The core pattern – select text, attach a note inline, export quote+note pairs to Markdown – matches what "Annotate highlights" describes. Rather than maintaining a parallel extension, I'd like to bring this pattern upstream to Web Clipper, where it already has cross-browser reach including Firefox Android and Safari iOS.

---

Current friction in the highlight workflow

Two specific pain points:

1. Popup-first requirement. There's no way to start highlighting without opening the full popup overlay first.
2. Annotation is only possible after the fact. Notes can be added in the popup during review, but not at the moment of selection – the context is already gone or needs to be resurfaced in the popup.

---

Proposed changes

Layer 1 — Data model

Extend the highlight object with an optional annotation field: { "text": "selected passage", "timestamp": "2024-10-20T...", "annotation": "user's note" }

annotation is optional: highlights without a note continue to work as today. The changelog note on the timestamp addition mentioned it was designed to make future metadata easier to add; this is that next step.

The field is persisted in browser.storage.local alongside text and timestamp, consistent with the existing storage model.

Layer 2 — UX

2a. Direct highlighter access:
Activate the highlighter from the page without opening the popup first. The goal is zero-friction access to highlight mode. On desktop, a keyboard shortcut or browser action toggle are both viable. On mobile the picture is less clear: keyboard shortcuts don't apply, and on Firefox Android the browser action is buried in a menu rather than immediately visible. Open question for the team (see below).

2b. Inline annotation card:
When a highlight is created, a card overlays the selection area. The quoted passage appears at the top of the card; a text input sits below it, composer-style. The user types their note and saves. The card dismisses, the highlight + annotation are stored, and the user continues making selections.

The card is a fixed, centered overlay rather than a tooltip anchored near the cursor. This avoids positioning issues on small screens and works identically across desktop and mobile without viewport-specific logic.

2c. Popup as staging area:
The popup collects all highlights and annotations in a pending state, persisted in browser.storage.local as they're created (no regression on browser restart). Nothing is written to the vault until the user explicitly commits. User reviews the full set of quote+note pairs, edits if needed, then ships to Obsidian or discards. The popup shifts from being the only place to annotate to being the review-and-commit step.

2d. Highlights review view (follow-up):
Extend Settings > Highlights to display annotations alongside each highlight. Happy to defer this to a separate PR.

Layer 3 — Template variable

{{highlights}} already supports map filter access to item.text and item.timestamp. Adding item.annotation follows the same pattern:

{{highlights|map:item => "> " + item.text + "\n\n**Note:** " + item.annotation + "\n\n---"|join:"\n\n"}}

The default template should handle highlights with and without annotations gracefully; no empty "Note:" lines if no annotation was added.

---

Cross-browser and cross-device compatibility

The proposal is intentionally scoped to APIs that work uniformly across all target platforms. Specifically:

• Content script + DOM overlay – standard WebExtension API, works on Chrome, Firefox, and Safari (including the Xcode wrapper)
• browser.storage.local – works on all platforms including Firefox Android; browser.storage.sync is explicitly avoided (Firefox Android bug #1625257 (https://bugzilla.mozilla.org/show_bug.cgi?id=1625257))
• menus API – not used; this API is unavailable on Firefox Android and would block the mobile flow
• sidebarAction – not used; unavailable on Firefox Android
• Text selection events – the card listens to both mouseup (desktop) and touchend/selectionchange (mobile); Web Clipper's existing highlight mode already handles this distinction and the same pattern will be followed
• obsidian:// URI scheme – works on macOS, Windows, iOS, and Android

The one open question on compatibility is highlighter activation (2a), which behaves differently per platform (see questions below.)

---

Proposed PR sequence

Smallest increments, each independently reviewable:

1. Data model: add annotation field to highlight object and storage layer
2. Template variable: expose item.annotation in {{highlights}}; update default template to handle missing annotations gracefully
3. Inline annotation card (2b): content script UI, selection → card → save flow
4. Direct highlighter access (2a): activation path without opening popup
5. Popup staging area (2c): popup holds pending highlights + annotations until commit or discard
6. Highlights review view (2d): separate follow-up PR

PRs 1 and 2 have no UI surface and are the easiest to review in isolation. Happy to start there.
  ---

Questions

1. Is this aligned with how you've been thinking about this feature? Any existing design direction or mockup?
2. Highlighter activation (2a): What's the preferred mechanism for activating the highlighter without opening the popup first? On desktop a keyboard shortcut or browser action toggle both seem viable, but on mobile the options are more constrained: no physical keyboard, and on Firefox Android the browser action isn't immediately reachable. Does the team have a preference or an existing pattern we should follow here?
3. Should annotations be editable after creation (before committing to Obsidian)?
4. Should the default template include annotations, or opt-in via custom templates?
5. Any preference on PR sequencing or scope?