feat(ckeditor): GPL AI assistant for text notes - #10730
Draft
eliandoran wants to merge 6 commits into
Draft
Conversation
…lace/insert A balloon on the selection asks the configured LLM provider to rewrite the selection or generate new content. The response streams into a detached ck-content preview (the document is never touched mid-stream); committing via Replace or Insert below is a single model.insertContent in one model.change(), so one undo step. Follow-up queries chain on the previous response, Try again re-runs against what the last run saw, and Stop keeps the partial result reviewable. The plugin is provider-agnostic: the client injects the transport through config.aiAssistant.stream (cumulative-HTML callback, the same shape as the premium AITextAdapter contract), implemented on top of the existing /api/llm-chat/stream SSE endpoint. Without a configured provider the command, toolbar button and /ai slash entry all disable themselves. The target selection is pinned with an affectsData:false marker (tinted via a highlight downcast) so it survives focus moving into the balloon; the preview sanitizes streamed HTML and strips markdown fences, both covered by unit tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The list view's `_rebuild` and the menu's `_populateMenuList` called `items.clear()` before recreating rows, but a ViewCollection's clear() only detaches its views — it does not destroy them. Rebuilding on every snippet change therefore leaked the old row/menu buttons and their `execute` listeners. Destroy the previous views (which cascades to their icon/text subviews) before discarding them, and cover it with a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…view When a run finishes, the review phase now offers an inline HTML diff of the response against the content it applies to, rendered with ins/del markup and toggled via Result/Changes buttons in the balloon heading. The diff defaults on when available, is computed only from a complete stream (a partial response would diff as mass deletion), and for follow-up queries compares against the response being refined rather than the original selection. The renderer is host-injected through config.aiAssistant.diff, keeping the plugin dependency-free; the client wires htmldiff-js, the same mechanism the revisions dialog uses. A diff-renderer failure only costs the Changes view, never the response. Generate-from-scratch runs have no diff (nothing to diff against), so the toggle stays hidden. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The preview now wraps instead of scrolling horizontally: the element opts out of CKEditor's UI reset via ck-reset_all-excluded (inside the balloon the reset forces white-space: nowrap, zeroed margins and the UI font onto every descendant), wraps text and code blocks (Trilium's theme pins `pre code` to `white-space: pre` when the editor's word-wrap class is absent, which this detached preview always lacks), and constrains images. Wide tables keep their scrollbar — clipping table data would be worse. The review actions row also gains a usage line (model · tokens · ~cost), right-aligned and muted. The stream contract now resolves with an optional AiCompletionUsage; the client transport fills it from the SSE usage chunk the chat already receives. Only provider-reported fields are shown, and sub-cent costs get four decimals where the chat's two would render ~$0.00. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A grouped "Quick actions" dropdown in the balloon's prompt row runs predefined instructions with one click — Edit or review (fix typos, improve writing, make shorter/longer, simplify), Generate (summarize, continue), Change tone, and Translate (the six UI languages). Modelled on the premium assistant's default command palette. The action set is host-injected through config.aiAssistant.quickActions with pre-translated labels; the plugin ships no prompts of its own and hides the dropdown when none are configured. Content-requiring actions are disabled while there is nothing to work on (collapsed selection) and unlock once a response exists, so quick actions chain on each other like typed queries. Everything downstream — streaming preview, Changes diff, usage line, Try again — applies to a quick action exactly as to a typed prompt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
🖥️ App preview is ready! 🔗 Preview URL: https://pr-10730.trilium-app.pages.dev ✅ All checks passed This preview will be updated automatically with new commits. |
Bundle ReportChanges will increase total bundle size by 19.18kB (0.02%) ⬆️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: standalone-esmAssets Changed:
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+16
to
+35
| export function sanitizeAiHtml(html: string): string { | ||
| const doc = new DOMParser().parseFromString(html, "text/html"); | ||
|
|
||
| for (const element of doc.querySelectorAll(DISALLOWED_ELEMENTS.join(","))) { | ||
| element.remove(); | ||
| } | ||
|
|
||
| for (const element of doc.body.querySelectorAll("*")) { | ||
| for (const attribute of Array.from(element.attributes)) { | ||
| const name = attribute.name.toLowerCase(); | ||
| const isEventHandler = name.startsWith("on"); | ||
| const isScriptUrl = (name === "href" || name === "src") | ||
| && attribute.value.trim().toLowerCase().startsWith("javascript:"); | ||
| if (isEventHandler || isScriptUrl) { | ||
| element.removeAttribute(attribute.name); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return doc.body.innerHTML; |
Contributor
There was a problem hiding this comment.
When streamed model or diff HTML is shown in the balloon, sanitizeAiHtml only strips a fixed tag list, on* attributes, and javascript: on href/src, then AiPreviewView.setContent assigns that string to innerHTML. Active content outside that strip list (for example xlink:href, data: URLs, base, SVG animate/foreignObject) survives into the editor origin. The client already uses a broader DOMPurify path in sanitize_content.ts for the same class of untrusted HTML.
How this was verified: Traced stream/diff HTML through sanitizeAiHtml into element.innerHTML and compared the strip rules to the existing DOMPurify forbid list.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Comment on lines
+17
to
+169
| export default function buildAiAssistantStream(): AiStreamFunction | undefined { | ||
| if (!readProviderConfigs().length) { | ||
| return undefined; | ||
| } | ||
|
|
||
| return async (request, onData, signal): Promise<AiCompletionUsage> => { | ||
| const messages: LlmMessage[] = [ | ||
| { role: "system", content: SYSTEM_PROMPT }, | ||
| { | ||
| role: "user", | ||
| content: request.context | ||
| ? `Content:\n${request.context}\n\nTask: ${request.query}` | ||
| : request.query | ||
| } | ||
| ]; | ||
|
|
||
| const config = pickDefaultModel(); | ||
| let cumulative = ""; | ||
| const usage = await new Promise<LlmUsage | null>((resolve, reject) => { | ||
| let reported: LlmUsage | null = null; | ||
| streamChatCompletion(messages, config, { | ||
| onChunk: (text) => { | ||
| cumulative += text; | ||
| onData(cumulative); | ||
| }, | ||
| onUsage: (chunk) => { | ||
| reported = chunk; | ||
| }, | ||
| onError: (error) => reject(new Error(error)), | ||
| onDone: () => resolve(reported) | ||
| }, signal).then( | ||
| // A stream that ends without a "done" event (connection dropped) still settles. | ||
| () => resolve(reported), | ||
| reject | ||
| ); | ||
| }); | ||
|
|
||
| return { | ||
| // The server reports the model's display name; fall back to the id we asked for. | ||
| model: usage?.model ?? config.model, | ||
| totalTokens: usage?.totalTokens, | ||
| cost: usage?.cost | ||
| }; | ||
| }; | ||
| } | ||
|
|
||
| /** | ||
| * The predefined instructions for the balloon's "Quick actions" dropdown, modelled on the premium | ||
| * AI assistant's default command set. Labels are translated here (the plugin renders them as | ||
| * given); prompts stay English — they are instructions to the model, not UI. | ||
| */ | ||
| export function buildAiAssistantQuickActions(): AiQuickActionGroup[] { | ||
| return [ | ||
| { | ||
| id: "edit", | ||
| label: t("ai_assistant.group_edit"), | ||
| actions: [ | ||
| action("fixTypos", t("ai_assistant.action_fix_typos"), | ||
| "Fix all spelling, grammar and punctuation mistakes. Do not change the meaning, tone or formatting."), | ||
| action("improveWriting", t("ai_assistant.action_improve_writing"), | ||
| "Improve the writing: fix mistakes, tighten the phrasing and apply good writing practices without changing the meaning."), | ||
| action("makeShorter", t("ai_assistant.action_make_shorter"), | ||
| "Shorten this content by removing repetition and non-essential details, without losing key information."), | ||
| action("makeLonger", t("ai_assistant.action_make_longer"), | ||
| "Expand this content with more detail and clearer explanations, keeping the original meaning."), | ||
| action("simplify", t("ai_assistant.action_simplify"), | ||
| "Rewrite this content in simpler language so that it is easier to understand.") | ||
| ] | ||
| }, | ||
| { | ||
| id: "generate", | ||
| label: t("ai_assistant.group_generate"), | ||
| actions: [ | ||
| action("summarize", t("ai_assistant.action_summarize"), | ||
| "Summarize this content into one short paragraph containing only the key ideas and conclusions."), | ||
| action("continue", t("ai_assistant.action_continue"), | ||
| "Continue writing from the end of the provided content, staying on topic and matching its style. Keep the continuation brief.") | ||
| ] | ||
| }, | ||
| { | ||
| id: "tone", | ||
| label: t("ai_assistant.group_tone"), | ||
| actions: [ | ||
| action("professional", t("ai_assistant.tone_professional"), | ||
| "Rewrite this content in a polished, formal, professional tone without changing the meaning."), | ||
| action("casual", t("ai_assistant.tone_casual"), | ||
| "Rewrite this content in a casual, conversational tone without changing the meaning."), | ||
| action("direct", t("ai_assistant.tone_direct"), | ||
| "Rewrite this content in a direct tone, keeping only the essential information."), | ||
| action("friendly", t("ai_assistant.tone_friendly"), | ||
| "Rewrite this content in a warm, friendly tone without changing the meaning.") | ||
| ] | ||
| }, | ||
| { | ||
| id: "translate", | ||
| label: t("ai_assistant.group_translate"), | ||
| actions: [ | ||
| action("translateEnglish", t("ai_assistant.lang_english"), "Translate the content to English."), | ||
| action("translateGerman", t("ai_assistant.lang_german"), "Translate the content to German."), | ||
| action("translateSpanish", t("ai_assistant.lang_spanish"), "Translate the content to Spanish."), | ||
| action("translateFrench", t("ai_assistant.lang_french"), "Translate the content to French."), | ||
| action("translateRomanian", t("ai_assistant.lang_romanian"), "Translate the content to Romanian."), | ||
| action("translateChinese", t("ai_assistant.lang_chinese"), "Translate the content to Simplified Chinese.") | ||
| ] | ||
| } | ||
| ]; | ||
| } | ||
|
|
||
| /** Shorthand for a quick-action entry; all defaults require content to work on. */ | ||
| function action(id: string, label: string, prompt: string): AiQuickAction { | ||
| return { id, label, prompt }; | ||
| } | ||
|
|
||
| /** | ||
| * The assistant works HTML-in/HTML-out: the context is the selection's HTML and the response is | ||
| * committed through the editor's data pipeline, so anything but clean HTML (markdown, fences, | ||
| * commentary) would end up as literal text in the note. | ||
| */ | ||
| const SYSTEM_PROMPT = `You are a writing assistant embedded in a rich text editor of a note-taking application. | ||
| The user gives you a task, usually together with the HTML of the content it applies to. | ||
|
|
||
| Rules: | ||
| - Respond ONLY with HTML. No markdown, no code fences, no explanations, no preamble. | ||
| - Use simple HTML: <p>, <strong>, <em>, <ul>, <ol>, <li>, <h2>-<h5>, <table>, <blockquote>, <code>, <a>. | ||
| - When rewriting content, preserve its structure and formatting unless the task says otherwise. | ||
| - Respond in the same language as the content, unless the task says otherwise.`; | ||
|
|
||
| /** The subset of a stored `llmProviders` entry this module reads. */ | ||
| interface StoredProviderConfig { | ||
| id: string; | ||
| provider: string; | ||
| selectedModels?: LlmModelInfo[]; | ||
| } | ||
|
|
||
| function readProviderConfigs(): StoredProviderConfig[] { | ||
| return (options.getJson("llmProviders") as StoredProviderConfig[] | null) ?? []; | ||
| } | ||
|
|
||
| /** | ||
| * The provider/model the assistant uses: the first configured provider's default model (or its | ||
| * first model). The same resolution the LLM chat starts out with — a per-run model picker in the | ||
| * balloon can come later. | ||
| */ | ||
| function pickDefaultModel(): LlmChatConfig { | ||
| for (const config of readProviderConfigs()) { | ||
| const model = config.selectedModels?.find((m) => m.isDefault) ?? config.selectedModels?.[0]; | ||
| if (model) { | ||
| return { model: model.id, provider: config.provider, providerId: config.id }; | ||
| } | ||
| } | ||
| // No selected models anywhere: let the server resolve the provider's own default. | ||
| return {}; | ||
| } |
Contributor
There was a problem hiding this comment.
When any llmProviders entry exists, the assistant enables even if every entry lacks selectedModels. pickDefaultModel then returns {}, and the stream request reaches the server without provider/providerId/model, so the route falls back to getProviderByType("anthropic"). Runs fail with a missing Anthropic provider when only another provider is configured, or they use the wrong provider/model when Anthropic is present but not the intended default.
Knowledge Base Used: LLM Chat and MCP Integration
Contributor
Greptile SummaryAdds a GPL CKEditor AI assistant balloon for text notes (stream preview, replace/insert, diff review, quick actions) wired to the existing LLM chat stream, plus a small snippets UI leak fix.
Confidence Score: 3/5Not safe to merge until the preview HTML sanitization gap and default LLM provider/model resolution are fixed. Streamed assistant HTML is rendered with innerHTML behind a narrow custom sanitizer, and empty selectedModels can send the stream with no provider so the server hard-defaults to anthropic, breaking or misrouting runs for otherwise valid setups. Files Needing Attention: packages/ckeditor5/src/plugins/ai_assistant/ai_html.ts, apps/client/src/widgets/type_widgets/text/ai_assistant_stream.ts
|
| Filename | Overview |
|---|---|
| packages/ckeditor5/src/plugins/ai_assistant/ai_assistant_ui.ts | Orchestrates balloon, streaming, diff, and single-step commit; solid design but depends on host sanitizer and marker lifecycle. |
| packages/ckeditor5/src/plugins/ai_assistant/ai_html.ts | Hand-rolled HTML sanitizer for preview is incomplete versus existing DOMPurify-based client sanitization. |
| apps/client/src/widgets/type_widgets/text/ai_assistant_stream.ts | Transport and quick actions; enablement/default model resolution can miss or mis-route non-Anthropic providers without selectedModels. |
| packages/ckeditor5/src/plugins/ai_assistant/ai_assistant_form.ts | Presentational form/phases with tests; preview uses innerHTML by design after caller sanitize. |
| packages/ckeditor5/src/plugins/snippets/snippetlistview.ts | Destroys previous list rows on rebuild to fix view/listener leaks. |
Sequence Diagram
sequenceDiagram
participant User
participant Balloon as AiAssistantUI
participant Stream as buildAiAssistantStream
participant API as POST /api/llm-chat/stream
participant Preview as AiPreviewView
User->>Balloon: Open / prompt / quick action
Balloon->>Balloon: Pin AI_TARGET_MARKER, capture context HTML
Balloon->>Stream: stream(query, context, signal)
Stream->>API: SSE messages + LlmChatConfig
loop chunks
API-->>Stream: text delta
Stream-->>Balloon: cumulative HTML
Balloon->>Preview: sanitize + throttled innerHTML
end
API-->>Stream: usage / done
User->>Balloon: Replace or Insert below
Balloon->>Balloon: data pipeline insert at marker (one undo step)
Reviews (1): Last reviewed commit: "feat(ckeditor): add quick actions to the..." | Re-trigger Greptile
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A GPL-licensed AI assistant for the rich-text editor, replicating the core of CKEditor's premium
AIAssistantwith zero premium dependencies and zero new server code:/aislash entry) with a free-form prompt and a grouped Quick actions dropdown (fix typos, improve writing, shorter/longer, simplify, summarize, continue, tone, translate).ck-contentpreview; the document is never touched mid-stream. Committing via Replace / Insert below is onemodel.insertContentinside onemodel.change()→ one undo step, schema-filtered through the data pipeline.ins/deldiff of the response against the content it replaces, rendered withhtmldiff-js(the same mechanism as the revisions dialog). Computed only from complete streams; follow-up queries diff against the response they refine.usagechunk the LLM chat already receives.Architecture
The plugin (
packages/ckeditor5/src/plugins/ai_assistant/) is fully provider-agnostic: the client injectsstream,diffandquickActionsthroughconfig.aiAssistant(same pattern assnippets.definitions). The transport (apps/client/.../ai_assistant_stream.ts) reuses the existing/api/llm-chat/streamendpoint and the user's configured LLM provider; without a provider the command, button and slash entry disable themselves. The target selection is pinned with anaffectsData: falsemarker (highlight-tinted) so it survives focus moving into the balloon. The stream contract ({query, context, cumulative onData, abort}) mirrors the premiumAITextAdaptershape.Testing
pnpm typecheckclean; no new lint findingsKnown limitations / follow-ups
/slash entries🤖 Generated with Claude Code