LLM stream to generate markdown text and insert to editor

[laotala828]

Hello! I'm quite new to Lexical and have been experimenting with using AI streams to create markdown text in an editor.
Typing directly into the editor works fine.

keyboard.mov

But I'm encountering issues when trying to insert text from the AI stream into the editor.

llm.mov

Currently, I'm using selection.insertRawText(chunk) for this purpose. Could you help me figure out if I'm missing something? Thanks a lot!

[jessequartey]

Hello were you able to figure it out.

[whoisandy]

@laotala828 What you might need is a markdown to lexical state transformer and vice versa (optional). As the text streams in markdown, the editors internal state can be constructed.

[peterzagoranski]

I had a similar issue. U can solve this if u replace selection.insertRawText(chunk) with:

import { $convertFromMarkdownString, TRANSFORMERS } from '@lexical/markdown';
$convertFromMarkdownString(chunk, TRANSFORMERS, selection?.getNodes()?.[0]);

but in my case i'm not streaming chunk by chunk ... but instead I wait for the whole result and add this result only once ...

[andrewlu0]

For anyone else trying to implement this, I got a version working that supports markdown with tables using lexical's HTML converter and showdown, and vercel AI sdk:

  const {
    completion,
    input,
    handleInputChange,
    handleSubmit,
    isLoading: isGenerating,
  } = useCompletion({
    api: `/api/competitors/${id}/comparison`,
    initialInput: `You are a professional product marketer.`,
    experimental_throttle: 200,
  });

  // Update editor content when completion changes
  useEffect(() => {
    if (!completion) return;
    const converter = new showdown.Converter({
      tables: true,
    });
    const html = converter.makeHtml(completion);
    const parser = new DOMParser();
    const dom = parser.parseFromString(html, "text/html");
    editor.update(
      () => {
        const root = $getRoot();
        root.clear();
        const nodes = $generateNodesFromDOM(editor, dom);
        $insertNodes(nodes);
      },
      {
        discrete: true,
      },
    );
  }, [editor, completion]);

Not sure if clearing the editor and inserting all the nodes again is the best way... I couldn't find a way to set the editor state from a set of nodes directly.

lexicalexample.mp4

[jLynx]

This is rerendering the entire thing on each chunk, correct?
Or am I misunderstanding and its only rerendering the last chunk to come in?

[theabhayprajapati]

got any way to insert nodes?? rather rerendering whole thing?

[matsuyama-k1]

maybe you can do by separating streamed markdown into blocks and insert as Element Node.

This will help how to separate blocks. ( I did not tried)
https://github.com/vercel/streamdown/blob/main/packages/streamdown/lib/parse-blocks.tsx