Skip to content

Annotate reports with an AI-written story#1923

Open
ghostiee-11 wants to merge 15 commits into
holoviz:mainfrom
ghostiee-11:feat/report-story-annotation
Open

Annotate reports with an AI-written story#1923
ghostiee-11 wants to merge 15 commits into
holoviz:mainfrom
ghostiee-11:feat/report-story-annotation

Conversation

@ghostiee-11

@ghostiee-11 ghostiee-11 commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Adds an Annotate button to the report toolbar that uses an LLM to write a story about the selected tables and charts: an overall summary at the top plus a short note under each section. The narrative renders in the report and flows into both the Notebook and HTML export. Closes #1912.

Note: this stacks on #1911 (#1922); the first commits belong to that PR, so please review after it merges.

Untitled.-.09.July.2026.at.20.50.01.1.mp4

Add a Section.include_in_export flag and route to_notebook/to_html
through _export_views/_export_view so a report can be exported with a
subset of its sections. Defaults keep every section included, so
existing exports are unchanged.
Render each report accordion card header as a checkbox bound to the
section include_in_export flag next to its title, so users can pick
which sections to keep before exporting. The checkbox appears once the
report has outputs and leaves the rendered report unchanged.
Apply the HoloViz cleanup review: replace the list-building loop in
Report._export_views with a comprehension. No behavior change.
@codecov

codecov Bot commented Jul 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.27507% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.84%. Comparing base (eaf1c6d) to head (6bd11e3).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
lumen/ai/report.py 93.87% 19 Missing ⚠️
lumen/ai/agents/story.py 91.83% 4 Missing ⚠️
lumen/tests/ai/test_story.py 99.17% 2 Missing ⚠️
lumen/tests/ai/test_report.py 98.96% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1923      +/-   ##
==========================================
+ Coverage   71.22%   71.84%   +0.62%     
==========================================
  Files         198      200       +2     
  Lines       34163    34923     +760     
==========================================
+ Hits        24332    25092     +760     
  Misses       9831     9831              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ahuang11

ahuang11 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Can the story interweave plots and tables like a blog post? Also, do you have an updated video?

Also can user write a prompt to ask the LLM to focus on a specific angle?

@ghostiee-11

Copy link
Copy Markdown
Collaborator Author

Yup.. sharing the video only compressing that

@ghostiee-11

Copy link
Copy Markdown
Collaborator Author
Untitled.-.10.July.2026.at.02.51.11.3.mp4

Only Report is ever exported, so the base TaskGroup _export_views /
_export_view wrappers (which just returned self.views / self._view) were
dead code. Drop them and move to_notebook / to_html onto Report next to
its _export_views / _export_view overrides.
@ahuang11

ahuang11 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Okay thanks for sharing the video.

Ideally, the story interweaves pictures and texts. Also, on clicking the annotate, it should show a popup to enter text to guide the story.

@ghostiee-11

Copy link
Copy Markdown
Collaborator Author

Hmm Okay Andrew, will refactor this and add this

@ghostiee-11
ghostiee-11 force-pushed the feat/report-story-annotation branch from 3f567c6 to 6c01e04 Compare July 10, 2026 13:04
@ghostiee-11

Copy link
Copy Markdown
Collaborator Author
Screen.Recording.2026-07-10.at.6.25.28.PM.mp4

@ahuang11

Copy link
Copy Markdown
Contributor

I like it (poor low performer 😭)!

Let's think about UI/UX. The current implementation is a bit weird ongoing back and forth on base report vs story report. I wonder if we can use Tabs or some other layout to differentiate. Also, what if user wants to tweak a specific section, or make manual edits (should this be supported)? Lastly, what if user simply wants to regenerate the full story? Or wants to create a new version?

@ghostiee-11

Copy link
Copy Markdown
Collaborator Author

Haha 😅

Yupp, need suggestions over the UI/UX, planning it, I too feel back and forth weird.

if user wants to tweak a specific section, or make manual edits (should this be supported)?

For this, I was thinking to add some button edit with AI.

Lastly, what if user simply wants to regenerate the full story? Or wants to create a new version?

Hmm, Better UX needed for this....

@ahuang11

ahuang11 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Hmm, Better UX needed for this....

If it can work like Google docs, that'd be cool
https://discourse.holoviz.org/t/how-to-bind-html-contents-using-reactivehtml/5444/2?u=ahuang11

(ReactiveHTML is outdated now, use JSComponent)

Nested sections rendered without a checkbox because only the report drew section headers, and the export read each top level section's aggregated views, so a nested section's selection was ignored. Render a heading and checkbox for nested sections, and walk the tasks when collecting the views so a discarded section is left out of the notebook and the HTML at any depth.
Selection only reached whole sections, so two charts made in one session could not be kept or dropped separately. Move include_in_export onto Task and give every task its own checkbox, and walk the tasks when collecting the views so discarding one chart leaves the rest of its section intact. The report explains the checkboxes at the top rather than on a per-checkbox info icon.
Add a StoryAgent that turns the report's charts and tables into ordered prose and view blocks, plus the catalog builder that describes each view for the prompt.
Add an Annotate action that opens a guidance popup (tone presets plus free text) and writes an AI story that interleaves prose with the charts and tables it discusses. The story is a separate view reachable from the report, an Arrange dialog reorders sections via a JSONEditor, and the notebook and HTML exports follow whichever view is shown.
Replace the toggle that swapped the report view for the story view with a Tabs layout, so the two are told apart at a glance instead of by flipping back and forth. The Story tab appears once a story is generated and is removed when it is discarded, and the exports follow whichever tab is selected.
Render each prose block as a contenteditable JSComponent so the story can be tweaked by hand like a document, and keep the prose as text on the story blocks so an edit flows into the notebook, HTML and Word exports. The text is only pushed back into the element when it changes elsewhere, otherwise the caret would jump to the start while typing.
Add a Regenerate action on the story that reopens the guidance popup so the whole story can be written again with a different angle, and give each paragraph an edit action that asks the LLM to rewrite just that paragraph from an instruction, leaving the charts and the rest of the story untouched.
Regenerating now adds a version instead of replacing the story, and a picker on the story switches between them, so a version the user has edited by hand is never thrown away by asking for a fresh take. Each version owns its blocks, so edits stay with the version they were made on.
The editable paragraphs showed the raw Markdown the model writes, so the story read as source rather than prose. Render each paragraph and swap to the source only while it is being edited, rendering again on blur, using the same Markdown engine the Markdown pane uses.
@ghostiee-11
ghostiee-11 force-pushed the feat/report-story-annotation branch from 6c01e04 to 6bd11e3 Compare July 17, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Annotate charts / tables

2 participants