Skip to content

feat(import): support Google Takeout Gemini Apps activity exports - #1330

Open
rodboev wants to merge 15 commits into
kenn-io:mainfrom
rodboev:pr/1295-gemini-takeout-import
Open

feat(import): support Google Takeout Gemini Apps activity exports#1330
rodboev wants to merge 15 commits into
kenn-io:mainfrom
rodboev:pr/1295-gemini-takeout-import

Conversation

@rodboev

@rodboev rodboev commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Adds agentsview import --type gemini-apps support for Google Takeout Gemini Apps activity exports. Whole-hour numeric zones could be interpreted as UTC when the timestamp matcher accepted only the GMT prefix after the numeric form failed, so the import could persist a valid-looking record with the wrong instant and stable session ID.

The importer admits Gemini Apps records by their document structure and compatibility grammar, skips non-prompt activity, and emits each compatible Prompted activity as one stable plain-text message. It keeps visible text from inline and preformatted code, including spaces, tabs, line breaks, and authored backticks, without treating HTML presentation as speaker roles or generating Markdown delimiters. The parser now consumes each complete timestamp zone token and validates it once at the existing Gemini Apps zone authority. It accepts the existing named zones plus complete GMT±H, GMT±HH, GMT±H:MM, and GMT±HH:MM forms, with omitted minutes treated as zero. Malformed and unsupported complete tokens fail before import callbacks, while canonical UTC timestamps continue to drive timestamp-local stable IDs.

Unsupported locale or timestamp grammar fails before callbacks, while unrelated Takeout products are ignored and valid Gemini records in the same archive remain importable. The import-only provider, shared persistence, reimport identity, FTS handling, CLI dispatch, and Gemini CLI behavior remain unchanged. Browser upload, HTTP, schema, sync, remote transfer, frontend import UI, and raw archive handling are out of scope.

Closes #1295

@roborev-ci

roborev-ci Bot commented Aug 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (3109b2c)

The Gemini Takeout import has two medium-severity parsing issues that can corrupt message boundaries and code formatting.

Medium

  • internal/parser/gemini_apps_takeout.go:533 — Content blocks containing timestamp-shaped strings are discarded. Prompts such as “Explain Jan 2, 2025, 3:04:05 PM EDT” can cause prompt/response splitting to merge or misassign messages. Do not filter timestamps from content blocks; timestamps are already extracted from the header zone.

  • internal/parser/gemini_apps_takeout.go:735cleanGeminiAppsText trims every line and collapses whitespace with strings.Fields, corrupting indentation and repeated spaces in <pre> and code content. Preserve whitespace in preformatted regions and limit normalization to structural HTML whitespace outside them.


Reviewers: 2 done | Synthesis: codex, 10s | Total: 5m35s

@roborev-ci

roborev-ci Bot commented Aug 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (688683f)

Import support is functional, but three medium-severity parser issues could silently reject or corrupt valid Gemini Apps Takeout content.

Medium

  • internal/parser/gemini_apps_takeout.go:533 — Content blocks containing any timestamp-shaped substring are discarded, so legitimate prompts or responses mentioning dates such as “Jan 2, 2025, 3:04:05 PM EDT” can be silently lost. Exclude only structurally identified metadata, or require the entire normalized block to match the record’s header timestamp.

  • internal/parser/gemini_apps_takeout.go:735cleanGeminiAppsText trims each line and collapses whitespace with strings.Fields, corrupting indentation and spacing in code or preformatted content. Track <pre>/<code> context and preserve whitespace there while normalizing ordinary HTML text separately.

  • internal/parser/gemini_apps_takeout.go:40 — Admission and parsing depend on English titles, activity labels, month names, and AM/PM markers. Valid localized Google Takeout exports may be rejected or have every record reported as skipped. Prefer locale-neutral structural or machine-readable fields; otherwise support documented locales or explicitly detect and report the language limitation.


Reviewers: 2 done | Synthesis: codex, 11s | Total: 6m40s

@roborev-ci

roborev-ci Bot commented Aug 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (4487a28)

The Gemini Apps importer has two medium-severity parsing issues that can block valid mixed-product imports and corrupt rendered text.

Medium

  • Mixed-product Takeout archives can abort before importing Gemini activityinternal/parser/gemini_apps_takeout.go:166
    Non-Gemini HTML using Takeout’s generic outer-cell/header-cell/content-cell structure is classified as unsupported vocabulary. A multi-product archive containing valid Gemini Apps activity alongside another My Activity product can therefore fail entirely.
    Fix: Ignore files positively identified as another product, apply locale/format rejection only to Gemini Apps candidates, and add a mixed-product archive test.

  • Inline whitespace and generated list formatting are corruptedinternal/parser/gemini_apps_takeout.go:981
    Text nodes are trimmed independently, causing inline elements that emit no marker to remove surrounding spaces—for example, See <a>source</a> for details becomes Seesourcefor details. Generated list markers such as "\n- " are also normalized to "-".
    Fix: Preserve whitespace across non-boundary inline tokens, preserve generated list separators as boundaries, and add coverage for links, spans, and lists.


Reviewers: 2 done | Synthesis: codex, 10s | Total: 8m25s

@roborev-ci

roborev-ci Bot commented Aug 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (062faf5)

The Gemini Apps importer has three medium-severity parser correctness issues; no security vulnerabilities were identified.

Medium

  • internal/parser/gemini_apps_takeout.go:190 — Any unrecognized activity label marks the entire archive unsupported, rejecting valid prompts instead of counting unknown activity types as skipped. Allow unrecognized English labels to reach the normal skip path; detect unsupported localization using locale metadata and timestamp grammar rather than an activity-label whitelist.

  • internal/parser/gemini_apps_takeout.go:678 — When block elements exist, top-level text and inline content outside those blocks are discarded. For example, prompt<p>answer</p> loses prompt, while direct ul/ol content may be split into separate turns by its li children. Preserve top-level inline runs as blocks, treat list containers as single blocks, and add coverage for mixed inline/block content and direct lists.

  • internal/parser/gemini_apps_takeout.go:824 — Ignored-subtree depth is tracked incorrectly: nested ignored tags reset the depth, and void descendants increment it without a matching end tag. This can expose hidden template content or suppress visible content after a template. Use a proper stack, increment only for non-void start tags, and do not reset depth for nested ignored tags.


Reviewers: 2 done | Synthesis: codex, 10s | Total: 6m46s

@roborev-ci

roborev-ci Bot commented Aug 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (3ba30d5)

Gemini Takeout import support is promising, but three medium-severity parser issues can incorrectly abort or reject valid imports.

Medium

  • internal/parser/gemini_apps_takeout.go:102 — Locale validation runs before confirming the HTML contains Gemini Apps data. An unrelated non-English HTML file in a mixed Takeout directory can abort the entire import. Detect Gemini Apps cells first, then validate locale only for files containing Gemini candidates.

  • internal/parser/gemini_apps_takeout.go:182 — Timestamp and content validation happen before checking the activity label. Canvas, feedback, and unknown records with unsupported timestamps or missing content can therefore abort the import or count as errors instead of being skipped. Skip non-Prompted records before validating prompt-specific fields.

  • internal/parser/gemini_apps_takeout.go:242 — Leading whitespace or comments inside a content-cell become a synthetic empty block, causing valid formatted HTML such as <div class="content-cell">\n<p>prompt</p> to be rejected as having no prompt. Exclude whitespace-only and non-rendering nodes from synthetic runs while preserving validation for explicitly empty semantic blocks.


Reviewers: 2 done | Synthesis: codex, 10s | Total: 4m20s

@roborev-ci

roborev-ci Bot commented Aug 1, 2026

Copy link
Copy Markdown

roborev: Combined Review (d19c78f)

The Gemini Apps Takeout importer has two medium-severity parsing issues involving inline and multiline code.

Medium

  • internal/parser/gemini_apps_takeout.go:319<code> is treated as a message-block boundary despite being inline HTML. For example, Use <code>go test</code> now becomes three blocks, potentially storing Use as the prompt and the remainder as an assistant response. Remove code from isGeminiAppsBlock and add a mixed inline-code test.

  • internal/parser/gemini_apps_takeout.go:380<pre><code> content is wrapped in single backticks, causing Markdown to treat multiline content as inline code, normalize whitespace, and break on embedded backticks. Render <pre> content as a fenced code block using a delimiter longer than any contained backtick run, while retaining inline delimiters for standalone <code> elements.


Reviewers: 2 done | Synthesis: codex, 9s | Total: 7m44s

@roborev-ci

roborev-ci Bot commented Aug 2, 2026

Copy link
Copy Markdown

roborev: Combined Review (434cdbd)

Medium-severity issue found in Gemini Apps Takeout session identity; no security issues identified.

Medium

  • Unstable session IDsinternal/parser/gemini_apps_takeout.go:66
    Session IDs use a global ordinal across all parsed records. Prepending or reordering records changes subsequent IDs, causing re-imported archives to create duplicate sessions rather than update or skip existing ones. Derive IDs from the timestamp plus an occurrence index scoped to records sharing that timestamp, and add a regression test confirming that prepending a record preserves existing IDs.

Reviewers: 2 done | Synthesis: codex, 10s | Total: 5m27s

@roborev-ci

roborev-ci Bot commented Aug 2, 2026

Copy link
Copy Markdown

roborev: Combined Review (4b2b8ae)

Medium-severity timestamp parsing bug found in the Gemini Apps import path; no security regressions identified.

Medium

  • internal/parser/gemini_apps_takeout.go:25 — Whole-hour numeric zones such as GMT+8 fall through to the alphabetic alternative, which matches only GMT. The record is silently interpreted as UTC, producing an incorrect timestamp and stable session ID.
    • Fix: Accept both GMT±H and GMT±H:MM, require consumption of the complete zone token, and add coverage for whole-hour offsets.

Reviewers: 2 done | Synthesis: codex, 10s | Total: 10m25s

@roborev-ci

roborev-ci Bot commented Aug 2, 2026

Copy link
Copy Markdown

roborev: Combined Review (62aecd8)

No Medium, High, or Critical findings; the reviewed changes are clean at the requested severity threshold.


Reviewers: 2 done | Synthesis: codex, 8s | Total: 9m40s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: Support importing Google takeout for Gemini

1 participant