Skip to content

TODO.md

Latest commit

 

History

History
472 lines (434 loc) · 28.1 KB

TODO.md

File metadata and controls

472 lines (434 loc) · 28.1 KB
Raw

OfficeCLI parity handoff

Last updated: 2026-07-28 (Asia/Shanghai)

This file is the handoff for the non-PPT OfficeCLI parity effort tracked by #139. PowerPoint and MCP are deliberate non-goals. There are no third-party compatibility obligations, so future work should prefer a clean, durable contract over compatibility aliases.

Big picture at this handoff

This repository is no longer missing the basic agent-facing Office shell. On current main, one unified office command can discover its own contracts and perform bounded XLSX and DOCX read, create, batch, template, annotate, validate, issues, preview, raw-OOXML, dump, and replay workflows. Native and Wasm are both first-class acceptance targets. The remaining draft F1b landing in PR #237 is the proof that a brand-new Codex session can discover and complete representative installed-command workflows without repository-only instructions.

That phase boundary is significant, but it is not the end of the broader OfficeCLI parity program. Merging #237 completes the installed-command baseline and closes #169; it does not close the 384-row non-PPT delivery ledger or epic #139. The next relay should treat the baseline as a stable platform and resume the dependency-ordered feature ledger through small PRs, beginning with N0b4 #234. Do not turn the ledger row count into a percentage: the rows deliberately have different implementation and validation weight.

Functionality is the current product priority. PR #237 must prove real XLSX and DOCX outcomes, deterministic behavior, cross-runtime agreement, and typed atomic refusal. Additional security-grade build/runtime provenance is useful hardening but is not part of the functional F1b merge gate; it is recorded separately in #267, #268, and #269 so it cannot silently expand this already large PR.

PR #237 closeout checkpoint

  • Worktree: /Users/dii/git/mbtexcel-worktrees/f1b-installed-baseline-20260724
  • Branch: agent/f1b-installed-baseline
  • The branch contains current origin/main (484390e5 at this checkpoint), so no stale-base rebase is pending.
  • Functional semantic-scenario work is committed through 9ff66cd8. It makes the retained acceptance evidence prove actual XLSX and DOCX create/batch/template/annotate/read/preview/dump/replay workflows rather than command-shaped success alone. It also checks deterministic previews, dump/replay/dump fixpoints, native/Wasm semantic agreement, and typed refusal with mutation atomicity, and independently recomputes the scenario claims from retained ledgers and artifacts.
  • Local functional gates on 2026-07-28 are green: focused scenario and evidence policy tests pass; shell syntax and Python compilation pass; moon check reports 0 errors; moon test passes 2,011/2,011 Wasm tests and 3,089/3,089 native tests. The very long adversarial run_test.sh matrix was exercised through its successful scenario/evidence path and many negative cases, but the final quiet local run was intentionally stopped after the product priority changed to functional completeness. Do not record that local matrix as a full PASS; exact-head GitHub CI remains authoritative for it.
  • Before merge, commit this handoff, run moon info && moon fmt, obtain a brand-new ephemeral gpt-5.6-sol Ultra review of the exact HEAD with functional correctness and maintainability as its merge criteria, push that exact HEAD, and require its GitHub CI matrix to be green. If either gate finds a real functional or maintainability blocker, fix only that blocker and repeat both exact-head gates. Otherwise mark #237 ready, merge it, close #169, update #139 with the baseline result, and stop without opening another PR.

Landing state

  • PR #172 is merged into main. It delivered the bounded unified XLSX read surface (outline, get, text, and query) while retaining authenticated OOXML relationship discovery, Markup Compatibility processing, cancellation checks, preserved drawing state, and fail-closed resource accounting.
  • PR #173 is merged into main and closed #162. It delivered transactional XLSX creation and strict batch mutation.
  • PR #182 is merged into main and closed #176. It isolated the post-#173 compatibility fix for Excel-produced overlaps between distinct shared-formula indexes.
  • PR #185 is merged into main and closed #165 and #75. It delivered cross-format office validate and office issues on the exact pre-commit mutation gate, with versioned bounded findings and non-zero validate exit status.
  • PR #186 is merged into main and closed #166. It delivered office preview: one deterministic self-contained HTML document per input through the atomic create-new transaction path, with supported charts (vertical clustered bar, non-stacked line, pie) rendered as accessible inline SVG, bounded data-URI DOCX images, and a truthful office.preview/1 report (charts rendered vs placeholder, merge-aware sheet truncation, surrogate-sanitized outputs). Six review rounds; the final round reported zero findings with byte-identity verification. Rendered-document UTF-16 sanitization also hardened the pre-existing xlsx html publication paths.
  • R1 is complete and closed #167 through PRs #192-#204. office dump emits a versioned provenance envelope around canonical batch-op streams for XLSX and DOCX (styles, tables, validations, conditional formats, content-addressed image assets, notes, and threaded comments), office replay rebuilds documents through the existing strict engines only, and dump→replay→dump is op-identical for both formats. Non-replayable content is disclosed as machine-readable residuals, never silently dropped.
  • T1 scalar scope is complete and closed #168 through PRs #205, #206, and #208. office template merges strict office.template.data/1 values into XLSX cells and DOCX stories (body, headers, footers) across split runs, with a non-executable {{key}} grammar, refusal precedence malformed > unsupported > missing, --allow-missing, and a byte-fidelity gate so substitution never rewrites DOCX runs whose stored bytes do not round-trip to model text. Bounded row/table repetition was split to #207.
  • #207 is closed through PRs #210-#214. office template now also clones a marked template row once per record: the additive regions map addresses an XLSX sheet row (cloned through the atomic grid-bounded insert; any formula-bearing workbook refuses) or a DOCX body table row (cloned through a fail-closed namespace/attribute whitelist that strips w14 paragraph ids), with records under the scalar data gates and a values/record disjoint-union rule. The office.template/1 record gains a bounded regions array and regions_total. Repetition depth beyond v1 (nested regions, column repetition) remains out of scope.
  • PR #216 is merged into main and closed #164. It delivered office annotate add/reply/resolve/unresolve operations through the source-pinned DOCX edit session, with strict annotation validation and preservation-safe publication.
  • PR #217 is merged into main and closed #163. It delivered office create docx and office batch --format docx with the strict docx.batch/2 authoring contract, bounded embedded images, transactional publication, and native/Wasm/OpenXML acceptance coverage.
  • PR #218 is merged into main. It added the unified native/Wasm acceptance task matrix. This is the checked-in matrix half of F1, not the constrained installed-command probe.
  • PR #225 is merged into main and closed #220. It makes the unified office command the repository skill's default entrypoint and reserves legacy commands for explicit capability gaps.
  • PR #224 is merged into main and closed #219. It is only N0a's exact lexical DOCX token map, ahead of the four N0b projection foundations tracked by #221 as #231 through #234, whole-run N0c1 #222, partial-boundary N0c2 #236, and the later transaction SDK and unified CLI slices.
  • PR #228 is merged into main and closed #223. Installed office help now exposes the four consumed input contracts with parser-owned constraints, canonical examples, deterministic fingerprints, and human/JSON/JSONL output. This unblocks the F1b fresh-agent probe in #169.
  • PR #235 is merged into main and closed #231. It landed N0b1's bounded namespace-resolved source-tree identities and exact source spans without changing the ordinary non-retaining scanner contract.
  • PR #242 is merged into main and closed #226. It established the version-aware xlsx.batch/2 registry and retained exact xlsx.batch/1 compatibility for historical dump/replay envelopes.
  • PR #247 and PR #251 are merged into main and closed #232 and #233. N0b2 now mirrors reader-order projection and N0b3 classifies bounded story-wide nested fields. N0b4 #234 is the remaining projection-oracle foundation under #221.
  • The three P2 gaps discovered by the provisional F1b exercise are closed by PRs #243, #241, and #244: formatted XLSX text readback, installed template grammar, and supported-chart dump/replay. PRs #248-#250 also fixed Wasm CLI startup, Unicode date-format scanning, and quoted chart-series sheet names.
  • PR #253 is merged into main and closed #252, landing X4b1's bounded, merged-cell-aware XLSX hyperlink engine foundation.
  • PR #258 and PR #259 are merged into main and closed #254 and #255. They land strict classic-comment validation plus preservation-safe replace/delete VML reconciliation. Prefix-independent VML scanning #256 and semantic form-control provenance #257 remain separate slices.
  • At the 2026-07-26 rescan, draft PR #237 was the only in-flight PR. It carries the F1b installed-command baseline harness, exact-head publication checks, real Linux/macOS Codex canaries, and structured evidence. It closes #169 only after exact-head CI and a brand-new ephemeral ultra Codex CLI review both approve; this handoff intentionally travels in that PR.

The reference OfficeCLI checkout remains .repos/OfficeCLI in the primary repository working tree. This handoff and the major ledger were audited against commit b8669389dbe1f8a5fd0927a51b5ccf91b1dfe3e6; re-audit and update that pin before changing the parity denominator or declaring the ledger complete.

What is complete on current main

The following major-parity foundations are implemented and should not be rebuilt in later PRs:

  • A1-A5: the umbrella office command, versioned capability/help protocol, canonical selectors, atomic validated transactions, and validated raw OOXML fallback.
  • D1-D2: preservation-safe DOCX edit sessions plus bounded DOCX outline, get, text, and query.
  • X1-X2: provenance-checked bounded XLSX reads plus bounded XLSX outline, get, text, and query.
  • X3 (#173): office create xlsx, strict xlsx.batch/1 parsing, transactional XLSX mutation, dry-run/no-replace/overwrite behavior, preservation reports, bounded serialization, and OpenXML validation.
  • D3 (#163/#217): office create docx, strict fresh-document docx.batch/2 authoring, and transactional office batch --format docx.
  • D4 (#164/#216): preservation-safe DOCX comment add, reply, resolve, and unresolve through office annotate.
  • F1 matrix/help (#218/#228): checked-in native/Wasm task acceptance plus the exact installed input contracts consumed by the command. The separate protocol-driven installed baseline is scoped to #237/#169; it does not close #139.
  • N0a/N0b1-N0b3 (#224/#235/#247/#251): exact DOCX lexical tokens, bounded physical source-tree identity, reader-order projection, and story-wide field classification are landed. N0b4 reader transforms and the N0c surgery proofs remain later, independently reviewed slices.
  • X4a1 (#242): the preferred version-aware XLSX mutation registry is landed, with historical batch envelopes remaining replayable.
  • X4b1 (#253): the bounded XLSX hyperlink engine lifecycle is landed. Unified Office exposure remains a separate ledger slice.
  • X4d1a/X4d1b (#258/#259): classic comments are validated without truncation and matched note VML is reconciled without destroying unrelated retained VML. The adversarial scanner and form-control provenance work remain #256/#257.
  • #176: tolerant reads for Excel-produced overlaps between distinct shared-formula indexes, while duplicate masters and out-of-range followers remain rejected per index.

Important implementation invariants:

  • File publication uses moonbitlang/async; no new C stubs are required.
  • Sync and async paths share the same semantic implementation and resource policy. Async scheduling cooperation inside long parser work is intentionally deferred to #174 rather than being faked with entry- or sheet-level yields.
  • XLSX full rewrites must preserve unsupported drawing anchors, relationships, and dependent parts already retained by the reader. Generated drawing object and relationship ids must avoid preserved ids.
  • Read limits remain cumulative across a package. Relationship defaults match the stricter #172 parser boundaries so a workbook written by the library is not rejected by the default reader; callers can still select lower limits.
  • Capability output is schema-driven and snapshot-tested. Never advertise a command or format variant before its end-to-end implementation exists.

The detailed contracts live in:

  • docs/office-major-parity.md
  • docs/office-xlsx-read.md
  • docs/office-xlsx-mutations.md
  • docs/office-docx-read.md
  • docs/office-transactions.md
  • docs/agent-json-schemas.md

Remaining work, in recommended order

The original initial-parity implementation issues, F1a unified entrypoint, installed-help prerequisite, and F1b harness/evidence implementation are complete. PR #237 is the single landing vehicle for #169 — F1b installed-command baseline acceptance. #169 is complete only if #237 is merged after exact-head CI and a fresh ephemeral ultra review both approve. If a relay sees #237 still open, finish those gates and land it before starting another PR. Passing #169 does not close the broader non-PPT parity epic.

After F1b, resume the dependency ledger with small PRs. N0b4 #234 is the next projection foundation under #221; whole-run N0c1 #222 and partial-boundary N0c2 #236 follow it. XLSX VML hardening #256/#257 is independent. The four non-blocking baseline residuals are already filed as #260-#263 and must not be folded into an unrelated feature PR.

Three additional non-blocking harness-hardening issues preserve the unresolved parts of the earlier security-focused Ultra review without holding the functional baseline open: #267 pre-verifies the complete build closure, #268 binds native-loader and script-interpreter closures, and #269 closes the remaining provenance schemas and independent recomputation surfaces. Each is a future dedicated small PR (or smaller issue split), not an invitation to extend #237.

The dependency-ordered small-PR plan for closing the broader agent-relevant OfficeCLI gaps now lives in docs/office-major-parity.md. Give every proposed slice its own issue before implementation and do not fold it into #169. Issue #139 remains open until the major ledger is complete. At the pinned baseline the full delivery denominator has 384 unique acceptance rows: 371 parity/QA/export rows plus 13 explicit beyond-parity/delivery rows. That total combines 18 landed initial foundations with the 366-row incremental ledger and does not double-count overlapping status rows, using the counting rule in the ledger. The ledger explicitly includes bounded XLSX formula calculation/lint and cache refresh, worksheet tab-color/reorder, visible/hidden/veryHidden, freeze/unfreeze, and print-title row/column lifecycle, bounded boolean and/or and row-by-column-name query, full common cell/range formatting, row/column dimension/outline/autofit, row/column structural edits, preservation-safe cell shifts, cascade-safe worksheet rename, unmerge, AutoFilter lifecycle, stable chart enumerate/create/update/remove with relationship cleanup, array-formula add/set/readback, ordinary table/data-validation/conditional-format update and removal, row/column page-break lifecycle, workbook core properties (title, author/creator, subject, description, keywords, category, lastModifiedBy, and revisionNumber) plus read-only created/modified; mutable extended template/manager/company properties plus the eight OfficeCLI-visible read-only application/statistics values, with only lastModifiedBy, timestamps, and application-derived values excluded from typed semantic replay; date1904 conversion, dateCompatibility, calculation settings, remaining workbookPr readback and object-display/privacy state, explicit workbook lock/password protection, activeTab/firstSheet state, and RTL/zoom/view state; preservation-safe existing-DOCX table content, separately reviewable complete common table/row/cell-property changes (including style/grid widths/banding, direction/indent/spacing/padding/metadata/conditional state, diagonal borders, cell flow, spans, and horizontal/vertical merges), paragraph/table/row/column move-copy-swap, row/column edits, and table structural edits; paragraph, character, table, and numbering-style lifecycle; numbering, first/even/default header/footer, paragraph-tab-stop, positional-tab, break, editing-permission range, and section inventory/mutation/exposure, including columns, page-number format/start, line numbering, titlePage, evenAndOddHeaders, page borders, paper sources, vertical alignment/text direction, and footnote/endnote numbering and placement controls; DOCX field and bookmark inventory, typed authoring across pagination, cross-reference, metadata, time/lifecycle, statistics, section, SEQ, STYLEREF, DOCPROPERTY, and IF families, source-pinned existing add/update/removal, and refresh; embedded-chart authoring/readback and source-pinned add/update/delete lifecycle; existing comment, footnote/endnote, hyperlink, and picture add/update/remove lifecycle; DOCX core properties title, author/creator, subject, description, keywords, category, lastModifiedBy, and revisionNumber, read-only created/modified, plus the same three mutable and eight read-only extended properties, with only save/application-managed replay exclusions; deterministic locale/script-font and RTL defaults plus language/direction authoring; root document defaults plus grid/compatibility/ CJK/default-tab/hyphenation, trackRevisions/updateFields/recalcFields, mirror/ gutter/book-fold/privacy/font-embedding settings, and document-root page background color distinct from the background-display flag; tab stops and page/column breaks; fresh and source-pinned add/update/remove for equations, watermarks, drawing text boxes, and shapes; content/move/property revision readback, separately scoped accept/reject, attributed insertion/deletion/move/ formatting authoring, and installed revision exposure/QA; fresh and source-pinned add/update/remove for SDTs and legacy text/checkbox/dropdown forms; CSV/TSV file-or-stdin import with start-cell/inference/header behavior; path-scoped dump; and DOCX/XLSX-to-PDF parity. XLSX engine hardening can proceed beside the versioned registry/common-receipt work; only each feature's office exposure depends on both layers. Worksheet clone and multi-sheet selection state remain in the ledger as explicitly counted beyond-parity differentiators rather than being attributed to OfficeCLI.

Architecture work that can proceed independently, but must stay in its own PR:

  • #174 — scheduler-cooperative Office parsing. Implement pure-MoonBit fuel-bounded resumable machines with sync and async drivers. Use public moonbitlang/async APIs, no C stubs, native threads, private runtime imports, or duplicated parsers. Scheduling fuel is separate from security budgets.

Other existing component issues (#74, #76, #94, #95, and #155) remain open and must not be silently folded into an Office parity PR. Publication in #155 needs separate authorization.

Small-PR rule

Do not repeat the size of #172 or #173. A PR should have one root cause or one user-visible capability, one independently testable acceptance boundary, and a clear revert story.

  • Do not stack a new capability on an unmerged feature branch.
  • Separate reusable foundation, CLI exposure, and unrelated compatibility fixes when each can land and be useful independently.
  • As a review trigger, stop and split when a PR approaches roughly 1,000 hand-written changed lines or touches more than about 15 production files. Generated interfaces/snapshots are excluded, but they do not justify mixing scopes.
  • If a review discovers a non-blocking issue outside the stated acceptance criteria, file an issue and fix it in a new small PR. Do not let it expand the current PR.
  • Commit logical, buildable steps regularly. Avoid one final catch-all commit.

Required review and landing protocol

For every PR:

  1. Rebase on current origin/main; do not review a stale stacked base.
  2. Run moon info && moon fmt, inspect all .mbti changes, then run moon check and the focused tests.
  3. Run the relevant native, Wasm, and JS gates. For Office CLI changes, build the native CLI and run moon cram test office/cmd/office/cram from the same stub setup used by CI. Mutation work also needs Microsoft DocumentFormat.OpenXml validation with .NET 8. Pin all validation runs to one exact commit. To reduce wall time, use a separate detached worktree (or at least a separate --target-dir) for each target so native, Wasm, and JS can run concurrently without contending on .moon-lock; do not edit the source while those runs are active.
  4. Ask a brand-new, ephemeral Codex CLI session to review the exact pushed head. Use xhigh for normal reviews; escalate to max or ultra whenever uncertainty remains, with ultra required for security architecture and final epic acceptance. Never reuse the implementation session as the approving reviewer.
  5. If the reviewer changes the code, commit the fix and run a new fresh review over the changed exact head (or a clearly bounded delta ending at it).
  6. Merge only when exact-head CI and the fresh review are both green. Update or close the corresponding issue immediately after landing.

CI-equivalent local commands are documented in .github/workflows/ci.yml. The complete matrix currently includes:

moon info
moon fmt
moon check
moon check --target wasm
moon check --target js
moon test --target wasm
moon test --target js
moon test --target native

The workspace currently resolves moonbitlang/async@0.20.2. An async upgrade is allowed when it has a concrete benefit, but it must be an isolated dependency PR with native/Wasm scheduler and filesystem tests; do not combine it with a feature PR.

Deferred from the initial F1 baseline

  • PowerPoint/PPTX.
  • MCP, resident mode, and live watch/selection.
  • Plugin and language-SDK wrappers.
  • Formula verification/cache refresh and array-formula add/set/readback, XLSX cascade-safe worksheet rename plus reorder/tab-color/visibility and print-title rows/columns, bounded boolean/row-by-column-name query, pane, cell/range-formatting, row/column dimension/autofit/outline and structural-edit lifecycle, partial-cell shifts, workbook/date-system/calculation/active-tab/first-sheet/view state, stable chart enumerate/create/update/remove with cleanup, unmerge/AutoFilter and ordinary table/validation/conditional-format/page-break lifecycle, existing-DOCX paragraph/table/row/column move-copy-swap, full common table/row/ cell properties, all four style types, numbering, header/footer, paragraph/positional-tab, permission-range, break, and advanced section editing (including page borders, paper sources, vertical alignment/text direction, and section note controls), DOCX locale/RTL, complete typed field/bookmark existing add/update/removal/refresh, root document defaults, page-background color, and settings, content/move/property revision read/accept/reject and matching tracked-change authoring, embedded charts, source-pinned SDT/legacy-form lifecycle, and existing add/update/remove for notes, pictures, equations, watermarks, text boxes, and shapes; path-scoped dump, DOCX/XLSX-to-PDF export, and richer engine-backed XLSX operations are not part of the initial F1 gate; they are explicit later slices in the ledger. Whole-worksheet clone, multi-sheet selection state, and typed DOCX/XLSX user-custom-property inventory/lifecycle for string, bool, signed-32-bit integer, finite-double, and UTC-date values are separately labeled beyond-parity slices; OfficeCLI has no typed user-custom-property lifecycle, and its lossless Word dump falls back to whole-part raw carriage.
  • Word-identical portable DOCX pagination remains a differentiator beyond basic backend-provenance PDF export. OLE, diagrams, and other low-frequency long-tail Office features still require demonstrated workflow demand and a reviewed issue sequence.
  • Backwards-compatibility aliases for unreleased APIs.

Keep docs/office-major-parity.md, issue #139, and this handoff synchronized as work lands. Issue #169 records only the installed-command baseline. Issue #139 and completion evidence for every in-scope major-ledger slice are the authority for declaring the non-PPT parity effort complete.