OneNote: Improve reliability and enterprise account support#528
OneNote: Improve reliability and enterprise account support#528gurusince88 wants to merge 1 commit into
Conversation
Addresses issues obsidianmd#440, obsidianmd#398, obsidianmd#390, obsidianmd#334, and obsidianmd#313. Changes: - Add notes.read.all OAuth scope for M365/SharePoint enterprise accounts - Improve error messages to show specific details (401/403/429/504) - Replace fixed 60s retry with exponential backoff and jitter - Switch from data.json to frontmatter-based import tracking (onenote-id) - Fix mobile portrait UI: ensure sign-in button is visible The frontmatter approach (similar to Notion API importer) fixes the issue where "Skip previously imported" would incorrectly skip pages that had been deleted from the vault. Made-with: Cursor
|
Thanks for tackling this — the error message improvements and exponential backoff are solid. A few things worth addressing before merge: 🔴 Performance: O(n×m) file reads in const files = this.vault.getMarkdownFiles().filter(f => f.path.startsWith(outputFolder.path));
for (const file of files) {
const content = await this.vault.read(file); // full disk read per file, per pageThis reads every file in the output folder for every page being imported. With 500 OneNote pages and 500 existing files, that's 250,000 sequential disk reads. CONTRIBUTING.md calls this out explicitly: "Be performance minded. Your code will be used in vaults with 10,000 or even 100,000 notes." The fix is to build an in-memory // Before the import loop:
const existingIds = new Map<string, string>();
for (const file of this.vault.getMarkdownFiles()) {
if (!file.path.startsWith(outputFolder.path)) continue;
const content = await this.vault.read(file);
const match = content.match(/^onenote-id:\s*["']?([^"'\n]+)["']?$/m);
if (match) existingIds.set(match[1].trim(), file.path);
}
// Inside the loop:
if (!this.importPreviouslyImported && page.id && existingIds.has(page.id)) {
progress.reportSkipped(page.title!, 'previously imported');
continue;
}🔴 Rate limit retries are now bounded — regression from current behaviour Original code deliberately did not increment // don't increment the retryCount because we were told to backoff
retryCount // <-- originalThis PR changes it to Suggest keeping 429 retries unbounded (don't increment 🔴 The method is defined (~line 1107) but never called — 🟡 Personal account users don't need 🟡 YAML quoting on OneNote page IDs contain // Current — may break YAML parsers:
const frontmatter = `---\nonenote-id: ${page.id}\n---\n\n`;
// Safer:
const frontmatter = `---\nonenote-id: "${page.id}"\n---\n\n`;The regex match should also handle quoted values: 🟡 Migration: existing users will silently re-import everything Switching from The error branching in |
Summary
This PR improves the reliability of the OneNote importer, particularly for enterprise M365/SharePoint accounts.
Addresses: #440, #398, #390, #334, #313
Changes
notes.read.allfor M365/SharePoint enterprise accounts (fixes Importer for OneNote does not import anything #440, Importer Broken for OneNote - Limited how fast can be imported #334)data.jsontoonenote-idfrontmatter (like Notion API importer) to fix false "previously imported" skips (fixes Importer skipping items that are not previously imported #398)Test plan
Breaking changes
Notes imported after this change will have
onenote-idin their frontmatter. The olddata.jsontracking is no longer used, so users who previously imported notes will need to either: