Skip to content

Commit daf0775

Browse files
brianckeeganclaude
andauthored
Fix Zotero 8+ compatibility: file picker crash and ChromeUtils.import errors (#41)
* Fix Save As crash on Zotero 8+ by using Zotero's FilePicker module Firefox 128+ (Zotero 8) changed nsIFilePicker.init() to take a BrowsingContext instead of a window, so the raw nsIFilePicker code in promptSavePath() threw NS_ERROR_XPC_BAD_CONVERT_JS on Zotero 8/9 and the markdown .zip export never showed a save dialog. Route the dialog through the toolkit's FilePickerHelper, which delegates to Zotero's own FilePicker module (chrome://zotero/content/modules/ filePicker.mjs). Zotero's wrapper passes window.browsingContext on new platforms and keeps working on Zotero 7, so the plugin no longer touches the changing XPCOM signature directly. Fixes #39 https://claude.ai/code/session_01KUFXvTTsWqrPNDJ8Vi1oPL * Silence ChromeUtils.import() errors on Zotero 8+ via build-time toolkit patch zotero-plugin-toolkit (through at least 5.1.2) detects the legacy JSM loader with `typeof ChromeUtils.import === "undefined"`. On Zotero 8+ (Firefox 140) ChromeUtils.import still exists but only throws "ChromeUtils.import() has been removed. Use importESModule()", so the toolkit rewrote Console.sys.mjs back to Console.jsm and the import threw once per helper constructed at startup — 16 console errors on every launch. Add an esbuild plugin that patches the bundled toolkit's _importESModule to try ChromeUtils.importESModule first (available since Firefox 106, so on every supported Zotero) and keep the legacy loader as a fallback for Zotero 7. The patch is an exact-match string replacement that fails the build loudly if the toolkit source changes, so a future toolkit upgrade cannot silently un-apply it; once the toolkit fixes its feature detection upstream, the plugin block can simply be deleted. Fixes #40 https://claude.ai/code/session_01KUFXvTTsWqrPNDJ8Vi1oPL * README: add a novice-friendly quick start and smooth the install path Getting-started editing pass aimed at first-time users: - Add a Quick start section that spells out the two-part setup (local docling-serve + plugin) and walks the happy path end to end, with a heads-up about the first-run model download. - Link installation guides for uv and pipx, and point readers without Python tooling at the container option. - Add a browser-based sanity check (the /ui playground) alongside curl. - Turn plugin installation into numbered steps, drop the stale "(when available)" hedge, and warn about Firefox auto-installing the .xpi. - Mark the from-source path as development-oriented (Node 20+) and scope the macOS .env note to the npm start workflow. - Fix the "within seconds" promise that contradicted the 2-10 minute first-run download, and de-emoji the model-weights heading so section anchors stay stable. https://claude.ai/code/session_01KUFXvTTsWqrPNDJ8Vi1oPL * Add an Exclude images option for text-and-tables-only markdown docling-serve defaults image_export_mode to "embedded", which inlines every figure as a base64 data URI and can grow a paper's .md by tens of megabytes. A new opt-in "Exclude images" checkbox in Settings → Output sends image_export_mode=placeholder instead, so each image becomes a tiny <!-- image --> comment and the markdown keeps only text and tables. The plugin now always sends image_export_mode explicitly ("embedded" when the box is unticked — identical to the previous server-default behavior), and the advancedJson escape hatch still overrides it for users who want "referenced" or other modes. Wired through prefs.js defaults, generated pref typings, the preferences pane (+ en-US strings), reset-to-defaults, buildConvertForm and its unit tests, and the README feature list. https://claude.ai/code/session_01KUFXvTTsWqrPNDJ8Vi1oPL * Add a Remove-images utility for existing docling markdown Markdown converted before the Exclude-images option (or with it off) carries every figure as an embedded base64 data URI, so a single paper's .md can be tens of megabytes. This adds a retroactive cleanup: select items and right-click "Remove images from markdown" (or Tools → Docling: Remove images from markdown…) to rewrite their .md attachments in place, replacing each markdown image or raw <img> tag with the same <!-- image --> placeholder docling emits in image_export_mode=placeholder. The pure transform lives in utils/stripImages.ts: line-based so fenced code blocks are skipped (image syntax shown in a code sample isn't an image), inline-safe so images inside table cells keep the table shape, and idempotent. The orchestrator in modules/removeImages.ts resolves selections the same way the zip export does (md attachments directly, PDFs via their matching .md sibling, parents via their PDF children), guards the destructive rewrite behind a confirm dialog, refuses to run while a conversion batch is in flight, and reports replaced-image count and bytes saved (new formatBytes helper) in the completion toast. Covered by unit tests for the transform (embedded/table/HTML/fence/ idempotence cases plus fast-check properties) and formatBytes. https://claude.ai/code/session_01KUFXvTTsWqrPNDJ8Vi1oPL --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 43f910e commit daf0775

18 files changed

Lines changed: 815 additions & 50 deletions

README.md

Lines changed: 98 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
# zotero-docling
22

3-
A Zotero 7+/9 plugin that converts PDF attachments to structured Markdown using
4-
the [Docling](https://github.com/docling-project/docling) document-understanding
5-
pipeline, and attaches the resulting `.md` file back to the same parent item.
3+
A Zotero plugin (Zotero 7 or later) that converts PDF attachments to structured
4+
Markdown using the [Docling](https://github.com/docling-project/docling)
5+
document-understanding pipeline, and attaches the resulting `.md` file back to
6+
the same parent item.
67

78
[![Release](https://img.shields.io/github/v/release/max3925vats/zotero-docling?style=flat-square)](https://github.com/max3925vats/zotero-docling/releases/latest)
89
[![License](https://img.shields.io/github/license/max3925vats/zotero-docling?style=flat-square)](LICENSE)
@@ -13,9 +14,44 @@ pipeline, and attaches the resulting `.md` file back to the same parent item.
1314

1415
---
1516

16-
Structured markdown makes a clean upstream for note apps, RAG pipelines,
17-
citation extraction, summarisation, literature reviews, and any downstream
18-
tool that prefers plain text over PDFs.
17+
Markdown is plain text, so converted papers drop straight into note apps like
18+
Obsidian, AI/RAG pipelines, citation extraction, summarisation, literature
19+
reviews — anything that prefers text over PDFs.
20+
21+
---
22+
23+
## Quick start
24+
25+
The setup has two halves: a small **local server**
26+
([docling-serve](https://github.com/docling-project/docling-serve)) does the
27+
actual PDF → Markdown conversion, and this **plugin** connects Zotero to it.
28+
You need both.
29+
30+
1. **Start the converter server.** The quickest path uses
31+
[uv](https://docs.astral.sh/uv/getting-started/installation/) (a Python
32+
tool installer); see [Install the server](#install-the-server-docling-serve)
33+
for pipx and Docker alternatives that may suit you better.
34+
35+
```bash
36+
uv tool install "docling-serve[ui]" # one-time install
37+
docling-serve run # leave this terminal open
38+
```
39+
40+
2. **Install the plugin.** Download the latest `.xpi` file from the
41+
[Releases page](https://github.com/max3925vats/zotero-docling/releases/latest)
42+
(if your browser tries to install it itself, right-click the link and
43+
choose "Save Link As…"). Then in Zotero: **Tools → Plugins → ⚙ →
44+
Install Plugin From File…** → pick the `.xpi`.
45+
46+
3. **Connect them.** In Zotero, open **Settings → zotero-docling** and click
47+
**Test Connection** — it should turn green. Then right-click any PDF in
48+
your library → **Convert with Docling**.
49+
50+
> [!NOTE]
51+
> Your very first conversion downloads model weights and can take 2–10
52+
> minutes while appearing to hang — that's normal, and later conversions
53+
> take seconds. See
54+
> [First conversion downloads model weights](#first-conversion-downloads-model-weights).
1955
2056
---
2157

@@ -34,6 +70,14 @@ tool that prefers plain text over PDFs.
3470
- **Full Docling options surfaced**: pipeline (standard / VLM), OCR + language,
3571
table mode, formula / code / chart / picture enrichments, VLM presets, plus
3672
an Advanced JSON escape hatch for anything not in the UI.
73+
- **Exclude images** (opt-in): keep the markdown text-and-tables only — each
74+
figure becomes a tiny placeholder comment instead of megabytes of embedded
75+
base64 image data. Tick **Exclude images** in **Settings → Output**.
76+
- **Remove images from existing markdown**: right-click already-converted
77+
items (or **Tools → Docling: Remove images from markdown…**) to rewrite
78+
their `.md` attachments in place, swapping every embedded image for a
79+
`<!-- image -->` placeholder — the retroactive version of Exclude images,
80+
no re-conversion needed.
3781
- **Per-parent status tags**: parents get tagged `docling/done`,
3882
`docling/incomplete`, or `docling/error` so you can filter your library by
3983
conversion status.
@@ -61,7 +105,8 @@ tool that prefers plain text over PDFs.
61105
## Install the server (docling-serve)
62106

63107
Pick one of the three paths below — they all yield a `docling-serve` you can
64-
point the plugin at. Roughly:
108+
point the plugin at. If you've never touched Python tooling, the container
109+
path avoids it entirely. Roughly:
65110

66111
| Option | Best for | Needs |
67112
| ------------- | --------------------------------------------------------------- | -------------------- |
@@ -71,13 +116,19 @@ point the plugin at. Roughly:
71116

72117
### Option A — uv (recommended)
73118

119+
Don't have `uv` yet? It's a one-line install — see the
120+
[uv installation guide](https://docs.astral.sh/uv/getting-started/installation/).
121+
74122
```bash
75123
uv tool install "docling-serve[ui]" # one-time install
76124
docling-serve run # leave this terminal open
77125
```
78126

79127
### Option B — pipx
80128

129+
Don't have `pipx` yet? See the
130+
[pipx installation guide](https://pipx.pypa.io/stable/installation/).
131+
81132
```bash
82133
pipx install "docling-serve[ui]" # one-time install
83134
docling-serve run
@@ -99,13 +150,16 @@ docker run --gpus all -p 5001:5001 \
99150

100151
### Sanity check (any option)
101152

153+
Open <http://localhost:5001/ui> in your browser — the docling-serve
154+
playground page should load. Or, from a terminal:
155+
102156
```bash
103157
curl http://localhost:5001/health # → {"status":"ok"}
104158
```
105159

106-
### ⚠️ First conversion downloads model weights
160+
### First conversion downloads model weights
107161

108-
The first time you convert a PDF, `docling-serve` downloads the model files
162+
⚠️ The first time you convert a PDF, `docling-serve` downloads the model files
109163
from Hugging Face. Expect **2–10 minutes** for the standard pipeline and
110164
**significantly longer** (multi-GB) for VLM presets. Subsequent conversions
111165
are fast. The Zotero progress window will appear to hang during the
@@ -123,13 +177,18 @@ docling-serve run
123177

124178
## Install the plugin
125179

126-
### From a release (when available)
180+
### From a release (recommended)
181+
182+
1. Download the latest `.xpi` from the
183+
[Releases page](https://github.com/max3925vats/zotero-docling/releases/latest).
184+
If your browser tries to install the file itself (Firefox does this),
185+
right-click the link and choose **Save Link As…** instead.
186+
2. In Zotero: **Tools → Plugins** → click the gear icon (⚙) →
187+
**Install Plugin From File…** → pick the downloaded `.xpi`.
127188

128-
Download the latest `.xpi` from the [Releases page](https://github.com/max3925vats/zotero-docling/releases),
129-
then in Zotero: **Tools → Plugins → ⚙ → Install Plugin From File…** → pick the
130-
`.xpi`.
189+
### From source (for development)
131190

132-
### From source
191+
Requires [Node.js](https://nodejs.org/) 20 or later:
133192

134193
```bash
135194
git clone https://github.com/max3925vats/zotero-docling.git
@@ -140,7 +199,8 @@ npm run build # outputs .scaffold/build/*.xpi
140199

141200
Then install the `.xpi` via **Tools → Plugins** as above.
142201

143-
> **macOS note**: profile paths that contain spaces (for example
202+
> **macOS note** (for the `npm start` development workflow): profile paths
203+
> that contain spaces (for example
144204
> `~/Library/Application Support/Zotero/...`) must be written literally in
145205
> `.env`**no** backslash escapes. See [`.env.example`](.env.example) for
146206
> the exact format.
@@ -151,9 +211,12 @@ Then install the `.xpi` via **Tools → Plugins** as above.
151211

152212
1. **Settings**: open Zotero → **Settings → zotero-docling**.
153213
2. Verify the server URL (default `http://localhost:5001`) and click **Test
154-
Connection** — it should turn green.
214+
Connection** — it should turn green. If it doesn't, make sure the
215+
`docling-serve` terminal from the install step is still running.
155216
3. Right-click a PDF (or parent item) in your library → **Convert with
156-
Docling**. Within seconds you'll see a `.md` child appear under the parent.
217+
Docling**. A `.md` child appears under the parent — within seconds once
218+
the [first-run model download](#first-conversion-downloads-model-weights)
219+
is out of the way.
157220
4. (Optional) tick **Auto-convert new PDF attachments** in Behavior to run
158221
conversion automatically as you import new PDFs.
159222

@@ -183,6 +246,24 @@ hasn't assigned a citation key). When a parent has multiple converted
183246
PDFs, the second and subsequent get `.1.md`, `.2.md`, … suffixes to
184247
prevent collisions.
185248

249+
### Removing images from already-converted markdown
250+
251+
Markdown converted without the **Exclude images** option embeds every
252+
figure as base64 data, which can make a single paper's `.md` tens of
253+
megabytes. To slim down existing files without re-converting:
254+
255+
1. Select the items (parents, PDF attachments, or `.md` attachments) in
256+
your library.
257+
2. **Right-click → Remove images from markdown**, or open
258+
**Tools → Docling: Remove images from markdown…**.
259+
3. Confirm the dialog. Each `.md` attachment is rewritten in place with
260+
every image replaced by a `<!-- image -->` placeholder; a toast reports
261+
how many images were removed and how much space was saved.
262+
263+
The image data is gone for good after this — re-convert the PDF if you
264+
ever need the figures back. Code blocks inside the markdown are left
265+
untouched.
266+
186267
---
187268

188269
## Known limitations

addon/content/preferences.xhtml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,12 @@
157157
data-l10n-id="pref-add-frontmatter"
158158
/>
159159
<description data-l10n-id="pref-frontmatter-help"></description>
160+
<checkbox
161+
id="zotero-docling-exclude-images"
162+
preference="excludeImages"
163+
data-l10n-id="pref-exclude-images"
164+
/>
165+
<description data-l10n-id="pref-exclude-images-help"></description>
160166
<hbox align="center">
161167
<html:label
162168
for="zotero-docling-export-folder"

addon/locale/en-US/addon.ftl

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,6 @@ menuitem-convert = Convert with Docling
22
menuitem-reconvert = Re-convert with Docling (replace)
33
menuitem-export-md-zip = Export markdown to .zip
44
menuitem-tools-export-md-zip = Docling: Export markdown to .zip…
5+
menuitem-remove-images = Remove images from markdown
6+
menuitem-tools-remove-images = Docling: Remove images from markdown…
57
pref-pane-label = zotero-docling

addon/locale/en-US/preferences.ftl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,9 @@ pref-attach-to-item =
3737
pref-add-frontmatter =
3838
.label = Prepend YAML frontmatter with Zotero metadata
3939
pref-frontmatter-help = Adds title, authors, year, doi, url, zotero_key, citation_key to every .md output as a YAML --- block at the top.
40+
pref-exclude-images =
41+
.label = Exclude images — keep only text and tables
42+
pref-exclude-images-help = By default every figure is embedded in the .md as base64 data, which can grow a paper's markdown by tens of megabytes. When enabled, each image is replaced with a tiny <!-- image --> placeholder comment instead.
4043
pref-export-folder = Export folder
4144
pref-export-folder-help = Absolute path. If set, every converted .md is ALSO written here. Filename uses the item's citation key when available, otherwise the Zotero item key. Leave empty to disable.
4245

addon/prefs.js

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,10 @@ pref("pipeline", "standard"); // "standard" | "vlm"
1616
pref("doOcr", true); // do_ocr
1717
pref("forceOcr", false); // force_ocr
1818
pref("tableMode", "accurate"); // table_mode: "accurate" | "fast"
19+
// image_export_mode: false → "embedded" (server default, base64 images
20+
// inline); true → "placeholder" (each image becomes a tiny <!-- image -->
21+
// comment, so the .md keeps text and tables only).
22+
pref("excludeImages", false);
1923

2024
// Tier 2: enrichments
2125
pref("doFormulaEnrichment", false); // do_formula_enrichment

src/modules/convert.ts

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,17 @@ export function buildConvertForm(
185185
form.append("force_ocr", String(getPref("forceOcr") ?? false));
186186
form.append("table_mode", String(getPref("tableMode") ?? "accurate"));
187187

188+
// Markdown image handling. "embedded" (the server default) inlines every
189+
// figure as a base64 data URI, which can bloat a paper's .md by tens of
190+
// megabytes. excludeImages swaps in "placeholder" — each image becomes a
191+
// tiny <!-- image --> comment, leaving text and tables only.
192+
form.append(
193+
"image_export_mode",
194+
((getPref("excludeImages") ?? false) as boolean)
195+
? "placeholder"
196+
: "embedded",
197+
);
198+
188199
// --- Tier 2: enrichments ---
189200
form.append(
190201
"do_formula_enrichment",

src/modules/markdownZipExport.ts

Lines changed: 18 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -201,48 +201,34 @@ function promptMissingMd(missing: number, total: number): MissingMdChoice {
201201
}
202202

203203
// ---------------------------------------------------------------------------
204-
// Native file-save dialog (nsIFilePicker)
204+
// Native file-save dialog
205205
// ---------------------------------------------------------------------------
206206

207207
/**
208208
* Show a Save As dialog and return the chosen absolute path, or null if
209209
* the user cancelled.
210+
*
211+
* Goes through Zotero's FilePicker module (via the toolkit helper) rather
212+
* than raw nsIFilePicker: Firefox 128+ (Zotero 8) requires a
213+
* BrowsingContext as the first argument to nsIFilePicker.init, and
214+
* Zotero's wrapper absorbs that difference across supported versions.
210215
*/
211216
async function promptSavePath(defaultName: string): Promise<string | null> {
212-
const Cc = (globalThis as any).Components?.classes;
213-
const Ci = (globalThis as any).Components?.interfaces;
214-
if (!Cc || !Ci || !Ci.nsIFilePicker) return null;
215-
const fp = Cc["@mozilla.org/filepicker;1"].createInstance(Ci.nsIFilePicker);
216-
const win =
217-
(Zotero as any).getMainWindow?.() ??
218-
(Zotero as any).getActiveZoteroPane?.()?.document?.defaultView ??
219-
null;
220-
fp.init(win, "Save markdown .zip", Ci.nsIFilePicker.modeSave);
221-
fp.defaultString = defaultName;
222-
fp.appendFilter("Zip archive", "*.zip");
223-
224-
const result: number = await new Promise((resolve) => {
225-
try {
226-
fp.open((rv: number) => resolve(rv));
227-
} catch {
228-
// Some platform builds expose open() as a sync return.
229-
try {
230-
resolve(fp.show());
231-
} catch {
232-
resolve(Ci.nsIFilePicker.returnCancel);
233-
}
234-
}
235-
});
236-
if (
237-
result !== Ci.nsIFilePicker.returnOK &&
238-
result !== Ci.nsIFilePicker.returnReplace
239-
) {
217+
let picked: string | false;
218+
try {
219+
picked = await new ztoolkit.FilePicker(
220+
"Save markdown .zip",
221+
"save",
222+
[["Zip archive", "*.zip"]],
223+
defaultName,
224+
).open();
225+
} catch (e) {
226+
log(`save dialog failed: ${(e as Error).message}`);
240227
return null;
241228
}
242-
const path: string | null = fp.file?.path ?? null;
243-
if (!path) return null;
229+
if (!picked) return null;
244230
// Force the .zip extension if the user typed one without it.
245-
return /\.zip$/i.test(path) ? path : `${path}.zip`;
231+
return /\.zip$/i.test(picked) ? picked : `${picked}.zip`;
246232
}
247233

248234
// ---------------------------------------------------------------------------

src/modules/menu.ts

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,12 +38,15 @@ import { notifyOnBatchComplete } from "../utils/notification";
3838
import { ConcurrencyLimiter } from "../utils/concurrencyLimiter";
3939
import { truncateMiddle, formatDuration } from "../utils/format";
4040
import { onExportMarkdownZipClick } from "./markdownZipExport";
41+
import { onRemoveImagesClick, resolveMdTargets } from "./removeImages";
4142

4243
const LOG = "[Docling/menu]";
4344
const MENU_CONVERT_ID = "zotero-docling-convert";
4445
const MENU_RECONVERT_ID = "zotero-docling-reconvert";
4546
const MENU_EXPORT_MD_ZIP_ID = "zotero-docling-export-md-zip";
4647
const TOOLS_EXPORT_MD_ZIP_ID = "zotero-docling-tools-export-md-zip";
48+
const MENU_REMOVE_IMAGES_ID = "zotero-docling-remove-images";
49+
const TOOLS_REMOVE_IMAGES_ID = "zotero-docling-tools-remove-images";
4750

4851
// Re-exports — used by other modules (markdownZipExport.ts) that need to
4952
// resolve a Zotero selection in the same way the right-click handlers do.
@@ -412,6 +415,8 @@ const ALL_MENU_IDS = [
412415
MENU_RECONVERT_ID,
413416
MENU_EXPORT_MD_ZIP_ID,
414417
TOOLS_EXPORT_MD_ZIP_ID,
418+
MENU_REMOVE_IMAGES_ID,
419+
TOOLS_REMOVE_IMAGES_ID,
415420
];
416421

417422
export function registerMenu(): void {
@@ -472,6 +477,30 @@ export function registerMenu(): void {
472477
},
473478
});
474479

480+
// Item right-click: Remove images from markdown — only when the selection
481+
// resolves to ≥1 markdown attachment to rewrite. The handler re-confirms
482+
// before touching any file.
483+
ztoolkit.Menu.register("item", {
484+
tag: "menuitem",
485+
id: MENU_REMOVE_IMAGES_ID,
486+
label: getString("menuitem-remove-images"),
487+
commandListener: () => {
488+
void onRemoveImagesClick("selection");
489+
},
490+
getVisibility: () => resolveMdTargets(getSelectedItems()).length > 0,
491+
});
492+
493+
// Tools → Docling: Remove images from markdown…. Same handler as the
494+
// right-click; toasts a hint when nothing usable is selected.
495+
ztoolkit.Menu.register("menuTools", {
496+
tag: "menuitem",
497+
id: TOOLS_REMOVE_IMAGES_ID,
498+
label: getString("menuitem-tools-remove-images"),
499+
commandListener: () => {
500+
void onRemoveImagesClick("tools");
501+
},
502+
});
503+
475504
log(`registerMenu: registered ${ALL_MENU_IDS.join(", ")}`);
476505
}
477506

src/modules/preferenceScript.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ const ALL_PREF_KEYS: ReadonlyArray<string> = [
2626
"doOcr",
2727
"forceOcr",
2828
"tableMode",
29+
"excludeImages",
2930
"doFormulaEnrichment",
3031
"doCodeEnrichment",
3132
"doChartExtraction",

0 commit comments

Comments
 (0)