|
| 1 | +# fleischwolf (Node.js / Bun bindings) |
| 2 | + |
| 3 | +Native [Node.js](https://nodejs.org) / [Bun](https://bun.sh) bindings for |
| 4 | +[Fleischwolf](https://github.com/artiz/fleischwolf) — a Rust port of |
| 5 | +[docling](https://github.com/docling-project/docling). Convert Markdown, HTML, |
| 6 | +DOCX, PPTX, XLSX, EPUB, ODF, LaTeX, email, PDF, images and more into a unified |
| 7 | +`DoclingDocument`, and export it as **Markdown** or docling-core **JSON**. |
| 8 | + |
| 9 | +Built with [napi-rs](https://napi.rs), so it ships a real native addon (`.node`) |
| 10 | +that loads in both Node.js and Bun (Bun implements N-API) — the same binary, no |
| 11 | +rebuild between runtimes. |
| 12 | + |
| 13 | +## Install & build |
| 14 | + |
| 15 | +This package lives in the Fleischwolf Cargo workspace and builds the addon from |
| 16 | +the Rust source. You need a Rust toolchain (1.82+) and Node.js 14+ (or Bun). |
| 17 | + |
| 18 | +```bash |
| 19 | +cd crates/fleischwolf-node |
| 20 | +npm install # installs @napi-rs/cli |
| 21 | +npm run build # release build → index.<platform>.node + native.js/.d.ts |
| 22 | +# npm run build:debug # faster, unoptimized |
| 23 | +``` |
| 24 | + |
| 25 | +> The addon statically links the ONNX runtime used by the PDF/image pipeline, so |
| 26 | +> the built `.node` is large. Declarative formats (Markdown, HTML, DOCX, …) don't |
| 27 | +> touch it; only PDF/image conversion loads the ML models (downloaded on first |
| 28 | +> use, like the CLI). |
| 29 | +
|
| 30 | +## Quick start |
| 31 | + |
| 32 | +```js |
| 33 | +import { convertFile, convert, DocumentConverter } from 'fleischwolf' |
| 34 | + |
| 35 | +// Convert a file — format detected from the extension. |
| 36 | +const { content } = convertFile('report.docx') |
| 37 | +console.log(content) // Markdown |
| 38 | + |
| 39 | +// Convert in-memory bytes (e.g. an upload) — pass the format explicitly. |
| 40 | +const md = convert({ name: 'notes', data: Buffer.from('# Hi\n'), format: 'md' }) |
| 41 | + |
| 42 | +// docling-core JSON instead of Markdown. |
| 43 | +const json = convertFile('report.docx', { to: 'json' }) |
| 44 | + |
| 45 | +// Reuse a converter across many documents. |
| 46 | +const converter = new DocumentConverter({ strict: true }) |
| 47 | +const a = converter.convert({ name: 'a.md', data: Buffer.from('# A\n') }) |
| 48 | +``` |
| 49 | + |
| 50 | +CommonJS works too: `const { convertFile } = require('fleischwolf')`. |
| 51 | + |
| 52 | +### Async (off the event loop) |
| 53 | + |
| 54 | +Conversion is CPU-bound; the `*Async` variants run it on the libuv thread pool |
| 55 | +so the event loop stays free. Prefer these for PDF/image and for servers. |
| 56 | + |
| 57 | +```js |
| 58 | +import { convertFileAsync } from 'fleischwolf' |
| 59 | + |
| 60 | +const res = await convertFileAsync('paper.pdf', { to: 'json' }) |
| 61 | +``` |
| 62 | + |
| 63 | +### Streaming Markdown |
| 64 | + |
| 65 | +`streamFileMarkdown` yields Markdown chunks in document order as conversion |
| 66 | +progresses. For PDF (whose pages convert in parallel) output starts flowing |
| 67 | +before the whole document is done; concatenating the chunks reproduces the |
| 68 | +buffered `content` byte-for-byte. |
| 69 | + |
| 70 | +```js |
| 71 | +import { streamFileMarkdown } from 'fleischwolf' |
| 72 | + |
| 73 | +for await (const chunk of streamFileMarkdown('paper.pdf')) { |
| 74 | + process.stdout.write(chunk) |
| 75 | +} |
| 76 | +``` |
| 77 | + |
| 78 | +### Images |
| 79 | + |
| 80 | +Pick how pictures render in Markdown with `imageMode`: |
| 81 | + |
| 82 | +```js |
| 83 | +// Inline, self-contained:  |
| 84 | +convertFile('slides.pptx', { imageMode: 'embedded' }) |
| 85 | + |
| 86 | +// Referenced: links + the image bytes to write yourself. |
| 87 | +const res = convertFile('slides.pptx', { imageMode: 'referenced', artifactsDir: 'assets' }) |
| 88 | +for (const img of res.images) { |
| 89 | + await fs.writeFile(img.path, img.data) // e.g. assets/image_000000.png |
| 90 | +} |
| 91 | +``` |
| 92 | + |
| 93 | +JSON output always embeds extracted images as data URIs. |
| 94 | + |
| 95 | +## API |
| 96 | + |
| 97 | +### Functions |
| 98 | + |
| 99 | +| Function | Returns | Notes | |
| 100 | +| --- | --- | --- | |
| 101 | +| `convertFile(path, options?)` | `ConvertResult` | Detects format from the extension. | |
| 102 | +| `convert(input, options?)` | `ConvertResult` | In-memory bytes (`{ name, data, format? }`). | |
| 103 | +| `convertFileAsync(path, options?)` | `Promise<ConvertResult>` | Off the event loop. | |
| 104 | +| `convertAsync(input, options?)` | `Promise<ConvertResult>` | Off the event loop. | |
| 105 | +| `streamFileMarkdown(path, options?)` | `AsyncGenerator<string>` | Markdown chunks in document order. | |
| 106 | +| `supportedFormats()` | `string[]` | Supported input format ids. | |
| 107 | +| `formatFromName(name)` | `string \| null` | Detect a format id from a filename/extension. | |
| 108 | + |
| 109 | +`DocumentConverter` is the reusable form: `new DocumentConverter(converterOptions)` |
| 110 | +then `convert` / `convertFile` / `convertFileAsync` / `convertAsync` / |
| 111 | +`convertFileStreaming`. Converter config (`strict`, `fetchImages`, |
| 112 | +`allowedFormats`) is set once on the constructor; output options (`to`, |
| 113 | +`imageMode`, `artifactsDir`) are per call. |
| 114 | + |
| 115 | +### Options |
| 116 | + |
| 117 | +- `to`: `"markdown"` (default) or `"json"`. |
| 118 | +- `imageMode`: `"placeholder"` (default), `"embedded"`, or `"referenced"`. |
| 119 | +- `artifactsDir`: directory name used in `referenced` links (default `"artifacts"`). |
| 120 | +- `strict`: cleaner, more conformant Markdown instead of docling's byte-for-byte |
| 121 | + legacy output (Markdown only). |
| 122 | +- `fetchImages`: for HTML/EPUB, resolve and embed external `<img src>`. Off by |
| 123 | + default; fetches http(s) URLs over the network — enable only for trusted input. |
| 124 | +- `allowedFormats`: restrict the converter to these format ids/extensions. |
| 125 | + |
| 126 | +### `ConvertResult` |
| 127 | + |
| 128 | +```ts |
| 129 | +interface ConvertResult { |
| 130 | + content: string // Markdown or JSON, per `to` |
| 131 | + format: string // detected input format id |
| 132 | + status: string // "success" | "partial_success" | "failure" |
| 133 | + inputName: string |
| 134 | + images: { path: string; data: Buffer }[] // for the `referenced` image mode |
| 135 | +} |
| 136 | +``` |
| 137 | + |
| 138 | +Full TypeScript types are generated into `index.d.ts` / `native.d.ts`. |
| 139 | + |
| 140 | +## Examples |
| 141 | + |
| 142 | +- [`examples/node-basic.mjs`](examples/node-basic.mjs) — Node.js (ESM). |
| 143 | +- [`examples/bun-basic.ts`](examples/bun-basic.ts) — Bun + TypeScript, with async and streaming. |
| 144 | + |
| 145 | +```bash |
| 146 | +npm run build # once |
| 147 | +node examples/node-basic.mjs |
| 148 | +bun run examples/bun-basic.ts |
| 149 | +node test/smoke.mjs # or: bun test/smoke.mjs |
| 150 | +``` |
| 151 | + |
| 152 | +## License |
| 153 | + |
| 154 | +MIT, same as the rest of Fleischwolf. |
0 commit comments