Skip to content

Commit 459b647

Browse files
committed
feat(node): add Node.js/Bun TypeScript bindings (fleischwolf-node)
Add a napi-rs binding crate exposing the DocumentConverter to Node.js and Bun via a single native N-API addon (Bun implements N-API, so the same binary loads in both — no rebuild). The TypeScript surface mirrors the Rust API: - convertFile / convert (in-memory bytes) plus Promise-returning *Async variants that run the CPU-bound work off the event loop - Markdown or docling-core JSON output; strict Markdown; placeholder / embedded / referenced image modes; allowedFormats; fetchImages - a reusable DocumentConverter class - streamFileMarkdown: an async generator over Markdown chunks in document order (the streaming win for PDF), wrapping the native callback API - supportedFormats / formatFromName helpers Packaging: @napi-rs/cli build, generated + hand-written TypeScript types, a self-referencing exports map (works from ESM and CommonJS), a smoke test that passes identically under Node and Bun, and runnable Node + Bun/TS examples. Generated artifacts (.node, native.js/.d.ts, node_modules) are gitignored; run `npm install && npm run build`. The crate joins the workspace (covered by fmt/clippy/test) but is marked publish=false — it ships to npm, not crates.io. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L7XwjsWSHWjrh39JxMLuB3
1 parent be7be9c commit 459b647

14 files changed

Lines changed: 1287 additions & 0 deletions

File tree

Cargo.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ members = [
1414
"crates/fleischwolf",
1515
"crates/fleischwolf-cli",
1616
"crates/fleischwolf-pdf",
17+
# Node.js / Bun N-API bindings (published to npm, not crates.io).
18+
"crates/fleischwolf-node",
1719
]
1820

1921
[workspace.package]

README.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,31 @@ buffered `export_to_markdown_with_images` path. Use
152152
The CLI streams Markdown by default (`--no-stream` opts back into buffering;
153153
`--to json` and `--images referenced` always buffer).
154154

155+
## Node.js / Bun bindings
156+
157+
Native TypeScript bindings live in
158+
[`crates/fleischwolf-node`](./crates/fleischwolf-node) (built with
159+
[napi-rs](https://napi.rs)). They ship a real `.node` addon that loads in both
160+
Node.js and Bun (Bun implements N-API — same binary, no rebuild), exposing the
161+
converter with the same knobs as the Rust API: Markdown / docling JSON output,
162+
`strict` mode, image modes, allowed-format restriction, `fetchImages`, sync +
163+
async (`Promise`) calls, and a `streamFileMarkdown` async generator.
164+
165+
```bash
166+
cd crates/fleischwolf-node && npm install && npm run build
167+
```
168+
169+
```ts
170+
import { convertFile, convertFileAsync, streamFileMarkdown } from 'fleischwolf'
171+
172+
const { content } = convertFile('report.docx') // Markdown
173+
const json = await convertFileAsync('paper.pdf', { to: 'json' })
174+
for await (const chunk of streamFileMarkdown('paper.pdf')) process.stdout.write(chunk)
175+
```
176+
177+
See [`crates/fleischwolf-node/README.md`](./crates/fleischwolf-node/README.md)
178+
for the full API and runnable Node + Bun examples.
179+
155180
## Testing
156181

157182
All commands run from the `fleischwolf/` workspace root.
@@ -286,7 +311,9 @@ fleischwolf — bigger means Rust wins by more.
286311
|---|---|---|
287312
| `fleischwolf-core` | `DoclingDocument` model + serializers | `docling-core` |
288313
| `fleischwolf` | `DocumentConverter`, source loading, backends | `docling` |
314+
| `fleischwolf-pdf` | PDF/image ML pipeline (pdfium + ONNX layout/table/OCR) | `docling` PDF pipeline |
289315
| `fleischwolf-cli` | command-line interface | `docling.cli` |
316+
| `fleischwolf-node` | Node.js / Bun N-API bindings (npm package) ||
290317

291318
## License
292319

crates/fleischwolf-node/.gitignore

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# npm
2+
node_modules/
3+
package-lock.json
4+
5+
# Build artifacts generated by `napi build` (regenerate with `npm run build`):
6+
# - the native addon (large; carries the linked ONNX runtime)
7+
# - the platform loader + its types
8+
*.node
9+
native.js
10+
native.d.ts
11+
12+
# Example scratch output
13+
examples/artifacts/

crates/fleischwolf-node/Cargo.toml

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Node.js / Bun bindings for Fleischwolf, built with napi-rs.
2+
#
3+
# Produces a native N-API addon (`.node`) that loads in both Node.js and Bun
4+
# (Bun implements N-API). The TypeScript surface mirrors the Rust
5+
# `DocumentConverter`: convert a file or in-memory bytes to Markdown or docling
6+
# JSON, with the same knobs (strict Markdown, image modes, allowed formats,
7+
# external image fetching) plus incremental Markdown streaming.
8+
#
9+
# This crate is intentionally kept OUT of the default workspace members' publish
10+
# flow (it ships to npm, not crates.io). It is a `cdylib` — the only artifact is
11+
# the addon, there is no Rust library API here.
12+
[package]
13+
name = "fleischwolf-node"
14+
description = "Node.js / Bun bindings for Fleischwolf (a Rust port of docling)."
15+
version.workspace = true
16+
edition.workspace = true
17+
license.workspace = true
18+
repository.workspace = true
19+
homepage.workspace = true
20+
readme = "README.md"
21+
# ort (via fleischwolf-pdf) needs a newer compiler than the std-only crates.
22+
rust-version = "1.82"
23+
# Not a crates.io library — the deliverable is the npm package.
24+
publish = false
25+
26+
[lib]
27+
# napi addons are C-ABI dynamic libraries loaded by the JS runtime.
28+
crate-type = ["cdylib"]
29+
30+
[dependencies]
31+
fleischwolf = { path = "../fleischwolf", version = "0.6.1" }
32+
# Node-API bindings. `napi4` enables threadsafe functions, used for streaming.
33+
napi = { version = "2", default-features = false, features = ["napi4"] }
34+
napi-derive = "2"
35+
36+
[build-dependencies]
37+
napi-build = "2"

crates/fleischwolf-node/README.md

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# fleischwolf (Node.js / Bun bindings)
2+
3+
Native [Node.js](https://nodejs.org) / [Bun](https://bun.sh) bindings for
4+
[Fleischwolf](https://github.com/artiz/fleischwolf) — a Rust port of
5+
[docling](https://github.com/docling-project/docling). Convert Markdown, HTML,
6+
DOCX, PPTX, XLSX, EPUB, ODF, LaTeX, email, PDF, images and more into a unified
7+
`DoclingDocument`, and export it as **Markdown** or docling-core **JSON**.
8+
9+
Built with [napi-rs](https://napi.rs), so it ships a real native addon (`.node`)
10+
that loads in both Node.js and Bun (Bun implements N-API) — the same binary, no
11+
rebuild between runtimes.
12+
13+
## Install & build
14+
15+
This package lives in the Fleischwolf Cargo workspace and builds the addon from
16+
the Rust source. You need a Rust toolchain (1.82+) and Node.js 14+ (or Bun).
17+
18+
```bash
19+
cd crates/fleischwolf-node
20+
npm install # installs @napi-rs/cli
21+
npm run build # release build → index.<platform>.node + native.js/.d.ts
22+
# npm run build:debug # faster, unoptimized
23+
```
24+
25+
> The addon statically links the ONNX runtime used by the PDF/image pipeline, so
26+
> the built `.node` is large. Declarative formats (Markdown, HTML, DOCX, …) don't
27+
> touch it; only PDF/image conversion loads the ML models (downloaded on first
28+
> use, like the CLI).
29+
30+
## Quick start
31+
32+
```js
33+
import { convertFile, convert, DocumentConverter } from 'fleischwolf'
34+
35+
// Convert a file — format detected from the extension.
36+
const { content } = convertFile('report.docx')
37+
console.log(content) // Markdown
38+
39+
// Convert in-memory bytes (e.g. an upload) — pass the format explicitly.
40+
const md = convert({ name: 'notes', data: Buffer.from('# Hi\n'), format: 'md' })
41+
42+
// docling-core JSON instead of Markdown.
43+
const json = convertFile('report.docx', { to: 'json' })
44+
45+
// Reuse a converter across many documents.
46+
const converter = new DocumentConverter({ strict: true })
47+
const a = converter.convert({ name: 'a.md', data: Buffer.from('# A\n') })
48+
```
49+
50+
CommonJS works too: `const { convertFile } = require('fleischwolf')`.
51+
52+
### Async (off the event loop)
53+
54+
Conversion is CPU-bound; the `*Async` variants run it on the libuv thread pool
55+
so the event loop stays free. Prefer these for PDF/image and for servers.
56+
57+
```js
58+
import { convertFileAsync } from 'fleischwolf'
59+
60+
const res = await convertFileAsync('paper.pdf', { to: 'json' })
61+
```
62+
63+
### Streaming Markdown
64+
65+
`streamFileMarkdown` yields Markdown chunks in document order as conversion
66+
progresses. For PDF (whose pages convert in parallel) output starts flowing
67+
before the whole document is done; concatenating the chunks reproduces the
68+
buffered `content` byte-for-byte.
69+
70+
```js
71+
import { streamFileMarkdown } from 'fleischwolf'
72+
73+
for await (const chunk of streamFileMarkdown('paper.pdf')) {
74+
process.stdout.write(chunk)
75+
}
76+
```
77+
78+
### Images
79+
80+
Pick how pictures render in Markdown with `imageMode`:
81+
82+
```js
83+
// Inline, self-contained: ![Image](data:image/png;base64,…)
84+
convertFile('slides.pptx', { imageMode: 'embedded' })
85+
86+
// Referenced: links + the image bytes to write yourself.
87+
const res = convertFile('slides.pptx', { imageMode: 'referenced', artifactsDir: 'assets' })
88+
for (const img of res.images) {
89+
await fs.writeFile(img.path, img.data) // e.g. assets/image_000000.png
90+
}
91+
```
92+
93+
JSON output always embeds extracted images as data URIs.
94+
95+
## API
96+
97+
### Functions
98+
99+
| Function | Returns | Notes |
100+
| --- | --- | --- |
101+
| `convertFile(path, options?)` | `ConvertResult` | Detects format from the extension. |
102+
| `convert(input, options?)` | `ConvertResult` | In-memory bytes (`{ name, data, format? }`). |
103+
| `convertFileAsync(path, options?)` | `Promise<ConvertResult>` | Off the event loop. |
104+
| `convertAsync(input, options?)` | `Promise<ConvertResult>` | Off the event loop. |
105+
| `streamFileMarkdown(path, options?)` | `AsyncGenerator<string>` | Markdown chunks in document order. |
106+
| `supportedFormats()` | `string[]` | Supported input format ids. |
107+
| `formatFromName(name)` | `string \| null` | Detect a format id from a filename/extension. |
108+
109+
`DocumentConverter` is the reusable form: `new DocumentConverter(converterOptions)`
110+
then `convert` / `convertFile` / `convertFileAsync` / `convertAsync` /
111+
`convertFileStreaming`. Converter config (`strict`, `fetchImages`,
112+
`allowedFormats`) is set once on the constructor; output options (`to`,
113+
`imageMode`, `artifactsDir`) are per call.
114+
115+
### Options
116+
117+
- `to`: `"markdown"` (default) or `"json"`.
118+
- `imageMode`: `"placeholder"` (default), `"embedded"`, or `"referenced"`.
119+
- `artifactsDir`: directory name used in `referenced` links (default `"artifacts"`).
120+
- `strict`: cleaner, more conformant Markdown instead of docling's byte-for-byte
121+
legacy output (Markdown only).
122+
- `fetchImages`: for HTML/EPUB, resolve and embed external `<img src>`. Off by
123+
default; fetches http(s) URLs over the network — enable only for trusted input.
124+
- `allowedFormats`: restrict the converter to these format ids/extensions.
125+
126+
### `ConvertResult`
127+
128+
```ts
129+
interface ConvertResult {
130+
content: string // Markdown or JSON, per `to`
131+
format: string // detected input format id
132+
status: string // "success" | "partial_success" | "failure"
133+
inputName: string
134+
images: { path: string; data: Buffer }[] // for the `referenced` image mode
135+
}
136+
```
137+
138+
Full TypeScript types are generated into `index.d.ts` / `native.d.ts`.
139+
140+
## Examples
141+
142+
- [`examples/node-basic.mjs`](examples/node-basic.mjs) — Node.js (ESM).
143+
- [`examples/bun-basic.ts`](examples/bun-basic.ts) — Bun + TypeScript, with async and streaming.
144+
145+
```bash
146+
npm run build # once
147+
node examples/node-basic.mjs
148+
bun run examples/bun-basic.ts
149+
node test/smoke.mjs # or: bun test/smoke.mjs
150+
```
151+
152+
## License
153+
154+
MIT, same as the rest of Fleischwolf.

crates/fleischwolf-node/build.rs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
// napi-build wires up the N-API symbol resolution and, on Windows, the
2+
// delay-load shim so the addon links against the host `node.exe`/`bun`.
3+
fn main() {
4+
napi_build::setup();
5+
}
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
// Fleischwolf from Bun, in TypeScript. Run with:
2+
//
3+
// bun run examples/bun-basic.ts
4+
//
5+
// Bun implements N-API, so the exact same native addon loads — no rebuild. This
6+
// example leans on the TypeScript types and shows async conversion and the
7+
// streaming async generator. It also runs unchanged under `tsx`/`ts-node` on Node.
8+
9+
import { fileURLToPath } from 'node:url'
10+
import { dirname, join } from 'node:path'
11+
12+
import {
13+
convertFileAsync,
14+
streamFileMarkdown,
15+
DocumentConverter,
16+
type ConvertResult,
17+
} from 'fleischwolf'
18+
19+
const here = dirname(fileURLToPath(import.meta.url))
20+
const html = join(here, 'inputs', 'sample.html')
21+
22+
// 1. Async conversion — the CPU-bound work runs off the event loop, so this
23+
// scales to PDF/image without blocking. Fully typed: `res` is ConvertResult.
24+
const res: ConvertResult = await convertFileAsync(html, { to: 'markdown' })
25+
console.log(`converted "${res.inputName}" (${res.format}) → ${res.status}`)
26+
console.log(res.content)
27+
28+
// 2. Streaming Markdown: chunks arrive in document order as conversion
29+
// progresses. For PDF, pages stream out as each finishes — output starts
30+
// before the whole document is done. Concatenating the chunks equals the
31+
// buffered `content`.
32+
console.log('--- streamed ---')
33+
let streamed = ''
34+
for await (const chunk of streamFileMarkdown(html)) {
35+
process.stdout.write(chunk)
36+
streamed += chunk
37+
}
38+
console.log(`\n(streamed ${streamed.length} bytes)`)
39+
40+
// 3. Embedded images: pictures become inline base64 data URIs, so the Markdown
41+
// is self-contained. (This HTML has none; the option is a no-op here, but
42+
// the same call handles DOCX/PPTX/PDF figures.)
43+
const embedded = new DocumentConverter().convertFile(html, { imageMode: 'embedded' })
44+
console.log('embedded-image Markdown length:', embedded.content.length)
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
<!doctype html>
2+
<html>
3+
<head>
4+
<title>Fleischwolf sample</title>
5+
</head>
6+
<body>
7+
<h1>Quarterly report</h1>
8+
<p>Revenue grew <strong>18%</strong> year over year.</p>
9+
<h2>Highlights</h2>
10+
<ul>
11+
<li>Launched the new pipeline</li>
12+
<li>Signed three enterprise customers</li>
13+
</ul>
14+
<table>
15+
<thead>
16+
<tr><th>Region</th><th>Growth</th></tr>
17+
</thead>
18+
<tbody>
19+
<tr><td>EMEA</td><td>22%</td></tr>
20+
<tr><td>APAC</td><td>15%</td></tr>
21+
</tbody>
22+
</table>
23+
</body>
24+
</html>
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
// Fleischwolf from Node.js (ESM). Run with:
2+
//
3+
// node examples/node-basic.mjs
4+
//
5+
// Shows the four common paths: convert a file, convert in-memory bytes, emit
6+
// docling-core JSON, and reuse a converter. See bun-basic.ts for TypeScript and
7+
// streaming.
8+
9+
import { fileURLToPath } from 'node:url'
10+
import { dirname, join } from 'node:path'
11+
12+
import {
13+
convertFile,
14+
convert,
15+
DocumentConverter,
16+
supportedFormats,
17+
} from 'fleischwolf'
18+
19+
const here = dirname(fileURLToPath(import.meta.url))
20+
21+
// 1. Convert a file on disk — format detected from the extension.
22+
const html = join(here, 'inputs', 'sample.html')
23+
const doc = convertFile(html)
24+
console.log('--- Markdown from sample.html ---')
25+
console.log(doc.content)
26+
27+
// 2. Convert in-memory bytes (e.g. an upload) — pass the format explicitly.
28+
const md = '# Notes\n\nInline math $E = mc^2$ and a [link](https://example.com).\n'
29+
const fromBytes = convert({ name: 'notes', data: Buffer.from(md), format: 'md' })
30+
console.log('--- Round-tripped Markdown ---')
31+
console.log(fromBytes.content)
32+
33+
// 3. Emit docling-core's native JSON wire format instead of Markdown.
34+
const asJson = convertFile(html, { to: 'json' })
35+
const model = JSON.parse(asJson.content)
36+
console.log('--- docling JSON ---')
37+
console.log('schema:', model.schema_name, model.version)
38+
console.log('text nodes:', model.texts.length, '| tables:', model.tables.length)
39+
40+
// 4. Reuse a converter across many documents (config parsed once). `strict`
41+
// yields cleaner, more conformant Markdown.
42+
const converter = new DocumentConverter({ strict: true })
43+
for (const [name, body] of [
44+
['a.md', '# First\n'],
45+
['b.md', '# Second\n'],
46+
]) {
47+
const r = converter.convert({ name, data: Buffer.from(body) })
48+
console.log(`converted ${name}: ${r.content.trim()}`)
49+
}
50+
51+
console.log('\nsupported formats:', supportedFormats().join(', '))

0 commit comments

Comments
 (0)