Skip to content

Commit 6eca7be

Browse files
committed
feat: add async extraction API (extractAsync, renderAsync, renderMapAsync)
Cooperative async scan loop that yields to the event loop via scheduler.yield() (with setTimeout fallback), preventing main-thread blocking on large inputs. Supports AbortSignal cancellation. - src/async.ts: asyncScan with configurable yieldEvery (default 512) - mod.ts: extractAsync, renderAsync, renderMapAsync on Duckling() - All async methods tagged @experimental - 17 new tests (correctness, cancellation, interleaving, render parity) - Async benchmarks added to bench/extract.bench.ts - README updated with async API docs and AsyncScanOptions reference Bumps version to 0.3.0.
1 parent 93714ec commit 6eca7be

6 files changed

Lines changed: 561 additions & 2 deletions

File tree

README.md

Lines changed: 97 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ TypeScript and running anywhere — Deno, Node, or the browser.
6363
- [Redact PII](#redact-pii)
6464
- [Render entities](#render-entities)
6565
- [Map entities to components](#map-entities-to-components)
66+
- [Async extraction](#async-extraction)
6667
- [Custom entities](#custom-entities)
6768
- [Supported entities](#supported-entities)
6869
- [API reference](#api-reference)
@@ -71,6 +72,10 @@ TypeScript and running anywhere — Deno, Node, or the browser.
7172
- [`.render(text, fn)`](#rendertext-fn)
7273
- [`.renderMap(text, fn)`](#rendermaptext-fn)
7374
- [`.redact(text, opts?)`](#redacttext-opts)
75+
- [`.extractAsync(text, opts?)`](#extractasynctext-opts)
76+
- [`.renderAsync(text, fn, opts?)`](#renderasynctext-fn-opts)
77+
- [`.renderMapAsync(text, fn, opts?)`](#rendermapasynctext-fn-opts)
78+
- [`AsyncScanOptions`](#asyncscanoptions)
7479
- [`PIIParsers`](#piiparsers)
7580
- [`RedactOptions`](#redactoptions)
7681
- [`RenderFn`](#renderfn)
@@ -279,6 +284,43 @@ Like `.render()`, nested entities are handled automatically — child spans are
279284
mapped first, and the parent callback receives the already-mapped children as
280285
`(string | R)[]`.
281286

287+
### Async extraction
288+
289+
> **Experimental** — these methods are stable enough to use but the API may
290+
> change in a future minor release.
291+
292+
Every method has an async counterpart (`extractAsync`, `renderAsync`,
293+
`renderMapAsync`) that periodically yields to the browser event loop during
294+
scanning. This prevents long inputs from blocking the main thread — no Web
295+
Worker needed:
296+
297+
```ts
298+
import { Duckling } from "@claudiu-ceia/ts-duckling";
299+
300+
const controller = new AbortController();
301+
302+
const entities = await Duckling().extractAsync(longText, {
303+
signal: controller.signal, // cancel early if needed
304+
yieldEvery: 256, // yield every N scan positions (default 512)
305+
});
306+
```
307+
308+
Uses
309+
[`scheduler.yield()`](https://developer.mozilla.org/en-US/docs/Web/API/Scheduler/yield)
310+
when available (Chrome 129+, Edge 129+, Firefox 142+), falling back to
311+
`setTimeout(0)`.
312+
313+
`renderMapAsync` is especially useful in UI frameworks — extract and map
314+
entities to React elements without blocking paint:
315+
316+
```tsx
317+
const segments = await Duckling().renderMapAsync<JSX.Element>(
318+
text,
319+
({ entity, children }) => <mark data-kind={entity.kind}>{children}</mark>,
320+
);
321+
// → ["plain text", <mark>...</mark>, " more text"]
322+
```
323+
282324
### Custom entities
283325

284326
Define a parser that returns an `Entity`, then pass it to `Duckling`:
@@ -343,11 +385,22 @@ const entities = Duckling([Email.parser, Hashtag.parser]).extract(
343385
### `Duckling()`
344386

345387
```ts
346-
function Duckling(): { extract; render; renderMap; redact };
388+
function Duckling(): {
389+
extract;
390+
extractAsync;
391+
render;
392+
renderAsync;
393+
renderMap;
394+
renderMapAsync;
395+
redact;
396+
};
347397
function Duckling<T>(parsers: ParserTuple<T>): {
348398
extract;
399+
extractAsync;
349400
render;
401+
renderAsync;
350402
renderMap;
403+
renderMapAsync;
351404
redact;
352405
};
353406
```
@@ -400,6 +453,49 @@ Built on top of `.render()`. Extracts entities then replaces each matched span
400453
with `opts.mask` (default `"█"`). When `opts.kinds` is set, only those entity
401454
kinds are masked. Overlapping/nested spans are resolved via the span tree.
402455

456+
### `.extractAsync(text, opts?)`
457+
458+
> **Experimental** — this method is stable enough to use but may change in a
459+
> future minor release.
460+
461+
```ts
462+
extractAsync(text: string, opts?: AsyncScanOptions): Promise<Entity[]>
463+
```
464+
465+
Async version of `.extract()` that yields to the event loop periodically.
466+
Supports cancellation via `AbortSignal`.
467+
468+
### `.renderAsync(text, fn, opts?)`
469+
470+
> **Experimental** — this method is stable enough to use but may change in a
471+
> future minor release.
472+
473+
```ts
474+
renderAsync(text: string, fn: RenderFn<Entity>, opts?: AsyncScanOptions): Promise<string>
475+
```
476+
477+
Async version of `.render()`.
478+
479+
### `.renderMapAsync(text, fn, opts?)`
480+
481+
> **Experimental** — this method is stable enough to use but may change in a
482+
> future minor release.
483+
484+
```ts
485+
renderMapAsync<R>(text: string, fn: RenderMapFn<Entity, R>, opts?: AsyncScanOptions): Promise<(string | R)[]>
486+
```
487+
488+
Async version of `.renderMap()`.
489+
490+
### `AsyncScanOptions`
491+
492+
```ts
493+
interface AsyncScanOptions {
494+
signal?: AbortSignal; // cancel the scan
495+
yieldEvery?: number; // default: 512
496+
}
497+
```
498+
403499
### `PIIParsers`
404500

405501
```ts

bench/extract.bench.ts

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,45 @@ Deno.bench("extract: default Duckling (mixed)", () => {
5050
throw new Error("expected at least one time entity");
5151
}
5252
});
53+
54+
// ---------------------------------------------------------------------------
55+
// Async benchmarks
56+
// ---------------------------------------------------------------------------
57+
58+
// Compare sync vs async overhead on PII-heavy text
59+
Deno.bench("extractAsync: PII-heavy (yieldEvery=512)", async () => {
60+
const entities = await piiDuckling.extractAsync(textPII, { yieldEvery: 512 });
61+
if (entities.length < 6) throw new Error("unexpected low match count");
62+
});
63+
64+
Deno.bench("extractAsync: PII-heavy (yieldEvery=64)", async () => {
65+
const entities = await piiDuckling.extractAsync(textPII, { yieldEvery: 64 });
66+
if (entities.length < 6) throw new Error("unexpected low match count");
67+
});
68+
69+
Deno.bench("extractAsync: PII-heavy (yieldEvery=8)", async () => {
70+
const entities = await piiDuckling.extractAsync(textPII, { yieldEvery: 8 });
71+
if (entities.length < 6) throw new Error("unexpected low match count");
72+
});
73+
74+
// Large input: 5x repeated PII text (~1 KB)
75+
const textLarge = (textPII + " ").repeat(5);
76+
77+
Deno.bench("extract: PII-heavy ×5 (sync)", () => {
78+
const entities = piiDuckling.extract(textLarge);
79+
if (entities.length < 30) throw new Error("unexpected low match count");
80+
});
81+
82+
Deno.bench("extractAsync: PII-heavy ×5 (yieldEvery=512)", async () => {
83+
const entities = await piiDuckling.extractAsync(textLarge, {
84+
yieldEvery: 512,
85+
});
86+
if (entities.length < 30) throw new Error("unexpected low match count");
87+
});
88+
89+
Deno.bench("extractAsync: PII-heavy ×5 (yieldEvery=64)", async () => {
90+
const entities = await piiDuckling.extractAsync(textLarge, {
91+
yieldEvery: 64,
92+
});
93+
if (entities.length < 30) throw new Error("unexpected low match count");
94+
});

deno.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "@claudiu-ceia/ts-duckling",
3-
"version": "0.2.1",
3+
"version": "0.3.0",
44
"exports": {
55
".": "./mod.ts"
66
},

mod.ts

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ import {
1414
step,
1515
} from "@claudiu-ceia/combine";
1616
import { __, dot, word } from "./src/common.ts";
17+
import { asyncScan, type AsyncScanOptions } from "./src/async.ts";
1718
import { buildSpanTree, renderMapNode, renderNode } from "./src/render.ts";
1819
import type { RenderFn, RenderMapFn } from "./src/render.ts";
1920
import { Quantity, type QuantityEntity } from "./src/Quantity.ts";
@@ -141,6 +142,7 @@ export interface RedactOptions<K extends string = string> {
141142
kinds?: K[];
142143
}
143144

145+
export type { AsyncScanOptions } from "./src/async.ts";
144146
export type {
145147
RenderEntity,
146148
RenderFn,
@@ -163,9 +165,26 @@ export type {
163165
*/
164166
export function Duckling(): {
165167
extract: (text: string) => AnyEntity[];
168+
/** @experimental */
169+
extractAsync: (
170+
text: string,
171+
opts?: AsyncScanOptions,
172+
) => Promise<AnyEntity[]>;
166173
redact: (text: string, opts?: RedactOptions<AnyEntity["kind"]>) => string;
167174
render: (text: string, fn: RenderFn<AnyEntity>) => string;
175+
/** @experimental */
176+
renderAsync: (
177+
text: string,
178+
fn: RenderFn<AnyEntity>,
179+
opts?: AsyncScanOptions,
180+
) => Promise<string>;
168181
renderMap: <R>(text: string, fn: RenderMapFn<AnyEntity, R>) => (string | R)[];
182+
/** @experimental */
183+
renderMapAsync: <R>(
184+
text: string,
185+
fn: RenderMapFn<AnyEntity, R>,
186+
opts?: AsyncScanOptions,
187+
) => Promise<(string | R)[]>;
169188
};
170189
/**
171190
* Create an extractor with a specific set of parsers. The return type is
@@ -186,17 +205,34 @@ export function Duckling<T extends NonEmptyArray<unknown>>(
186205
parsers: ParserTuple<T>,
187206
): {
188207
extract: (text: string) => T[number][];
208+
/** @experimental */
209+
extractAsync: (
210+
text: string,
211+
opts?: AsyncScanOptions,
212+
) => Promise<T[number][]>;
189213
redact: (
190214
text: string,
191215
opts?: RedactOptions<
192216
T[number] extends { kind: infer K extends string } ? K : string
193217
>,
194218
) => string;
195219
render: (text: string, fn: RenderFn<T[number]>) => string;
220+
/** @experimental */
221+
renderAsync: (
222+
text: string,
223+
fn: RenderFn<T[number]>,
224+
opts?: AsyncScanOptions,
225+
) => Promise<string>;
196226
renderMap: <R>(
197227
text: string,
198228
fn: RenderMapFn<T[number], R>,
199229
) => (string | R)[];
230+
/** @experimental */
231+
renderMapAsync: <R>(
232+
text: string,
233+
fn: RenderMapFn<T[number], R>,
234+
opts?: AsyncScanOptions,
235+
) => Promise<(string | R)[]>;
200236
};
201237

202238
// deno-lint-ignore no-explicit-any
@@ -239,6 +275,34 @@ export function Duckling(parsers?: any): any {
239275

240276
return {
241277
extract: parse,
278+
/**
279+
* @experimental
280+
*
281+
* Async version of {@link extract} that yields to the event loop
282+
* periodically, preventing main-thread blocking on large inputs.
283+
*
284+
* Uses `scheduler.yield()` when available (Chrome 129+, Edge 129+,
285+
* Firefox 142+), falling back to `setTimeout(0)`.
286+
*
287+
* @example
288+
* ```ts
289+
* const controller = new AbortController();
290+
* const entities = await Duckling().extractAsync(longText, {
291+
* signal: controller.signal,
292+
* yieldEvery: 256,
293+
* });
294+
* ```
295+
*/
296+
extractAsync: (
297+
input: string,
298+
opts?: AsyncScanOptions,
299+
): Promise<unknown[]> => {
300+
return asyncScan(
301+
input,
302+
p as NonEmptyArray<Parser<unknown>>,
303+
opts,
304+
);
305+
},
242306
/**
243307
* Replace entity spans using a callback function.
244308
*
@@ -275,6 +339,31 @@ export function Duckling(parsers?: any): any {
275339
const tree = buildSpanTree(all, input.length);
276340
return renderNode(tree, input, fn);
277341
},
342+
/**
343+
* @experimental
344+
*
345+
* Async version of {@link render} that yields to the event loop
346+
* during entity extraction.
347+
*/
348+
renderAsync: async (
349+
input: string,
350+
fn: RenderFn<unknown>,
351+
opts?: AsyncScanOptions,
352+
): Promise<string> => {
353+
const all = (await asyncScan(
354+
input,
355+
p as NonEmptyArray<Parser<unknown>>,
356+
opts,
357+
)) as {
358+
kind: string;
359+
start: number;
360+
end: number;
361+
text: string;
362+
}[];
363+
if (all.length === 0) return input;
364+
const tree = buildSpanTree(all, input.length);
365+
return renderNode(tree, input, fn);
366+
},
278367
/**
279368
* Map entity spans to arbitrary values, returning an array of segments.
280369
*
@@ -309,6 +398,32 @@ export function Duckling(parsers?: any): any {
309398
const tree = buildSpanTree(all, input.length);
310399
return renderMapNode<R>(tree, input, fn);
311400
},
401+
/**
402+
* @experimental
403+
*
404+
* Async version of {@link renderMap} that yields to the event loop
405+
* during entity extraction.
406+
*/
407+
renderMapAsync: async <R>(
408+
input: string,
409+
// deno-lint-ignore no-explicit-any
410+
fn: RenderMapFn<any, R>,
411+
opts?: AsyncScanOptions,
412+
): Promise<(string | R)[]> => {
413+
const all = (await asyncScan(
414+
input,
415+
p as NonEmptyArray<Parser<unknown>>,
416+
opts,
417+
)) as {
418+
kind: string;
419+
start: number;
420+
end: number;
421+
text: string;
422+
}[];
423+
if (all.length === 0) return [input];
424+
const tree = buildSpanTree(all, input.length);
425+
return renderMapNode<R>(tree, input, fn);
426+
},
312427
/**
313428
* Extract entities and replace each matched span with a mask character.
314429
*

0 commit comments

Comments
 (0)