Skip to content

Commit ba81ff8

Browse files
committed
docs: link upstream projects in README and add PDF.js roadmap item
1 parent 67d6264 commit ba81ff8

2 files changed

Lines changed: 6 additions & 5 deletions

File tree

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,9 +54,9 @@ flowchart LR
5454

5555
## What Ships Today
5656

57-
- HTTP adapters for Open WebUI external web loader requests and limited Jina Reader-style `GET /r` requests.
58-
- A default `full` pipeline with source loading, Readability, mdream HTML-to-markdown conversion, optional LLM clean/summarize passes, URL verification, truncation, and optional XML diagnostics.
59-
- Source providers for native HTTP fetch, Firecrawl, and Docling.
57+
- HTTP adapters for [Open WebUI](https://github.com/open-webui/open-webui) external web loader requests and limited [Jina Reader](https://github.com/jina-ai/reader)-style `GET /r` requests.
58+
- A default `full` pipeline with source loading, [Readability](https://github.com/mozilla/readability), [mdream](https://github.com/harlan-zw/mdream) HTML-to-markdown conversion, optional LLM clean/summarize passes, URL verification, truncation, and optional XML diagnostics.
59+
- Source providers for native HTTP fetch, [Firecrawl](https://github.com/firecrawl/firecrawl), and [Docling](https://github.com/docling-project/docling).
6060
- Docker and GHCR image support.
6161

6262
The `smoke` pipeline is also shipped for testing and debugging. It uses only native HTTP fetch, aggressive mdream conversion, and truncation.

docs/ROADMAP.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,15 +24,16 @@ Current provider implementations are intentionally few. The interfaces exist so
2424
## Coming soon
2525

2626
1. **Playwright source provider** to remove the hard dependency on a running Firecrawl instance for HTML pages.
27-
2. **LLM pass auto-repair.** Repair URLs and code blocks, instead of mere detection.
27+
2. **Support PDF.js** to extract text from PDF documents without an external service.
2828
3. **Diagnostics footer cleanup.** Consolidate the `<loader_info ... />` payload for consistency and readability.
2929
4. **Better defaults and prompts.** Test and tune what ships.
3030

3131
## Short term
3232

3333
Near-term additions once the work above settles.
3434

35-
- **Docling content transformer** to convert PDFs, Office documents, and other document-type files to markdown.
35+
- **LLM pass auto-repair.** Repair URLs and code blocks, instead of mere detection.
36+
- **Jina Reader source provider** to get markdown from a self-hosted Jina Reader instance.
3637
- **Crawl4AI source provider** as a potentially better alternative to Firecrawl.
3738

3839
## Mid term

0 commit comments

Comments
 (0)