batch with output file

[mixxr]

when processing URLs in batch mode, it could be useful to save each single output in a dedicated file. Eg.
URL1, filename1
URL2, filename2
...

[0xMassi]

shipped in v0.1.5 — thanks for the suggestion.

--output-dir

saves each page to its own file. works with single URL, batch, and crawl.

# crawl a docs site, one file per page
webclaw https://docs.example.com --crawl --depth 2 --output-dir ./docs

# batch multiple URLs
webclaw https://example.com https://other.com --output-dir ./output

filenames are derived from URL paths (/docs/api → docs/api.md). root URLs use hostname/index.md to avoid collisions.

CSV with custom filenames

you can also specify exact filenames using --urls-file:

https://example.com/page1,page1.md
https://example.com/page2,page2.md

webclaw --urls-file urls.csv --output-dir ./output

release: https://github.com/0xMassi/webclaw/releases/tag/v0.1.5