|
5 | 5 | <h1 align="center">WebSkrap</h1> |
6 | 6 |
|
7 | 7 | <p align="center"> |
8 | | - <strong>Async-first Python scraping framework built on Playwright — and a first-class web tool for LLMs and agents.</strong><br> |
9 | | - <em>Coherent browser profiles, persistent sessions, resource routing, and Patchright-powered stealth for data collection workflows that need realistic browser behavior. Ships an MCP server so Claude, Codex, and any MCP agent can fetch live pages as clean, token-efficient text.</em> |
| 8 | + <strong>Async-first Python scraping framework built on Playwright that also works as a web tool for LLMs and agents.</strong><br> |
| 9 | + <em>Coherent browser profiles, persistent sessions, resource routing, and Patchright-powered stealth for data collection that needs realistic browser behavior. Ships an MCP server so Claude, Codex, and any MCP agent can fetch live pages as plain text instead of raw HTML.</em> |
10 | 10 | </p> |
11 | 11 |
|
12 | 12 | WebSkrap does not include CAPTCHA solving, login-wall bypassing, credential bypassing, or access-control circumvention. Use it only on targets you are allowed to access. |
@@ -403,11 +403,11 @@ tools over stdio: `fetch`, `stealth_fetch`, and `doctor`. |
403 | 403 | Built for LLMs: both `fetch` and `stealth_fetch` run the same CDP-leak-free |
404 | 404 | Patchright stealth path the CLI uses (headless Chrome, `networkidle` wait), so |
405 | 405 | JS-heavy and anti-bot pages that block naive scrapers still load. They return |
406 | | -**clean visible page text by default** — no HTML tags, scripts, or style noise — |
| 406 | +clean visible page text by default, with no HTML tags, scripts, or style noise, |
407 | 407 | so agents spend tokens on content, not markup (typically 5-10x fewer tokens than |
408 | 408 | raw HTML). Pass `text_only=False` when you actually need the HTML. Use |
409 | | -`stealth_fetch` for finer control (fingerprint surface, WebRTC, UA masking, |
410 | | -persistent profile). Every result carries `status`, `final_url`, `title`, |
| 409 | +`stealth_fetch` for finer control over fingerprint surface, WebRTC, UA masking, |
| 410 | +and persistent profiles. Every result carries `status`, `final_url`, `title`, |
411 | 411 | `text_length`, and truncation flags so the model knows exactly what it got. |
412 | 412 |
|
413 | 413 | ```bash |
|
0 commit comments