|
| 1 | +# CoreClaw Documentation |
| 2 | + |
| 3 | +Source content and configuration for [docs.coreclaw.com](https://docs.coreclaw.com/), built with [Astro](https://astro.build/) and [Starlight](https://starlight.astro.build/). |
| 4 | + |
| 5 | +Chinese version: [`README.zh-CN.md`](./README.zh-CN.md) |
| 6 | + |
| 7 | +## Documentation Architecture |
| 8 | + |
| 9 | +CoreClaw documentation is organized by audience: |
| 10 | + |
| 11 | +### Content Areas |
| 12 | + |
| 13 | +| Area | Audience | Content | |
| 14 | +|------|----------|---------| |
| 15 | +| User Guide | End users | Running Workers, viewing results, billing, API usage, FAQ | |
| 16 | +| Developer Guide | Worker developers | Building, testing, publishing, and monetizing Workers | |
| 17 | +| API Reference | API consumers | Full endpoint docs with request/response schemas | |
| 18 | +| Website Events | All users | Platform events and promotions | |
| 19 | +| Changelog | All users | Platform and documentation updates | |
| 20 | + |
| 21 | +### Multilingual Structure |
| 22 | + |
| 23 | +- **English** (`/`) — Default language, authoritative source |
| 24 | +- **Simplified Chinese** (`/zh-cn/`) — Full translation, maintained in parallel |
| 25 | + |
| 26 | +The directory structures under `src/content/docs/` and `src/content/docs/zh-cn/` mirror each other. Changes should be kept in sync across both languages. |
| 27 | + |
| 28 | +## Key Conventions |
| 29 | + |
| 30 | +### Worker / Scraper Terminology |
| 31 | + |
| 32 | +CoreClaw uses two terms for the same concept: |
| 33 | + |
| 34 | +- **Worker** — The name used in documentation and the UI; refers to data collection scripts |
| 35 | +- **Scraper** — The name used in API paths and field names for backward compatibility (e.g., `scraper_slug`, `/api/v1/scraper/run`) |
| 36 | + |
| 37 | +### API Documentation |
| 38 | + |
| 39 | +API reference is organized by endpoint groups: |
| 40 | + |
| 41 | +- **Worker** — `/api/v1/scraper/*` endpoints for launching, aborting, and viewing Worker runs |
| 42 | +- **Runs** — `/api/v1/run/*` endpoints for history, results, logs, and exports |
| 43 | +- **Tasks** — `/api/v1/task/*` endpoints for saved Task templates |
| 44 | +- **Account** — `/api/v1/account/*` endpoints for account information |
| 45 | + |
| 46 | +Each endpoint page documents the HTTP method, path, request parameters, response schema, and error codes. The [API index page](https://docs.coreclaw.com/api/) provides a complete endpoint quick-reference. |
| 47 | + |
| 48 | +The OpenAPI spec is at `public/openapi.json`, served at `/openapi.json`. The `openapi.swagger.json` file is for local reference only and is not version-controlled (see `.gitignore`). |
| 49 | + |
| 50 | +### Sidebar Configuration |
| 51 | + |
| 52 | +Navigation is defined manually via explicit `items` arrays in `astro.config.mjs`, giving full control over: |
| 53 | +- Section labels and ordering |
| 54 | +- Multilingual label translations |
| 55 | +- Collapsible groups and nesting |
| 56 | +- Badge annotations (e.g., "Required") |
| 57 | + |
| 58 | +## Repository Structure |
| 59 | + |
| 60 | +``` |
| 61 | +. |
| 62 | +├── public/ # Static assets (copied as-is) |
| 63 | +│ ├── openapi.json # OpenAPI spec (served at /openapi.json) |
| 64 | +│ ├── favicon.jpg # Site favicon |
| 65 | +│ └── logo.png # Site logo |
| 66 | +├── src/ |
| 67 | +│ ├── assets/ # Build-time assets (images, logos) |
| 68 | +│ │ └── docs/ # Documentation screenshots |
| 69 | +│ ├── components/ # Custom Astro components |
| 70 | +│ │ ├── ApiPlayground.astro # Interactive API testing form |
| 71 | +│ │ ├── CopyForLLMs.astro # Copy-for-LLMs header dropdown |
| 72 | +│ │ ├── Banner.astro # Site banner |
| 73 | +│ │ ├── Header.astro # Custom page header |
| 74 | +│ │ ├── Footer.astro # Custom page footer |
| 75 | +│ │ └── ... # Other override components |
| 76 | +│ ├── content/docs/ # English docs (default language) |
| 77 | +│ │ ├── api/ # API reference |
| 78 | +│ │ ├── developer-guide/ # Developer docs |
| 79 | +│ │ ├── user-guide/ # User docs |
| 80 | +│ │ ├── website-events/ # Events and promotions |
| 81 | +│ │ ├── home.mdx # Homepage |
| 82 | +│ │ ├── changelog.mdx # Changelog |
| 83 | +│ │ └── zh-cn/ # Simplified Chinese mirror |
| 84 | +│ ├── pages/ # Dynamic routes (Copy-for-LLMs .md exports) |
| 85 | +│ └── styles/common.css # Global style overrides |
| 86 | +├── scripts/ |
| 87 | +│ └── check-copy-for-llms.mjs # Post-build smoke test (.md export completeness) |
| 88 | +├── astro.config.mjs # Astro + Starlight configuration |
| 89 | +├── package.json |
| 90 | +└── tsconfig.json |
| 91 | +``` |
| 92 | + |
| 93 | +## Tech Stack |
| 94 | + |
| 95 | +| Component | Purpose | |
| 96 | +|-----------|---------| |
| 97 | +| [Astro](https://astro.build/) | Static site generation | |
| 98 | +| [Starlight](https://starlight.astro.build/) | Documentation theme (sidebar, search, i18n) | |
| 99 | +| [React](https://react.dev/) | Interactive UI components | |
| 100 | +| `starlight-image-zoom` | Image zoom plugin | |
| 101 | +| `sharp` | Image optimization | |
| 102 | +| `turndown` | HTML to Markdown (Copy-for-LLMs) | |
| 103 | + |
| 104 | +## Local Development |
| 105 | + |
| 106 | +```bash |
| 107 | +pnpm install # Install dependencies (Node.js 22+, pnpm 10+) |
| 108 | +pnpm run dev # Start dev server at http://localhost:4321 |
| 109 | +pnpm run build # Build to dist/ (includes Copy-for-LLMs check) |
| 110 | +pnpm run preview # Preview production build |
| 111 | +``` |
| 112 | + |
| 113 | +## Quality Checks |
| 114 | + |
| 115 | +Run `pnpm run build` before committing to verify: |
| 116 | + |
| 117 | +- No content rendering errors |
| 118 | +- English and Chinese sidebar ordering match |
| 119 | +- English and Chinese page structures align |
| 120 | +- No broken links or missing asset references |
| 121 | +- Copy-for-LLMs smoke test passes (every doc page exports non-empty Markdown) |
| 122 | + |
| 123 | +## References |
| 124 | + |
| 125 | +- [Astro Docs](https://docs.astro.build/) |
| 126 | +- [Starlight Docs](https://starlight.astro.build/) |
| 127 | +- [CoreClaw Docs (live)](https://docs.coreclaw.com/) |
0 commit comments