Documentation overhaul + examples

[KingAkeem]

Summary

Split docs into a small docs site with quickstarts, configuration reference, API reference, and an ops guide (metrics/pprof).

Motivation

• Faster onboarding
• Clear runbooks for operating at scale

Scope

• /docs with mkdocs or mdBook
• Quickstart: local & Docker/Tor
• Example: crawl a tiny local site; Prometheus+Grafana sample dashboard
• Architecture diagrams

Acceptance Criteria

• A new user can run a crawl and see metrics in <10 minutes
• README becomes a short entry point linking to docs

Tasks

• Create docs framework and structure
• Add examples and diagrams
• Update README to point to docs

[Ayush3603]

Assign it to me if you don't mind

[PSNAppz]

@Ayush3603 Feel free to open a PR.