The fastest web crawling, scraping, and browser automation server for AI agents. Gives Claude direct access to the web through 22 tools — crawl sites at 100K+ pages/sec, extract structured data with AI, and control remote browsers with built-in anti-bot bypass.
- Speed — Crawl 100K+ pages per second. Smart request routing picks HTTP or headless Chrome automatically. Streaming responses start delivering data immediately.
- Cost — Pay-per-use credits with no subscription required for core tools. Check your balance anytime with
spider_get_credits. AI tools available with an AI subscription. - Reliability — Anti-bot bypass with fingerprinting and proxy rotation. Browser fleet with automatic fallback across Chrome, Firefox, and more. Built-in retry and stealth escalation.
claude mcp add spider -- npx -y spider-cloud-mcpSet your API key:
export SPIDER_API_KEY="your-api-key"Add to claude_desktop_config.json:
{
"mcpServers": {
"spider": {
"command": "npx",
"args": ["-y", "spider-cloud-mcp"],
"env": {
"SPIDER_API_KEY": "your-api-key"
}
}
}
}Add to .cursor/mcp.json:
{
"mcpServers": {
"spider": {
"command": "npx",
"args": ["-y", "spider-cloud-mcp"],
"env": {
"SPIDER_API_KEY": "your-api-key"
}
}
}
}Add to ~/.codeium/windsurf/mcp_config.json:
{
"mcpServers": {
"spider": {
"command": "npx",
"args": ["-y", "spider-cloud-mcp"],
"env": {
"SPIDER_API_KEY": "your-api-key"
}
}
}
}| Variable | Required | Description |
|---|---|---|
SPIDER_API_KEY |
Yes | Your Spider API key. Get one at spider.cloud/api-keys |
These work on pay-per-use credits with no subscription required.
| Tool | Description |
|---|---|
spider_crawl |
Crawl a website and extract content from multiple pages. Follows links up to a depth/limit. |
spider_scrape |
Scrape a single page. Faster and cheaper than crawling when you need one URL. |
spider_search |
Search the web. Optionally fetch full page content from results. |
spider_links |
Extract all links from a page without fetching content. |
spider_screenshot |
Capture a page screenshot as base64 PNG. |
spider_unblocker |
Access bot-protected content with advanced anti-bot bypass. |
spider_transform |
Convert HTML to markdown or text without making web requests. |
spider_get_credits |
Check your API credit balance. |
Natural language web interaction. Describe what you want in plain English.
Requires an AI subscription.
| Tool | Description |
|---|---|
spider_ai_crawl |
AI-guided crawling — describe what content to find. |
spider_ai_scrape |
Extract structured data with plain English — no CSS selectors. |
spider_ai_search |
AI-enhanced search with intent understanding. |
spider_ai_browser |
Automate browser actions with natural language. |
spider_ai_links |
Find and filter links by description. |
Direct browser control via spider-browser. Browsers run in Spider's cloud with smart retry, browser switching, anti-bot protection, proxy rotation, and automatic stealth escalation. Supports CDP and BiDi protocols.
| Tool | Description |
|---|---|
spider_browser_open |
Open a remote browser session. Returns a session_id. |
spider_browser_navigate |
Navigate to a URL and wait for load. |
spider_browser_click |
Click an element by CSS selector. |
spider_browser_fill |
Fill a form field with text. |
spider_browser_screenshot |
Take a screenshot of the current page. Returns base64 PNG. |
spider_browser_content |
Get page HTML or visible text. |
spider_browser_evaluate |
Execute JavaScript in the page context. |
spider_browser_wait_for |
Wait for an element, navigation, or network idle. |
spider_browser_close |
Close the session and stop billing. |
Browser sessions auto-close after 5 minutes of inactivity. Always call spider_browser_close when done.
"Crawl the React documentation and summarize the hooks API"
Uses spider_crawl to fetch 50+ pages in seconds and return clean markdown ready for context.
spider_crawl: {
url: "https://react.dev/reference/react",
limit: 50,
return_format: "markdown",
filter_output_main_only: true
}
"Get all product names and prices from this e-commerce page"
Uses spider_ai_scrape to extract structured JSON with zero CSS selectors.
spider_ai_scrape: {
url: "https://example-store.com/products",
prompt: "Extract every product name, price, and availability status as JSON"
}
"Log into the dashboard, go to reports, and screenshot the monthly summary"
Uses spider_browser_* tools to drive a remote browser with full anti-bot protection.
1. spider_browser_open: { browser: "auto" }
2. spider_browser_navigate: { url: "https://app.example.com/login" }
3. spider_browser_fill: { selector: "input[name='email']", value: "user@example.com" }
4. spider_browser_fill: { selector: "input[name='password']", value: "..." }
5. spider_browser_click: { selector: "button[type='submit']" }
6. spider_browser_wait_for: { navigation: true }
7. spider_browser_navigate: { url: "https://app.example.com/reports/monthly" }
8. spider_browser_screenshot: {}
9. spider_browser_close: {}
"Search for recent AI startup funding rounds and get the details"
Uses spider_search with time filtering, then spider_scrape for details.
spider_search: {
search: "AI startup Series A funding 2025",
num: 10,
fetch_page_content: true,
return_format: "markdown",
tbs: "qdr:m"
}
All tools map directly to the Spider API. Core tools accept the same parameters as their API counterparts:
- Crawl/Scrape:
url,return_format,request,readability,root_selector,proxy_enabled,cache, and many more - Search:
search,num,fetch_page_content,country,language,tbs - AI tools: Add a
promptparameter describing what you want in natural language - Browser tools: Use
session_idfromspider_browser_openfor all operations
Full parameter reference: spider.cloud/docs/api
MIT