An MCP server for web search using DuckDuckGo and content extraction, with support for multiple URLs and memory optimizations.
- 🔍 DuckDuckGo Search: Fast and privacy-focused web search capability
- 📄 Content Extraction: Clean, readable text extraction from web pages
- 🚀 Parallel Processing: Support for extracting content from multiple URLs simultaneously
- 💾 Memory Optimization: Smart memory management to prevent application crashes
- ⏱️ Rate Limiting: Intelligent request throttling to avoid API blocks
- 🛡️ Error Handling: Robust error handling for reliable operation
To install Web Scout for Claude Desktop automatically via Smithery:
npx -y @smithery/cli install @pinkpixel-dev/web-scout-mcp --client claude
npm install -g @pinkpixel/web-scout-mcp
npm install @pinkpixel/web-scout-mcp
After installing globally, run:
web-scout-mcp
Add this to your MCP client's config.json
(Claude Desktop, Cursor, etc.):
{
"mcpServers": {
"web-scout": {
"command": "npx",
"args": [
"-y",
"@pinkpixel/web-scout-mcp"
]
}
}
}
The server provides the following MCP tools:
Initiates a web search query using the DuckDuckGo search engine and returns a well-structured list of findings.
Input:
query
(string): The search query stringmaxResults
(number, optional): Maximum number of results to return (default: 10)
Example:
{
"query": "latest advancements in AI",
"maxResults": 5
}
Output: A formatted list of search results with titles, URLs, and snippets.
Fetches and extracts clean, readable content from web pages by removing unnecessary elements like scripts, styles, and navigation.
Input:
url
: Either a single URL string or an array of URL strings
Example (single URL):
{
"url": "https://example.com/article"
}
Example (multiple URLs):
{
"url": [
"https://example.com/article1",
"https://example.com/article2"
]
}
Output: Extracted text content from the specified URL(s).
# Clone the repository
git clone https://github.com/pinkpixel-dev/web-scout-mcp.git
cd web-scout-mcp
# Install dependencies
npm install
# Build
npm run build
# Run
npm start
For more detailed information about the project, check out these resources:
- OVERVIEW.md - Technical overview and architecture
- CONTRIBUTING.md - Guidelines for contributors
- CHANGELOG.md - Version history and changes
- Node.js >= 18.0.0
- npm or yarn
This project is licensed under the MIT License.
Made with ❤️ by Pink Pixel
✨ Dream it, Pixel it ✨