Repositories list

• spider

  Public
  Low latency web data collector

  rustcrawlerautomation+ 6

  rustcrawlerautomationspiderweb-crawlerscrapingweb-scrapingheadless-chromeai-agent

  Rust

  •

  MIT License

  •206206 forks•2.5k2.5k stars•00 issues•00 pull requests•Updated May 29, 2026May 29, 2026

• llm_models

  Public
  Auto-updated registry of LLM model capabilities (vision, audio, etc.)

  Rust

  •

  MIT License

  •00 forks•22 stars•00 issues•00 pull requests•Updated May 29, 2026May 29, 2026

• gottem

  Public
  Router for scraping vendors unified

  scraperrouterscraping+ 1

  scraperrouterscrapingweb-scraper

  Rust

  •

  Apache License 2.0

  •22 forks•33 stars•00 issues•00 pull requests•Updated May 27, 2026May 27, 2026

• ua_generator

  Public
  Pre-compiled random real User-Agents. Updated weekly for windows, mac, linux, and android

  useragentuser-agent-stringua-generator

  useragentuser-agent-stringua-generator

  Rust

  •

  MIT License

  •99 forks•3939 stars•00 issues•00 pull requests•Updated May 24, 2026May 24, 2026

• spider-cloud-mcp-server

  Public
  MCP server for the Spider Cloud web crawling and scraping API

  spiderweb-crawlerscaper+ 4

  spiderweb-crawlerscapermcp-servermcp-scrapermcp-crawlermcp-spider

  JavaScript

  •

  MIT License

  •00 forks•22 stars•00 issues•00 pull requests•Updated May 22, 2026May 22, 2026

• spider-cloud-mcp-v2

  Public
  Spider MCP server v2: crawl, scrape, search, and automate browsers at scale

  TypeScript

  •

  MIT License

  •00 forks•22 stars•11 issue•00 pull requests•Updated May 22, 2026May 22, 2026

• http-cache

  Public
  A caching middleware that follows HTTP caching rules

  Rust

  •

  Apache License 2.0

  •2828 forks•00 stars•00 issues•00 pull requests•Updated May 14, 2026May 14, 2026

• firewall

  Public
  Firewall for Rust

  rustblacklistnetworking+ 3

  rustblacklistnetworkingfirewalladblockadguard

  Rust

  •

  MIT License

  •00 forks•11 star•00 issues•00 pull requests•Updated May 14, 2026May 14, 2026

• chromey

  Public
  Chrome DevTools Protocol for Rust

  automationcdpheadless-chrome+ 1

  automationcdpheadless-chromechrome-devtools-protocol

  Rust

  •

  Apache License 2.0

  •1111 forks•4545 stars•00 issues•00 pull requests•Updated May 7, 2026May 7, 2026

• znz

  Public
  embedded web tech

  Rust

  •

  Mozilla Public License 2.0

  •3.6k3.6k forks•00 stars•00 issues•00 pull requests•Updated May 3, 2026May 3, 2026

• spider_network_blocker

  Public
  Network blocking lists on ads, scripts, and analytics for Spider

  networking

  networking

  Rust

  •

  MIT License

  •00 forks•11 star•00 issues•00 pull requests•Updated May 3, 2026May 3, 2026

• html2md

  Public
  HTML to markdown for Rust

  htmlmarkdownrust+ 1

  htmlmarkdownrusthtml2md

  HTML

  •

  MIT License

  •33 forks•7272 stars•00 issues•00 pull requests•Updated Apr 30, 2026Apr 30, 2026

• markup5ever_rcdom

  Public
  Send/Sync-friendly fork of markup5ever_rcdom on the spider-* atomic-tendril stack

  Rust

  •

  Apache License 2.0

  •00 forks•00 stars•00 issues•00 pull requests•Updated Apr 30, 2026Apr 30, 2026

• spider_scraper

  Public
  html5ever web scraper used for some of spider-rs CSS transformations

  Rust

  •

  MIT License

  •00 forks•11 star•00 issues•00 pull requests•Updated Apr 30, 2026Apr 30, 2026

• spider_transformations

  Public
  HTML transformation library for Rust

  html-to-markdowntransform-datahtml-transformation

  html-to-markdowntransform-datahtml-transformation

  Rust

  •

  MIT License

  •11 fork•22 stars•00 issues•00 pull requests•Updated Apr 30, 2026Apr 30, 2026

• readability

  Public
  The readability library for Rust

  rustreadabilitydata-cleaning+ 3

  rustreadabilitydata-cleaningclean-datasafari-readerllm-training

  Rust

  •

  MIT License

  •22 forks•1818 stars•00 issues•00 pull requests•Updated Apr 30, 2026Apr 30, 2026

• html5ever

  Public
  Send-able fork of html5ever, depends on spider-markup5ever and spider-tendril

  Rust

  •

  Apache License 2.0

  •00 forks•00 stars•00 issues•00 pull requests•Updated Apr 30, 2026Apr 30, 2026

• markup5ever

  Public
  Send-able fork of markup5ever, depends on spider-tendril

  Rust

  •

  Apache License 2.0

  •00 forks•00 stars•00 issues•00 pull requests•Updated Apr 30, 2026Apr 30, 2026

• tendril

  Public
  Send-able Tendril fork (atomic refcount) for high-concurrency HTML parsing

  Rust

  •

  Apache License 2.0

  •00 forks•00 stars•00 issues•00 pull requests•Updated Apr 30, 2026Apr 30, 2026

• auto-encoder

  Public
  Auto encoding library bytes to strings

  encodingstring-encodingauto-encoder

  encodingstring-encodingauto-encoder

  Rust

  •

  MIT License

  •11 fork•11 star•00 issues•00 pull requests•Updated Apr 29, 2026Apr 29, 2026

• spider_remote_cache

  Public
  Rust

  •00 forks•00 stars•00 issues•00 pull requests•Updated Apr 24, 2026Apr 24, 2026

• spider-browser

  Public
  Remote AI Browser

  browserpuppeteerplaywright

  browserpuppeteerplaywright

  TypeScript

  •

  MIT License

  •11 fork•11 star•00 issues•00 pull requests•Updated Apr 22, 2026Apr 22, 2026

• hybrid_cache_server

  Public
  High performance cache index server

  cachecache-serverremote-caching

  cachecache-serverremote-caching

  Rust

  •

  MIT License

  •00 forks•00 stars•00 issues•00 pull requests•Updated Apr 15, 2026Apr 15, 2026

• n8n-nodes-spider

  Public
  Spider n8n community node — crawl, scrape, and extract structured data from any website inside your n8n workflows.

  automationaispider+ 7

  automationaispiderweb-scrapingdata-extractionweb-crawlingworkflow-automationn8nn8n-noden8n-community-node

  TypeScript

  •00 forks•00 stars•00 issues•00 pull requests•Updated Apr 12, 2026Apr 12, 2026

• case_insensitive_string

  Public
  A case-insensitive string for Rust

  ruststringcase-insensitive+ 1

  ruststringcase-insensitivecase-insensitive-strings

  Rust

  •

  MIT License

  •11 fork•44 stars•00 issues•00 pull requests•Updated Apr 8, 2026Apr 8, 2026

• spider_fingerprint

  Public
  The fingerprinting library for Rust

  emulationheadlessfingerprint

  emulationheadlessfingerprint

  Rust

  •

  MIT License

  •44 forks•1515 stars•00 issues•00 pull requests•Updated Mar 30, 2026Mar 30, 2026

• benchmark

  Public
  Python

  •1111 forks•11 star•00 issues•00 pull requests•Updated Mar 22, 2026Mar 22, 2026

• spider-clients

  Public
  Python, Javascript, and Rust libraries for the Spider Cloud API.

  crawlerscraperai+ 7

  crawlerscraperaispiderhtml-to-markdownweb-scrapingai-agentssupabaseai-scrapingllm-webcrawler

  Rust

  •

  MIT License

  •99 forks•2626 stars•00 issues•00 pull requests•Updated Mar 20, 2026Mar 20, 2026

• http-global-cache

  Public
  A global cache to use for http.

  Rust

  •00 forks•00 stars•00 issues•00 pull requests•Updated Mar 6, 2026Mar 6, 2026

• spider-google-sheets

  Public
  Google Sheets add-on for web scraping, AI extraction, and search powered by the Spider API

  spiderseospreadsheet+ 5

  spiderseospreadsheetgoogle-sheetsweb-scrapingapps-scriptgoogle-sheets-addonai-extraction

  TypeScript

  •11 fork•00 stars•00 issues•00 pull requests•Updated Mar 5, 2026Mar 5, 2026