Skip to content

feat: Add mobile proxy provider support for self-hosted deployments #2881

@bolivian-peru

Description

@bolivian-peru

Problem

Firecrawl's self-hosted deployments need proxy support for reliable scraping, but:

  • No built-in proxy provider configuration for self-hosted users
  • PROXY_SERVER and PROXY_USERNAME/PROXY_PASSWORD env vars exist but lack provider examples
  • Mobile carrier proxies (real 4G/5G IPs) provide the highest success rate but are undocumented
  • Self-hosted users must figure out proxy integration on their own

Proposed Solution

1. Simple env configuration (works today):

# .env for self-hosted Firecrawl
PROXY_SERVER=http://gw.proxies.sx:7000
PROXY_USERNAME=psx_USER-mbl-us
PROXY_PASSWORD=yourpassword

2. Country-targeted proxies:

# US mobile proxy
PROXY_USERNAME=psx_USER-mbl-us

# Germany mobile proxy
PROXY_USERNAME=psx_USER-mbl-de

# UK mobile proxy
PROXY_USERNAME=psx_USER-mbl-gb

3. Proxy rotation via username tokens:

# Session sticky (same IP per session)
PROXY_USERNAME=psx_USER-mbl-us-sid-mysession

# Auto-rotate every request
PROXY_USERNAME=psx_USER-mbl-us-rot-1

Why Mobile Proxies for Firecrawl

  • Real 4G/5G carrier tower IPs — highest trust score of any proxy type
  • Sites that block residential proxies cannot block mobile carrier IPs
  • 155+ real modems across US, DE, GB, FR, ES, PL
  • $4/GB — comparable to premium residential
  • Standard HTTP proxy auth — works with existing PROXY_SERVER env vars

Additional Context

  • This is primarily a documentation contribution for self-hosted users
  • Hosted Firecrawl manages its own proxy infrastructure (this is for self-hosted only)
  • Proxies.sx — mobile proxy infrastructure for AI agents
  • Gateway: gw.proxies.sx:7000 (HTTP) / gw.proxies.sx:7001 (SOCKS5)
  • x402 payment support: Pay-per-request with USDC micropayments

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions