|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Project Overview |
| 6 | + |
| 7 | +ProxyBroker2 is an async proxy finder, checker, and server that discovers and validates public proxies from multiple sources. It supports HTTP(S), SOCKS4/5 protocols and can operate as a proxy server with automatic rotation. |
| 8 | + |
| 9 | +## Development Commands |
| 10 | + |
| 11 | +### Dependencies |
| 12 | +Uses Poetry for dependency management: |
| 13 | +```bash |
| 14 | +poetry install # Install dependencies |
| 15 | +poetry shell # Activate virtual environment |
| 16 | +``` |
| 17 | + |
| 18 | +### Testing |
| 19 | +```bash |
| 20 | +pytest # Run all tests |
| 21 | +pytest tests/test_proxy.py # Run specific test file |
| 22 | +pytest -v # Verbose output |
| 23 | +pytest --flake8 # Run with linting |
| 24 | +pytest --isort # Run with import sorting |
| 25 | +pytest --cov # Run with coverage reporting |
| 26 | +``` |
| 27 | + |
| 28 | +### Linting and Code Quality |
| 29 | +```bash |
| 30 | +flake8 # Check code style |
| 31 | +isort . # Sort imports |
| 32 | +``` |
| 33 | + |
| 34 | +### Building |
| 35 | +```bash |
| 36 | +pip install pyinstaller && pip install . && mkdir -p build && cd build && pyinstaller --onefile --name proxybroker --add-data "../proxybroker/data:data" --workpath ./tmp --distpath . --clean ../py2exe_entrypoint.py && rm -rf tmp *.spec |
| 37 | +``` |
| 38 | + |
| 39 | +### Docker |
| 40 | +```bash |
| 41 | +docker build -t proxybroker2 . |
| 42 | +docker run --rm proxybroker2 --help |
| 43 | +``` |
| 44 | + |
| 45 | +## Architecture Overview |
| 46 | + |
| 47 | +### Core Components |
| 48 | + |
| 49 | +**Broker** (`api.py`): Central orchestrator that manages the entire proxy discovery and checking pipeline. Coordinates providers, checkers, and servers. |
| 50 | + |
| 51 | +**ProxyPool** (`server.py`): Manages proxy selection strategies and health tracking. Maintains separate pools for newcomers and established proxies with error rate monitoring. |
| 52 | + |
| 53 | +**Server** (`server.py`): HTTP/HTTPS proxy server that distributes incoming requests across the proxy pool with automatic rotation and failure handling. |
| 54 | + |
| 55 | +**Checker** (`checker.py`): Validates proxy functionality by testing connectivity, anonymity levels, and protocol support through configurable judges. |
| 56 | + |
| 57 | +**Provider** (`providers.py`): Web scrapers that extract proxy lists from various public sources (~50 different websites). |
| 58 | + |
| 59 | +**Negotiators** (`negotiators.py`): Protocol-specific handlers for HTTP CONNECT, SOCKS4, and SOCKS5 proxy connections. |
| 60 | + |
| 61 | +### Key Data Flow |
| 62 | + |
| 63 | +1. **Discovery**: Providers scrape proxy sources concurrently (max 3 at once, configurable via MAX_CONCURRENT_PROVIDERS) |
| 64 | +2. **Validation**: Checker tests each proxy against judge servers for connectivity and anonymity |
| 65 | +3. **Pooling**: Valid proxies enter ProxyPool with health tracking and rotation strategies |
| 66 | +4. **Serving**: Server distributes client requests across healthy proxies with automatic failover |
| 67 | + |
| 68 | +### Important Implementation Details |
| 69 | + |
| 70 | +**Async Architecture**: Entire codebase is built on asyncio with careful resource management for connections and timeouts. |
| 71 | + |
| 72 | +**Error Handling**: Comprehensive exception hierarchy in `errors.py` for different failure modes (connection, timeout, protocol-specific errors). |
| 73 | + |
| 74 | +**Geolocation**: Uses MaxMind GeoLite2 database (`data/GeoLite2-Country.mmdb`) for country-based proxy filtering. |
| 75 | + |
| 76 | +**Configuration**: CLI interface (`cli.py`) with extensive options for timeouts, concurrency limits, filtering criteria, and output formats. |
| 77 | + |
| 78 | +### Project Structure Quirks |
| 79 | + |
| 80 | +- `py2exe_entrypoint.py`: Special entry point for PyInstaller builds |
| 81 | +- Two package managers: Poetry (preferred) and setuptools (legacy) |
| 82 | +- GeoIP database included in package data (`proxybroker/data/GeoLite2-Country.mmdb`) |
| 83 | +- Docker support with multi-stage builds |
| 84 | +- Python 3.8+ support (current version targets 3.8-3.10) |
| 85 | +- CLI entry point defined via Poetry scripts: `proxybroker = "proxybroker.cli:cli"` |
| 86 | + |
| 87 | +### Python Version Support |
| 88 | + |
| 89 | +The project currently supports Python 3.8-3.10. Python 3.11+ support is in progress but not yet complete. Development dependencies include pytest plugins for async testing, linting (flake8), import sorting (isort), and code coverage. |
0 commit comments