Skip to content

CLAUDE.md

Latest commit

 

History

History
171 lines (134 loc) · 9.38 KB

CLAUDE.md

File metadata and controls

171 lines (134 loc) · 9.38 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Development Commands

Testing

  • API Testing: ./scripts/test-api.sh - Tests all API endpoints with optional authentication
  • Certificate Request Testing: ./scripts/test-certificate-request.sh - Tests certificate generation endpoints
  • Stick-table contract: python3 scripts/test-stick-table-contract.py - offline; holds the templates' store clauses, STICK_TABLE_FIELD_CONTRACT, and every consumer to each other. Run it after touching any stick-table line.
  • Runtime-map contract: python3 scripts/test-runtime-map-contract.py - offline; asserts the runtime map commands are @1-prefixed, reference the map by FILE PATH (never #<id>), carry the value 1, and that every captured rejection is classified as a failure. Run it after touching any add map/del map/clear map path.
  • Certificate destruction safety: python3 scripts/test-cert-write-safety.py - offline; asserts a live .pem is never truncated, removed, or its certbot lineage deleted while any configured domain still references it (one bundle serves many names, so ssl_cert_path is routinely shared). Run it after touching any os.remove/certbot delete/PEM-write path.
  • Manual Testing: Run curl commands against http://localhost:8000 endpoints as shown in README.md

Reading stick tables (and why it is easy to get silently wrong)

/tmp/haproxy-cli is HAProxy's master CLI socket. Worker commands (show table, show map, add map, ...) need an @1 prefix. Without it HAProxy answers Unknown command: 'show', ... and socat still exits 0 — so an exit-status check passes and the help text gets parsed as data. Always use haproxy_cli(cmd, worker=True) in Python, which inspects the response body.

Stick-table entries are name=value / name(window_ms)=value pairs, not fixed columns; the first token is an allocation pointer (0x...:), not the key. Parse by NAME, and treat a missing field as an ERROR — never default it to 0. The web table stores only conn_cur, conn_rate, http_req_rate, http_err_rate; it holds no history and no counter of past blocks. What was actually denied/tarpitted is in the edge access log on the host at /var/log/haproxy.log (shipped 2026.08.8), not in any stick table.

This is written down because /api/security/stats and show-tarpit-ips.sh reported "Scan Count"/"BLOCKED" figures parsed from gpc0/gpc1 — fields no stick table has ever stored — for their entire existence. See the header of haproxy_tarpit_config.txt and the contract test.

Changing a runtime map (add map / del map)

Same socket, two more ways to fail silently — and both were live in add_ip_to_runtime_map()/remove_ip_from_runtime_map() for their whole existence:

  • Reference the map by FILE PATH, never #<id>. Ids are assigned at config-parse time and move on every config regeneration (on whp01 blocked_ips.map is 37, trusted_ips.map is 10 — there is no id 0). Use add map /etc/haproxy/blocked_ips.map <ip> 1.
  • A mutation answers NOTHING on success, so an empty body is the only success — any output at all is a rejection. Worse, @1 add map #0 <ip> 1 also answers nothing and adds nothing, so the body cannot prove an add worked. Read it back with @1 get map <path> <key>.
  • Entries must carry the value 1; haproxy.cfg matches with map_ip(...,0) -m int gt 0, so a valueless entry does not block.

In Python use haproxy_cli(cmd, worker=True, expect_empty=True) for mutations and runtime_map_lookup() / runtime_map_keys() to verify. The runtime map is only a fast path: /etc/haproxy/blocked_ips.map is authoritative and HAProxy re-reads it on reload, so a failed runtime command must degrade to "enforced on reload" and be reported, never swallowed.

Running the Application

  • Docker Build: docker build -t haproxy-manager .
  • Local Development: python haproxy_manager.py (requires HAProxy, certbot, and dependencies installed)
  • Container Run: See README.md for various docker run configurations

Monitoring and Debugging

  • Error Monitoring: ./scripts/monitor-errors.sh - Monitor application error logs
  • External Monitoring: ./scripts/monitor-errors-external.sh - External monitoring script
  • Health Check: curl http://localhost:8000/health
  • Log Files:
    • /var/log/haproxy-manager.log - General application logs
    • /var/log/haproxy-manager-errors.log - Error logs for alerting

Architecture Overview

Core Components

  1. haproxy_manager.py - Main Flask application providing:

    • RESTful API for HAProxy configuration management
    • SQLite database integration for domain/backend storage
    • Let's Encrypt certificate automation
    • HAProxy configuration generation from Jinja2 templates
    • Optional API key authentication via HAPROXY_API_KEY environment variable

  2. Database Schema - SQLite database with three main tables:

    • domains - Domain configurations with SSL settings
    • backends - Backend service definitions linked to domains
    • backend_servers - Individual servers within backend groups

  3. Template System - Jinja2 templates for HAProxy configuration generation:

    • hap_header.tpl - Global HAProxy settings, defaults, and HTTP/2 tuning
    • hap_backend.tpl - Backend server definitions
    • hap_listener.tpl - Frontend listener configurations with rate limiting
    • hap_letsencrypt.tpl - SSL certificate configurations
    • hap_security_tables.tpl - Stats frontend and security stick tables
    • Template override support for custom backend configurations

  4. Certificate Management - Automated SSL certificate handling:

    • Let's Encrypt integration with certbot
    • Self-signed certificate fallback for development
    • Certificate renewal automation via cron
    • Certificate download endpoints for external services

Configuration Flow

  1. Domain added via /api/domain endpoint → Database updated
  2. generate_config() function → Reads database, renders Jinja2 templates → Writes /etc/haproxy/haproxy.cfg
  3. HAProxy reload via socket API (/tmp/haproxy-cli) or process restart
  4. SSL certificate generation via Let's Encrypt or self-signed fallback

Key Design Patterns

  • Template-driven configuration: HAProxy config generated from modular Jinja2 templates
  • Database-backed state: All configuration persisted in SQLite for reliability
  • API-first design: All operations exposed via REST endpoints
  • Process monitoring: Health checks and automatic HAProxy restart capabilities
  • Comprehensive logging: Operation logging with error alerting support

Authentication & Security

  • Optional API key authentication controlled by HAPROXY_API_KEY environment variable
  • All API endpoints (except /health and /) require Bearer token when API key is set
  • Certificate private keys combined with certificates in HAProxy-compatible format
  • Default backend page for unmatched domains instead of exposing HAProxy errors

Rate Limiting & Connection Limits (hap_listener.tpl)

  • Stick table: type ip size 200k expire 10m tracking conn_cur, conn_rate(10s), http_req_rate(10s), http_err_rate(30s)
  • Tracks real client IP via var(txn.real_ip) to work correctly behind Cloudflare/proxies
  • Rate limit thresholds:
    • Tarpit at 3000 req/10s (300 req/s)
    • Hard block (deny) at 5000 req/10s (500 req/s)
    • Connection rate limit: 500/10s
    • Concurrent connection limit: 500
    • Error rate limit: 100/30s
  • Whitelist bypasses (exempt from rate limits):
    • is_local — RFC1918 private address ranges
    • is_trusted_ip — source IPs listed in trusted_ips.list
    • is_whitelisted — real IPs (from proxy headers) matched in trusted_ips.map

Trusted IP Whitelist Files

  • trusted_ips.list — Source IP whitelist for rate limit bypass (one CIDR/IP per line)
  • trusted_ips.map — Real IP whitelist for proxy-header matching (format: <IP> 1)
  • Both files are baked into the Docker image via COPY in the Dockerfile
  • Ship as comment-only templates (no real IPs). Add trusted IPs locally and do not commit them — this repo is mirrored publicly. Entries persist in the /etc/haproxy named volume across recreates

Timeout Hardening (hap_header.tpl)

  • timeout http-request: 300s -> 30s (slowloris protection)
  • timeout connect: 120s -> 10s
  • timeout client: 10m -> 5m
  • timeout http-keep-alive: 120s -> 30s

HTTP/2 Protection (hap_header.tpl)

  • tune.h2.fe.max-total-streams 2000 — limits total streams per HTTP/2 connection
  • tune.h2.fe.glitches-threshold 50 — CVE-2023-44487 Rapid Reset protection

Stats Frontend (hap_security_tables.tpl)

  • HAProxy stats page bound to 127.0.0.1:8404 (localhost only, accessible inside container)
  • Template: templates/hap_security_tables.tpl

Deployment Context

  • Designed to run as Docker container with persistent volumes for certificates and configurations
  • Exposes ports 80 (HTTP), 443 (HTTPS), and 8000 (management API/UI)
  • Stats page on port 8404 (localhost only inside container)
  • Management interface on port 8000 should be firewall-protected in production
  • Dockerfile HEALTHCHECK verifies both port 8000 (Flask API) and port 80 (HAProxy), with start-period=60s and timeout=10s
  • Supports deployment on servers with git directory at /root/whp and web file sync via rsync to /docker/whp/web/
  • HAProxy is version 3.0.11