Skip to content

CONFIGURATION.md

Latest commit

 

History

History
520 lines (395 loc) · 11.5 KB

CONFIGURATION.md

File metadata and controls

520 lines (395 loc) · 11.5 KB
Raw

PulseCheck Configuration Guide

This guide provides comprehensive documentation for configuring PulseCheck in production environments.

Table of Contents

Configuration File

PulseCheck uses YAML for configuration. The system searches for config.yaml in the following locations (in order):

  1. Path specified by --config flag
  2. ./config.yaml (current directory)
  3. ~/.pulsecheck/config.yaml (user home directory)
  4. ~/.config/pulsecheck/config.yaml (XDG config directory)
  5. /etc/pulsecheck/config.yaml (system directory)

If no configuration file is found, PulseCheck uses sensible defaults.

Creating Your Configuration

# Copy the example configuration
cp config.example.yaml config.yaml

# Edit with your settings
vi config.yaml

Environment Variables

All configuration values can be overridden with environment variables using the format:

PULSECHECK_<SECTION>_<KEY>=value

Nested values use underscores:

export PULSECHECK_DATABASE_PATH=/data/pulsecheck.db
export PULSECHECK_SERVER_PORT=9090
export PULSECHECK_WORKER_POOL_NUM_WORKERS=50
export PULSECHECK_NOTIFICATIONS_SLACK_WEBHOOK_URL="https://hooks.slack.com/..."
export PULSECHECK_LOGGING_LEVEL=debug
export PULSECHECK_SERVER_TLS_ENABLED=true

Environment variables take precedence over file configuration.

Configuration Sections

Database Configuration

Controls SQLite database settings.

database:
  # Path to SQLite database file
  path: "./pulsecheck.db"

  # Maximum concurrent database connections
  max_connections: 10

  # Enable Write-Ahead Logging for better concurrency
  # Recommended: true for production
  enable_wal: true

Path Handling:

  • Relative paths are resolved from the working directory
  • Automatically converted to absolute paths on load
  • Directory must exist and be writable

WAL Mode:

  • Enables concurrent reads and writes
  • Improves performance under load
  • Recommended for production deployments

Server Configuration

HTTP server settings for the API server.

server:
  # Host to bind to
  # Use "0.0.0.0" to listen on all interfaces
  # Use "127.0.0.1" for local-only access
  host: "0.0.0.0"

  # Port to listen on
  port: 8080

  # HTTP timeout settings
  read_timeout: "30s"
  write_timeout: "30s"
  idle_timeout: "120s"

  # TLS/HTTPS configuration
  tls:
    enabled: false
    cert_file: "/path/to/cert.pem"
    key_file: "/path/to/key.pem"

  # CORS configuration
  cors:
    enabled: false
    allowed_origins:
      - "*"
    allowed_methods:
      - "GET"
      - "POST"
      - "PUT"
      - "DELETE"
    allowed_headers:
      - "Content-Type"
      - "Authorization"

Timeout Guidelines:

  • read_timeout: Time to read request headers and body
  • write_timeout: Time to write response
  • idle_timeout: Keep-alive timeout between requests

TLS Best Practices:

  • Use Let's Encrypt for free SSL certificates
  • Keep private keys secure (never commit to version control)
  • Use modern cipher suites (automatically configured)
  • Consider using a reverse proxy (nginx, Caddy) for TLS termination

Worker Pool Configuration

Controls the concurrency and scheduling of URL checks.

worker_pool:
  # Number of concurrent worker goroutines
  # Recommended: 10 for <100 URLs, 50 for <1000 URLs, 100 for >1000 URLs
  num_workers: 10

  # Channel buffer size for work distribution
  # Recommended: 2x num_workers
  channel_buffer_size: 20

  # How often to poll database for URLs due for checking
  check_interval: "10s"

Sizing Guidelines:

URLs Monitored Workers Buffer Size Check Interval
1-100 10 20 10s
100-500 25 50 15s
500-1000 50 100 30s
1000-5000 100 200 60s
5000+ 200 400 120s

Performance Tips:

  • More workers = higher throughput but more memory usage
  • Buffer size prevents blocking when submitting checks
  • Check interval should be less than shortest URL frequency

Notification Configuration

Configure alerting channels for down services.

Slack

notifications:
  slack:
    enabled: true
    webhook_url: "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"

Setup:

  1. Go to https://api.slack.com/messaging/webhooks
  2. Create a new webhook for your channel
  3. Copy the webhook URL to config
  4. Test with: curl -X POST -H 'Content-Type: application/json' -d '{"text":"Test"}' YOUR_WEBHOOK_URL

Discord

notifications:
  discord:
    enabled: true
    webhook_url: "https://discord.com/api/webhooks/YOUR/WEBHOOK"

Setup:

  1. Open Discord channel settings
  2. Go to Integrations > Webhooks
  3. Create webhook and copy URL
  4. Test with: curl -X POST -H 'Content-Type: application/json' -d '{"content":"Test"}' YOUR_WEBHOOK_URL

Email

notifications:
  email:
    enabled: true
    smtp_server: "smtp.gmail.com"
    port: 587
    username: "your-email@gmail.com"
    password: "your-app-password"
    from: "pulsecheck@example.com"
    recipients:
      - "oncall@example.com"
      - "devops-team@example.com"
    use_tls: true

SMTP Providers:

Provider Server Port TLS
Gmail smtp.gmail.com 587 true
Outlook smtp.office365.com 587 true
SendGrid smtp.sendgrid.net 587 true
Amazon SES email-smtp.region.amazonaws.com 587 true

Gmail Setup:

  1. Enable 2-factor authentication
  2. Generate app password: https://myaccount.google.com/apppasswords
  3. Use app password (not account password)

Security Note: Store passwords in environment variables, not in config files:

export PULSECHECK_NOTIFICATIONS_EMAIL_PASSWORD="your-app-password"

Logging Configuration

Configure logging format and verbosity.

logging:
  # Log level: debug, info, warn, error
  level: "info"

  # Log format: json, text
  format: "text"

  # Log output: stdout, stderr, or file path
  output: "stdout"

Level Guidelines:

  • debug: Development, troubleshooting (very verbose)
  • info: Production default, shows key operations
  • warn: Production alternative, only warnings and errors
  • error: Only errors, minimal logging

Format:

  • text: Human-readable, good for development
  • json: Machine-parsable, good for log aggregation (ELK, Splunk, etc.)

Output:

  • stdout: Standard output (default, works with systemd, Docker)
  • stderr: Standard error stream
  • /path/to/file.log: Write to file (ensure directory exists and is writable)

Features Configuration

Feature flags and security settings.

features:
  # Enable SSRF (Server-Side Request Forgery) protection
  # Blocks requests to private IPs, localhost, and cloud metadata
  # Recommended: true for production
  enable_ssrf_protection: true

  # Default timeout for URL checks (seconds)
  # Individual URLs can override this
  request_timeout: 30

  # Maximum response body size to read (bytes)
  # Prevents memory exhaustion from large responses
  max_response_body_size: 10485760  # 10MB

SSRF Protection: When enabled, blocks access to:

  • Loopback addresses (127.0.0.1, localhost)
  • Private IP ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16)
  • Link-local addresses (169.254.0.0/16)
  • Cloud metadata endpoints (169.254.169.254)

Disable only for testing or trusted internal networks.

Production Best Practices

1. Security

# Enable SSRF protection
features:
  enable_ssrf_protection: true

# Use TLS for API server
server:
  tls:
    enabled: true
    cert_file: "/etc/letsencrypt/live/yourdomain.com/fullchain.pem"
    key_file: "/etc/letsencrypt/live/yourdomain.com/privkey.pem"

# Store secrets in environment variables
# Don't commit passwords to git

2. Performance

# Enable WAL mode
database:
  enable_wal: true
  max_connections: 25

# Size worker pool appropriately
worker_pool:
  num_workers: 50
  channel_buffer_size: 100

3. Reliability

# Set appropriate timeouts
server:
  read_timeout: "30s"
  write_timeout: "30s"
  idle_timeout: "120s"

# Configure multiple notification channels
notifications:
  slack:
    enabled: true
  email:
    enabled: true

4. Monitoring

# Use structured logging
logging:
  level: "info"
  format: "json"
  output: "/var/log/pulsecheck/app.log"

5. Deployment

# Use systemd for process management
sudo systemctl enable pulsecheck
sudo systemctl start pulsecheck

# Or use Docker
docker run -v /data:/data -v ./config.yaml:/config.yaml pulsecheck

# Set resource limits
ulimit -n 10000  # File descriptors
ulimit -u 2048   # Processes

Examples

Development Configuration

database:
  path: "./dev.db"
  enable_wal: false

server:
  host: "127.0.0.1"
  port: 8080

worker_pool:
  num_workers: 5
  check_interval: "5s"

logging:
  level: "debug"
  format: "text"
  output: "stdout"

features:
  enable_ssrf_protection: false

Production Configuration

database:
  path: "/var/lib/pulsecheck/production.db"
  enable_wal: true
  max_connections: 25

server:
  host: "0.0.0.0"
  port: 443
  read_timeout: "30s"
  write_timeout: "30s"
  idle_timeout: "120s"
  tls:
    enabled: true
    cert_file: "/etc/letsencrypt/live/monitor.example.com/fullchain.pem"
    key_file: "/etc/letsencrypt/live/monitor.example.com/privkey.pem"

worker_pool:
  num_workers: 100
  channel_buffer_size: 200
  check_interval: "30s"

notifications:
  slack:
    enabled: true
    webhook_url: "${SLACK_WEBHOOK_URL}"
  email:
    enabled: true
    smtp_server: "smtp.sendgrid.net"
    port: 587
    username: "apikey"
    password: "${SENDGRID_API_KEY}"
    recipients:
      - "oncall@example.com"

logging:
  level: "info"
  format: "json"
  output: "/var/log/pulsecheck/app.log"

features:
  enable_ssrf_protection: true
  request_timeout: 30
  max_response_body_size: 10485760

High-Scale Configuration

For monitoring 5000+ URLs:

database:
  path: "/data/pulsecheck.db"
  enable_wal: true
  max_connections: 50

worker_pool:
  num_workers: 200
  channel_buffer_size: 400
  check_interval: "120s"

features:
  request_timeout: 20
  max_response_body_size: 5242880  # 5MB

logging:
  level: "warn"  # Reduce log volume
  format: "json"

Troubleshooting

Config Not Found

Warning: No config file found, using defaults

Solution: Place config.yaml in one of the search paths or use --config flag.

Database Permission Error

Failed to initialize database: unable to open database file

Solution: Ensure database directory exists and has write permissions.

Notification Failure

Error sending notification: webhook returned status 404

Solution: Verify webhook URL is correct and accessible. Test with curl.

Worker Pool Overload

Worker pool channel full, dropping checks

Solution: Increase num_workers and channel_buffer_size.

Memory Issues

fatal error: out of memory

Solution: Reduce num_workers, decrease max_response_body_size, or increase system memory.

Additional Resources