Skip to content

ronaldvdmeer/HomeLog-Sentinel

BranchesTags

Repository files navigation

HomeLog Sentinel

Lightweight local log analysis & security monitoring for homelab environments.

HomeLog Sentinel is a minimal, locally-running log analysis service that:

  • Collects logs centrally via syslog (UDP/TCP/TLS) and file ingestion
  • Performs rule-based security detection with smart filtering
  • Uses a local LLM (Ollama) for intelligent analysis with batched processing
  • Runs hourly AI-based analysis to find weak signals
  • Produces concise daily summaries with host identification
  • Alerts only on meaningful security events with configurable thresholds

Features

  • 🔒 Fully Local - No cloud services, no external APIs
  • 🔐 TLS Support - Encrypted syslog (RFC 5425) with optional mutual TLS
  • 🛡️ Security Hardened - Input validation, rate limiting, prompt injection protection
  • 📊 Rule-Based Detection - SSH brute-force, HTTP auth failures, error spikes (excludes cron/systemd noise)
  • 🤖 AI-Powered Analysis - Batched hourly and daily analysis using Ollama with keyword prioritization
  • 📧 Email Notifications - SMTP with TLS, configurable risk thresholds, and host identification
  • 💾 Simple Storage - SQLite or JSONL with configurable retention
  • 🏷️ Multi-Host Support - Track and identify issues across multiple systems
  • ⚙️ YAML Configuration - Single config file, no database UI needed
  • 🔍 Live Tail - Real-time log viewing with alert filtering

Quick Start

Prerequisites

  • Python 3.10+
  • Git
  • Ollama (optional, for AI analysis)

Installation

# Clone the repository
git clone https://github.com/ronaldvdmeer/HomeLog-Sentinel.git
cd HomeLog-Sentinel

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Linux/Mac
# or
.\venv\Scripts\activate   # Windows

# Install dependencies
pip install -r requirements.txt

# Or install as package
pip install -e .

Updating

For production deployments with systemd, use the included update script:

cd /opt/HomeLog-Sentinel
./update.sh

This will:

  1. Pull latest changes from git
  2. Update Python dependencies
  3. Restart the systemd service
  4. Show the new status

Manual update:

# Pull latest changes
cd HomeLog-Sentinel
git pull origin main

# Activate virtual environment
source venv/bin/activate  # Linux/Mac
.\venv\Scripts\activate   # Windows

# Update dependencies (if requirements changed)
pip install -r requirements.txt

# Restart service if running
systemctl restart homelog-sentinel  # Linux with systemd

Configuration

# Create default configuration
python -m homelog_sentinel init

# Edit configuration
nano config.yaml

Running

# Start the service
python -m homelog_sentinel start

# Or if installed as package
homelog-sentinel start

# Run in debug mode
homelog-sentinel start --debug

# Run without notifications (dry-run)
homelog-sentinel start --dry-run

CLI Commands

# Initialize configuration
homelog-sentinel init [--path ./config.yaml]

# Start the service
homelog-sentinel start [--dry-run] [--debug]

# Check status
homelog-sentinel status

# Manual analysis
homelog-sentinel analyze hourly [--notify]    # Add --notify to send email
homelog-sentinel analyze daily [--notify]

# Live log viewing
homelog-sentinel tail [-n COUNT] [--alerts] [--host HOSTNAME]

# Reset database (delete all logs and analysis)
homelog-sentinel reset --confirm

# Test configuration
homelog-sentinel test [--notification]

Tail Command

View logs in real-time:

# View last 20 logs and follow
homelog-sentinel tail -n 20

# Only show alerts
homelog-sentinel tail --alerts

# Filter by hostname
homelog-sentinel tail --host server1

# Verbose mode (shows parser used, host/IP fields, raw message)
homelog-sentinel tail --verbose

# Combine filters
homelog-sentinel tail --alerts --host server1 -n 50

Verbose mode is useful for debugging parser issues. It shows:

  • Which parser was used (syslog, raw, auth, etc.)
  • The values of source_host and source_ip fields
  • The raw message as received
  • Full message without truncation

Reset Command

Delete all logs and analysis results from the database:

# Reset the database
homelog-sentinel reset --confirm

⚠️ Warning: This permanently deletes all stored logs and analysis results. The database will be recreated automatically when the service starts. The --confirm flag is required to prevent accidental deletion.

Configuration

All configuration is done via a single YAML file. See config.yaml for a complete example.

Log Sources

Syslog (UDP/TCP)

syslog_sources:
  # UDP syslog (default)
  - protocol: udp
    host: 0.0.0.0
    port: 514
    enabled: true

  # TCP syslog (unencrypted)
  - protocol: tcp
    host: 0.0.0.0
    port: 514
    enabled: true

Supported Formats:

  • RFC 3164 (BSD syslog): <priority>timestamp hostname tag: message
  • RFC 5424 (structured syslog): ISO timestamps and structured data
  • CEF (Common Event Format): Security device logs (e.g., Ubiquiti UniFi)

Note: HomeLog Sentinel strictly adheres to RFC 3164 and RFC 5424 standards for traditional syslog. Non-compliant syslog implementations (e.g., missing priority tags or incorrect formatting) will be handled by the fallback RawParser and may not extract hostname information correctly. Configure your syslog sources to send RFC-compliant messages for optimal parsing.

CEF (Common Event Format)

HomeLog Sentinel automatically detects and parses CEF-formatted messages:

Jan 25 22:24:40 UDM Pro Max CEF:0|Ubiquiti|UniFi OS|4.4.6|admins|1|msg=Configuration changed

CEF Format:

  • Extracts hostname from syslog header (if present)
  • Parses CEF fields: Vendor, Product, Version, Signature ID, Name, Severity
  • Converts CEF severity (0-10 scale) to syslog severity levels
  • Parses extension fields (key=value pairs)
  • Stores metadata for advanced querying

CEF Severity Mapping:

  • 0-3 (Low) → INFO/NOTICE
  • 4-6 (Medium) → WARNING
  • 7-8 (High) → ERROR
  • 9-10 (Very High) → CRITICAL/ALERT

Syslog over TLS (RFC 5425)

syslog_sources:
  - protocol: tls
    host: 0.0.0.0
    port: 6514                        # Standard syslog-tls port
    enabled: true
    tls_cert: /path/to/server.crt     # Server certificate
    tls_key: /path/to/server.key      # Server private key
    tls_ca: /path/to/ca.crt           # CA for client verification (optional)
    tls_verify_client: false          # Set to true for mutual TLS

File-Based Ingestion

file_sources:
  - path: /var/log
    pattern: "auth.log"
    parser: auth
    enabled: true
  
  - path: /var/log/nginx
    pattern: "access.log"
    parser: webaccess
    enabled: true

Detection Rules

All detection rules have configurable thresholds and smart filtering:

detection:
  ssh_bruteforce:
    enabled: true
    threshold: 5
    window_minutes: 5
    severity: high
    
  http_auth_failures:
    enabled: true
    threshold: 10
    window_minutes: 5
    severity: medium
    
  sudo_login:
    enabled: true
    threshold: 1
    window_minutes: 1
    severity: medium
    # Note: Automatically ignores cron/systemd sessions
    
  error_spike:
    enabled: true
    threshold: 50
    window_minutes: 10
    severity: high
    
  repeated_errors:
    enabled: true
    threshold: 10
    window_minutes: 5
    severity: medium

Smart Filtering:

  • sudo_login automatically ignores pam_unix(cron:session) and systemd sessions
  • Only alerts on actual human sudo usage via SSH/terminal
  • Reduces false positives from automated system tasks

AI Analysis

Configure Ollama for AI-powered analysis with batched processing:

ollama:
  enabled: true
  endpoint: http://localhost:11434      # Or remote: http://server:11434
  model: gemma3:27b                     # Or llama3.1:8b, gemma3:12b, etc.
  timeout_seconds: 120
  max_context_lines: 100                # Logs per batch (prevents truncation)
  context_size: 16384                   # Model context window in tokens

analysis:
  hourly_enabled: true
  hourly_cron: "0 * * * *"              # Every hour
  daily_enabled: true
  daily_cron: "0 6 * * *"               # Daily at 6 AM
  notify_risk_level: medium             # low, medium, or high

Important: Set max_context_lines based on your model's context window:

  • 100-150 logs: Recommended for most models (prevents prompt truncation)
  • 200-250 logs: For models with large context windows (32K+ tokens)
  • 500+ logs: Only if you have massive context (65K+ tokens) and batching is working

The system will automatically process all logs in batches and combine findings.

Keyword Prioritization

Critical logs are prioritized for AI analysis based on keywords:

  • FAIL, CRITICAL, EMERGENCY, ALERT, FATAL, PANIC
  • DISK, HARDWARE, SMART, MEMORY, OOM
  • CORRUPT, BREACH, ATTACK

This ensures hardware failures and security issues are always analyzed first.

Notifications

notifications:
  smtp:
    enabled: true
    host: smtp.example.com
    port: 587
    username: user@example.com
    password: "your-password"
    use_tls: true
    helo_hostname: homelog-sentinel.yourdomain.com  # For anti-spam
    from_address: homelog@example.com
    to_addresses:
      - admin@example.com
  rate_limit_minutes: 60     # Prevent notification spam

Email Features:

  • ✅ Risk level in subject: [HIGH RISK] (server1, server2) AI Analysis Alert
  • ✅ Affected hosts listed in body and subject
  • ✅ Timestamps and sample log messages for verification
  • ✅ Configurable risk threshold (analysis.notify_risk_level)
  • ✅ Rate limiting per alert type

TLS Configuration

HomeLog Sentinel supports encrypted syslog using TLS (RFC 5425).

Generating Test Certificates

Use the included helper script:

python scripts/generate_tls_certs.py --output ./certs --hostname localhost

This creates:

  • ca.crt - CA certificate
  • server.crt / server.key - Server certificates
  • client.crt / client.key - Client certificates (for mutual TLS)

TLS Security Features

Feature Configuration
Minimum Version TLS 1.2
Cipher Suites ECDHE+AESGCM, DHE+AESGCM, CHACHA20
Client Auth Optional (mutual TLS)
Certificate Validation Required for enabled sources

Configuring rsyslog for TLS

# /etc/rsyslog.conf
module(load="omrelp")
$DefaultNetstreamDriverCAFile /path/to/ca.crt
$DefaultNetstreamDriverCertFile /path/to/client.crt
$DefaultNetstreamDriverKeyFile /path/to/client.key

*.* @@(o)your-server:6514

Security Features

HomeLog Sentinel includes comprehensive security hardening:

Input Validation

Component Protection
Syslog Messages Max size (8KB), encoding validation, priority check
Config Values Type checking, range validation, path traversal prevention
File Paths Forbidden characters, path traversal (..) rejection
IP Addresses Format validation, normalization

Rate Limiting

Per-IP Rate Limiting:
├── Default: 100 messages/second/IP
├── Burst: 500 messages allowed
├── Violations: Auto-block after 10 violations
└── Block Duration: 5 minutes

Sanitization

Data Type Sanitization
Log Messages Control character removal, null byte filtering
Hostnames Lowercase, invalid character stripping
AI Prompts Prompt injection detection, content wrapping

Prompt Injection Protection

Log data sent to AI is:

  1. Redacted - Passwords, tokens, keys removed
  2. Sanitized - Injection attempts filtered
  3. Wrapped - Clear boundaries between instructions and data

Detected patterns:

  • "ignore previous instructions"
  • "you are now..."
  • "pretend to be..."
  • System prompt extraction attempts

Running as a Service

Systemd (Linux)

First, install HomeLog Sentinel in /opt:

# Clone to /opt
cd /opt
sudo git clone https://github.com/ronaldvdmeer/HomeLog-Sentinel.git
cd HomeLog-Sentinel

# Create virtual environment
sudo python3 -m venv venv
sudo ./venv/bin/pip install -r requirements.txt

# Copy and edit configuration
sudo cp config.example.yaml config.yaml
sudo nano config.yaml

# Create service user
sudo useradd -r -s /bin/false homelog

# Set permissions
sudo chown -R homelog:homelog /opt/HomeLog-Sentinel
sudo chmod 600 /opt/HomeLog-Sentinel/config.yaml

Create /etc/systemd/system/homelog-sentinel.service:

[Unit]
Description=HomeLog Sentinel
After=network.target

[Service]
Type=simple
User=homelog
Group=homelog
WorkingDirectory=/opt/HomeLog-Sentinel
ExecStart=/opt/HomeLog-Sentinel/venv/bin/python3 -m homelog_sentinel start
Restart=always
RestartSec=10

# Security hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/HomeLog-Sentinel/data

[Install]
WantedBy=multi-user.target

Then:

sudo systemctl daemon-reload
sudo systemctl enable homelog-sentinel
sudo systemctl start homelog-sentinel

# Check status
sudo systemctl status homelog-sentinel
sudo journalctl -u homelog-sentinel -f

Detection Rules

Built-in Rules

Rule Description Default Threshold Notes
ssh_bruteforce Failed SSH login attempts 5 attempts in 5 minutes
http_auth_failures HTTP 401/403 responses 10 failures in 5 minutes
sudo_login Sudo/root command execution Every occurrence Ignores cron/systemd
error_spike Sudden increase in error messages 50 errors in 10 minutes
repeated_errors Same error repeated 10 times in 5 minutes

Note: The sudo_login rule automatically filters out cron and systemd sessions to prevent false positives.

AI Analysis

Batched Processing

HomeLog Sentinel uses intelligent batched processing to analyze large volumes of logs:

  1. Retrieves up to 1000 logs from the past hour
  2. Prioritizes critical logs by keywords (FAIL, DISK, HARDWARE, etc.)
  3. Splits into batches based on max_context_lines (default: 100)
  4. Analyzes each batch separately with the AI
  5. Combines findings and determines overall risk
  6. Deduplicates results to avoid repetition

This prevents prompt truncation and ensures all logs are analyzed.

Hourly Analysis

Runs every hour and:

  • Reviews logs from the past hour (in batches)
  • Identifies patterns missed by rules
  • Produces structured findings with risk levels
  • Includes affected hostnames for multi-system tracking
  • Shows timestamps and sample logs for verification
  • Only notifies if risk level meets threshold (notify_risk_level)

Daily Summary

Runs once daily and:

  • Summarizes significant events across all hosts
  • Compares trends to previous days
  • Lists all alerts raised with host information
  • Readable in under one minute

AI Data Handling

Before sending to AI:

  1. Sensitive data redacted (passwords, tokens, keys, SSNs, credit cards)
  2. Prompt injection attempts detected and filtered
  3. Data wrapped with clear <user_log_data> boundaries
  4. Connection strings anonymized
  5. Cron/systemd sessions explicitly ignored

Finding Structure

Each AI finding includes:

{
  "observation": "DISK FAILURE detected",
  "risk_level": "high",
  "explanation": "SMART failure on /dev/sda",
  "affected_hosts": ["server1", "server3"],
  "sample_timestamps": ["2026-01-25T20:15:30", "2026-01-25T20:16:45"],
  "sample_messages": ["[CRITICAL] server1/kernel: SMART failure..."]
}

This allows you to:

  • ✅ Verify the issue by checking sample logs
  • ✅ Identify affected systems quickly
  • ✅ Correlate events across multiple hosts

Storage

SQLite (Default)

storage:
  backend: sqlite
  path: ./data
  retention_days: 14

JSONL

storage:
  backend: jsonl
  path: ./data
  retention_days: 14

Testing

Test Configuration

python -m homelog_sentinel test

Simulate SSH Brute-Force

Send test syslog messages:

# Using logger (Linux)
for i in {1..10}; do
    logger -n localhost -P 514 "<38>Jan 25 10:00:00 testhost sshd[1234]: Failed password for invalid user admin from 192.168.1.100 port 22 ssh2"
done

PowerShell:

$msg = "<38>Jan 25 10:00:00 testhost sshd[1234]: Failed password for invalid user admin from 192.168.1.100 port 22 ssh2"
$udp = New-Object System.Net.Sockets.UdpClient
$bytes = [Text.Encoding]::ASCII.GetBytes($msg)
1..6 | ForEach-Object { $udp.Send($bytes, $bytes.Length, "127.0.0.1", 514) }
$udp.Close()

Run Tests

pip install pytest pytest-asyncio
python -m pytest tests/ -v

Project Structure

homelog-sentinel/
├── src/
│   └── homelog_sentinel/
│       ├── __init__.py
│       ├── __main__.py
│       ├── cli.py              # Command line interface
│       ├── config.py           # Configuration management
│       ├── models.py           # Data models
│       ├── parsers.py          # Log parsers
│       ├── ingest.py           # Log ingestion (UDP/TCP/TLS)
│       ├── storage.py          # Storage backends
│       ├── detection.py        # Detection engine
│       ├── ai_analysis.py      # AI analysis
│       ├── notifications.py    # Notification system
│       ├── app.py              # Main application
│       └── security/           # Security module
│           ├── __init__.py
│           ├── validators.py   # Input validation
│           ├── sanitizers.py   # Data sanitization
│           └── rate_limiter.py # Rate limiting
├── scripts/
│   └── generate_tls_certs.py   # TLS certificate generator
├── tests/
│   ├── test_parsers.py
│   ├── test_detection.py
│   └── test_security.py
├── config.yaml                 # Sample configuration
├── requirements.txt
├── pyproject.toml
└── README.md

Security Notes

  • ✅ Runs fully locally with no outbound traffic (except to local Ollama)
  • ✅ Sensitive data is redacted before AI analysis
  • ✅ Prompt injection protection for AI requests
  • ✅ TLS encryption for syslog (RFC 5425)
  • ✅ Per-IP rate limiting with auto-blocking
  • ✅ Input validation on all external data
  • ✅ Path traversal prevention
  • ✅ No root privileges required (unless reading protected logs)

License

MIT License

Contributing

Contributions welcome! Please ensure:

  • Code follows the existing style
  • No new external dependencies without discussion
  • Tests for new features
  • Documentation updates

About

Lightweight, local-first log ingestion and security monitoring for homelabs. Centralizes logs, applies simple detection rules, and adds hourly and daily insights using a local Ollama LLM. No SIEM overhead, no cloud dependencies.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages