Skip to content

docker.md

Latest commit

 

History

History
311 lines (218 loc) · 15.1 KB

docker.md

File metadata and controls

311 lines (218 loc) · 15.1 KB

Docker Integration

Access Method: Docker Socket

HELIOS accesses Docker via the Unix socket /var/run/docker.sock.

Why Docker Socket?

  • Industry standard (used by Portainer, Watchtower, etc.)
  • Full Docker API access
  • No additional configuration required
  • Works with Docker Compose operations

Required permissions:

  • Socket must be mounted into HELIOS container
  • HELIOS runs with access to docker group (or root)

File Access

HELIOS needs read/write access to:

  • compose.yaml – to add/modify services
  • .env – to manage environment variables

Solution: Mount the stack directory and the Docker socket into the HELIOS container (see helios.rb).

volumes:
  - .:/data # Compose resolves this relative to compose.yaml
  - /var/run/docker.sock:/var/run/docker.sock

Inside the container, /data holds compose.yaml, .env, and a helios/ subdirectory for config.yaml and HELIOS' own SQLite files.

Docker Compose Operations

HELIOS performs these operations via Docker API / CLI:

Operation How
Read running containers Docker API: GET /containers/json
Start stack docker compose up --no-build -d <services-except-helios>
Stop stack docker compose down <services-except-helios>
Recreate service docker compose pull <svc> → down <svc> → up --no-build -d
Pull new images docker compose pull
View logs docker compose logs (streamed) + Docker API for recent lines
Listen for events Docker API: GET /events (streaming)

Implementation

HELIOS uses a hybrid approach:

1. docker-api Gem – for direct Docker API access

# Gemfile
gem 'docker-api'

# Usage
Docker.url = 'unix:///var/run/docker.sock'
containers = Docker::Container.all
container.logs(stdout: true, tail: 100)
container.json['State']['Health']['Status']

Used for:

  • Reading container status and health
  • Streaming logs
  • Inspecting container details

2. CLI via Open3 – for Docker Compose operations (see Orchestration::Runner)

require 'open3'

def run_compose(*args)
  cmd = [
    'docker', 'compose',
    '-f', ::Compose.path,
    '--project-directory', host_data_path,
    '--env-file', ::Env.path,
    '--progress', 'plain',
    *args,
  ]
  output, status = Open3.capture2e(*cmd)
  raise CommandError, output unless status.success?
  output
end

Used for:

  • up --no-build -d – start stack (all services except helios itself)
  • down – stop stack
  • pull – pull new images
  • pull <svc> → down <svc> → up --no-build -d <svc> – recreate a service
  • logs -f --timestamps – live log streaming via IO.popen
  • config --hash '*' – detect services whose effective config has drifted

3. Events listener – streams GET /events from the Docker API in a background thread (see Orchestration::EventsListener and events_listener/streaming.rb) and broadcasts service status changes to the browser via Turbo Streams. Only active while at least one subscriber is connected.

Rationale: The docker-api gem provides clean Ruby access to container info and logs, but cannot execute docker compose commands — those require the CLI because they involve YAML parsing, service dependencies, and network setup that only Compose handles. The events listener complements both: it turns Docker's push-based event stream into live UI updates instead of per-request polling.

Generated Files

HELIOS generates two files in the stack directory on every configuration change (and before every compose operation). Both are fully owned by HELIOS — except for parts explicitly tracked as "unmanaged" (user-added services / env vars).

compose.yaml

Generated by Export::Builder, which iterates over all Export::Services::* classes and includes each whose enabled?(configuration) predicate returns true.

Service definitions (see app/services/export/services/):

Service Always / conditional
helios Always (except in development, where HELIOS runs natively)
dashboard Always
power_splitter When grid_import_power, house_power and ≥1 further consumer are mapped
forecast_collector Always
postgresql Always
redis Always
influxdb Always
watchtower Always
senec_collector When a sensor is mapped to a SENEC source
shelly_collector One instance per configured Shelly device
mqtt_collector When a sensor is mapped to an MQTT source
ingest When external push source (ioBroker / Home Assistant) is used
traefik When HTTPS / reverse proxy is enabled

Images come from a mix of hard-coded defaults and user-configurable values (configuration.<service>.image in config.yaml) — see each service class for details. Versioning follows the strategy in Image Versioning Strategy below.

Each service declares its own healthcheck, and dependent services use depends_on: service_healthy. Unmanaged services (imported from existing installations) are appended verbatim.

.env

Generated by Export::Env from the config.yaml singletons, on top of the low-level Env::File parser. Comments and unknown variables from an existing .env are preserved on round-trip (see ADR-0008).

Variables include timezone, installation date, admin password, all secrets (DB passwords, InfluxDB token, SECRET_KEY_BASE), INFLUX_SENSOR_* mappings, and per-service connection settings. Secrets are auto-generated via SecureRandom on first setup and persisted in config.yaml.

Installation Scenario Detection

HELIOS detects the installation scenario at startup (see product.md for details).

Detection method: Import::StackReader parses the existing compose.yaml / .env; Import::ConfigurationImporter classifies what it finds. If the stack contains no services other than HELIOS, the setup wizard runs. If a local target (dashboard / influxdb) or any known collector service is present, an import pass pre-fills config.yaml. A collectors-only install (collectors present, but no local dashboard/InfluxDB) is detected via ConfigurationImporter#collectors_only?.

Scenario A/B (Fresh install): Only HELIOS service exists → setup wizard. User configures devices and selects a data source per device (SENEC/Shelly/MQTT or ioBroker/HA). Collector services are generated only for devices with direct hardware data sources. The distinction between standalone and smart home setups is implicit — no separate wizard question.

Scenario C (Existing installation): Other services present → HELIOS automatically imports compose.yaml and .env on first access, reverse-maps configuration into internal singletons (best-effort), and shows the result to the user for review.

Error Handling

Docker Not Reachable

If Docker socket is not accessible (not mounted, daemon stopped):

  • HELIOS shows a dedicated error page
  • Message: "Cannot connect to Docker"
  • Troubleshooting hints provided (check socket mount, Docker daemon status)
  • No other functionality available until resolved

Missing or Corrupt compose.yaml

If compose.yaml is missing or invalid after initial setup:

  • HELIOS can regenerate the file from config.yaml (see ADR-0009)
  • User is prompted: "Configuration file missing. Regenerate from saved state?"
  • Regeneration restores all HELIOS-managed services
  • User-added services cannot be recovered (warning shown)

Stack Detection

HELIOS identifies containers in its stack via Docker Compose labels. Compose sets these on every container it manages:

  • com.docker.compose.project – project name
  • com.docker.compose.service – service name from compose.yaml
  • com.docker.compose.config-hash – hash of the effective service config (used to detect drift)

Fixed project name. Instead of deriving the project name at runtime, HELIOS requires a fixed value:

# app/services/orchestration.rb
PROJECT_NAME = 'solectrus'.freeze

On startup, StartupCheck#check_compose_project_name parses the top-level name: in compose.yaml and refuses to proceed unless it equals solectrus. This guarantees that container lookups by label always hit the right project, regardless of the host directory name.

Containers are then enumerated via the Docker API and filtered by com.docker.compose.project=solectrus (see Orchestration::Container.all).

Image Versioning Strategy

Defaults live in ConfigSchema; per-service images can be overridden via config.yaml.

Service Image Tag Rationale
SOLECTRUS Dashboard ghcr.io/solectrus/solectrus:latest Own service, always latest
HELIOS ghcr.io/solectrus/helios:develop Own service, currently pre-release
Power-Splitter ghcr.io/solectrus/power-splitter:latest Own service, always latest
Forecast-Collector ghcr.io/solectrus/forecast-collector:latest Own service, always latest
SENEC-Collector ghcr.io/solectrus/senec-collector:latest Own service, always latest
MQTT-Collector ghcr.io/solectrus/mqtt-collector:latest Own service, always latest
Shelly-Collector ghcr.io/solectrus/shelly-collector:latest Own service, always latest
Ingest ghcr.io/solectrus/ingest:latest Own service, always latest
PostgreSQL postgres:18-alpine Major version pinned
Redis redis:8-alpine Major version pinned
InfluxDB influxdb:2.9-alpine Minor version pinned
Traefik traefik:v3.7 Minor version pinned
Watchtower nickfedor/watchtower:latest Fork with additional features

Rationale:

  • Own services use latest – Watchtower handles updates automatically
  • Third-party services pin major version – prevents breaking changes, allows minor/patch updates
  • helios still tracks develop until its first stable release

Compose File Conflict Handling

When HELIOS needs to modify compose.yaml (e.g., adding a service), it may encounter user modifications.

Strategy: Detect and warn

  1. HELIOS tracks which services it manages (stored in config.yaml)
  2. Before modifying, compare current file with expected state
  3. If differences detected in managed services → show warning to user
  4. User decides: apply changes, skip, or review diff

What HELIOS tracks:

  • Services it created (e.g., dashboard, postgresql, influxdb)
  • Expected configuration for each service

What HELIOS preserves:

  • Services it doesn't manage (user-added, e.g., traefik, dozzle) are stored as "unmanaged" in Configuration#data
  • Unmanaged services are written back to compose.yaml verbatim (with ${VAR} references intact)
  • Unknown .env variables referenced by unmanaged services are preserved in a dedicated section
  • A future web-based editor will allow power-users to modify unmanaged services directly in HELIOS

Example conflict scenarios:

Scenario HELIOS behavior
User changed port of dashboard Warning: "Port was modified. Keep your change or reset?"
User added traefik service No warning, service is preserved as unmanaged
User removed redis Warning: "Required service missing. Re-add?"

Health Checks

Health checks are defined natively in Docker Compose. HELIOS reads the health status from Docker API.

Approach:

  • Each service defines its own healthcheck in compose.yaml
  • Docker reports status: healthy, unhealthy, starting
  • HELIOS queries: docker inspect --format '{{.State.Health.Status}}' <container>

Example health checks for compose.yaml:

postgresql:
  healthcheck:
    test: ['CMD-SHELL', 'pg_isready -U postgres']
    interval: 10s
    timeout: 5s
    retries: 5

redis:
  healthcheck:
    test: ['CMD', 'redis-cli', 'ping']
    interval: 10s
    timeout: 5s
    retries: 5

influxdb:
  healthcheck:
    test: ['CMD', 'influx', 'ping']
    interval: 10s
    timeout: 5s
    retries: 5

dashboard:
  healthcheck:
    test: ['CMD-SHELL', 'nc -z 127.0.0.1 3000 || exit 1']
    interval: 10s
    timeout: 5s
    retries: 3

HELIOS behavior:

  • Displays overall status: "All services healthy" or "Problem detected"
  • On problem: Shows which service is unhealthy
  • Status is pushed to the browser live via Orchestration::EventsListener, which streams Docker's GET /events endpoint and broadcasts via Turbo Streams — no per-request polling in the UI
  • Orchestration::StackStatus tightens the refresh cadence while services are in a transient :starting state, to catch fast health transitions the event stream may coalesce