Auto discovery and wallbox api (#78)

* Auto discovery and refactoring wallbox api

Author: derolli1976

.github/copilot-instructions.md

@@ -0,0 +1,133 @@
+# Enpal Solar Integration - AI Coding Agent Instructions
+
+## Project Overview
+This is a Home Assistant custom integration that parses data from local Enpal solar system web interfaces (e.g., `http://<enpal-box-ip>/deviceMessages`) and creates sensors dynamically. It supports optional wallbox control via an external add-on API.
+
+**Target**: First-generation Enpal boxes that expose HTML tables on local network (NOT all Enpal systems are supported). Enpal boxes get their IP address via DHCP from the router.
+
+## Architecture & Key Components
+
+### Data Flow
+1. **HTML Scraping** (`utils.py`): Fetches HTML from Enpal box → BeautifulSoup parsing → extracts sensor data from `<div class="card">` elements
+2. **Dynamic Entity Creation** (`sensor.py`, `entity_factory.py`): Parsed data → DataUpdateCoordinator → Auto-generated HA sensor entities
+3. **Optional Wallbox Control** (`button.py`, `switch.py`, `select.py`): Only loaded when `use_wallbox_addon=True` → HTTP POST to `localhost:36725/wallbox/*`
+
+### Critical Files
+- **`utils.py`**: Core parsing logic (`parse_enpal_html_sensors`, `expand_inverter_system_state`). All sensor extraction happens here.
+- **`const.py`**: All constants including `DEFAULT_GROUPS` (sensor categories), unit mappings, device class overrides, icon mappings
+- **`entity_factory.py`**: Factory pattern for creating sensor entities with proper device_class/state_class assignments
+- **`sensor.py`**: Platform setup with DataUpdateCoordinator, fallback to last known data on errors, cumulative energy sensors
+- **`config_flow.py`**: Multi-step UI configuration with auto-discovery and manual setup options, URL validation, group selection, wallbox toggle
+- **`discovery.py`**: Network scanning utilities for auto-discovering Enpal boxes on local subnets
+- **`wallbox_api.py`**: Centralized API client for all wallbox HTTP communication (added to eliminate code duplication)
+
+### Special Parsing Logic: Inverter System State
+The inverter "System State" sensor contains a 200+ character bitfield string like `"State Decimal: 1234 Bits: 0101010101..."`.
+
+**Problem**: String too long (>255 chars) for HA sensor states  
+**Solution**: `expand_inverter_system_state()` in `utils.py` splits it into multiple short sensors:
+- `sensor.inverter_system_state_decimal` (numeric value)
+- `sensor.inverter_system_state_flags` (comma-separated active flags)
+- Individual binary state sensors for each bit (e.g., `sensor.inverter_system_state_standby`)
+
+**Detection**: Uses regex `INV_STATE_RE` OR length check `len(value_raw) > 200 and "Bits" in value_raw`
+
+## Development Workflows
+
+### Running Tests
+```bash
+# Set PYTHONPATH to project root (critical for imports)
+$env:PYTHONPATH = "e:\Github\enpal"
+
+# Run all tests
+pytest
+
+# Run specific test file
+pytest custom_components/enpal_webparser/tests/test_utils.py
+
+# Run with verbose output
+pytest -v
+```
+
+**Test Structure**:
+- `conftest.py`: Fixtures like `real_html` (loads `fixtures/deviceMessages.html`)
+- Tests use real HTML fixtures to validate parsing against actual Enpal output
+- No mocking of BeautifulSoup - tests parse actual HTML structure
+
+### Adding New Sensors
+1. **If sensor appears in HTML but not in HA**: Check if group is enabled in `DEFAULT_GROUPS` (const.py)
+2. **Custom unit/device_class**: Add to `DEVICE_CLASS_OVERRIDES` or `STATE_CLASS_OVERRIDES` in `const.py`
+3. **Custom icon**: Add mapping to `ICON_MAP` using sensor's `unique_id` format (e.g., `"iotedgedevice_cpu_load": "mdi:cpu-64-bit"`)
+
+### Debugging Sensor Issues
+- Check logs with prefix `[Enpal]` - all modules use this
+- Sensor unique_ids generated via `make_id()`: lowercase, non-alphanumeric → `_`
+- Friendly names created via `friendly_name(group, sensor_name)` - handles special formatting like unit letters in parentheses
+
+## Project-Specific Conventions
+
+### Wallbox API Communication Pattern
+All wallbox API calls use the centralized `WallboxApiClient` class (`wallbox_api.py`):
+```python
+from .wallbox_api import WallboxApiClient
+
+api_client = WallboxApiClient(hass)
+success = await api_client.start_charging()  # Returns True/False
+data = await api_client.get_status()  # Returns dict or None
+
+# For actions that need sensor refresh:
+await api_client.call_and_refresh_sensors(
+    "/start",
+    sensor_entities=["sensor.wallbox_status"]
+)
+```
+
+**Key benefits**: Centralized error handling, logging, timeout management, and sensor refresh logic.
+
+### Logging Pattern
+```python
+_LOGGER.info("[Enpal] Your message: %s", variable)  # Always prefix with [Enpal]
+```
+
+### Entity Naming
+- **Unique ID**: `make_id("Inverter Power DC Total")` → `"inverter_power_dc_total"`
+- **Friendly Name**: `friendly_name("Inverter", "Power.DC.L1")` → `"Inverter: Power DC (L1)"`
+
+### Coordinator Pattern
+- Sensors use `DataUpdateCoordinator` with fallback: If fetch fails but `last_successful_data` exists, reuse old values (prevents all sensors going unavailable during transient network issues)
+- Wallbox has separate coordinator because it polls different endpoint (`localhost:36725/wallbox/status`)
+
+### Platform Loading
+- Base platforms: Always `["sensor"]`
+- Optional platforms: `["button", "switch", "select"]` only when `use_wallbox_addon=True`
+- Dynamic loading in `__init__.py`: `async_forward_entry_setups(entry, platforms)`
+
+## Integration Points
+
+### External Dependencies
+- **BeautifulSoup** (`bs4`): HTML parsing - assumes specific structure with `<div class="card"><h2>GroupName</h2><table><tr><td>Sensor</td><td>Value</td><td>Timestamp</td></tr>`
+- **Wallbox Add-on**: Separate Home Assistant add-on (not part of this repo) exposes HTTP API on port 36725
+  - Endpoints: `/start`, `/stop`, `/set_eco`, `/set_solar`, `/set_full`, `/status`
+  - This integration only consumes the API, doesn't implement it
+
+### Home Assistant Integration
+- **Config Flow**: Multi-step UI-only configuration (no YAML)
+  - Step 1: Choose between auto-discovery and manual setup
+  - Step 2 (discovery): Scan local network subnets for Enpal boxes, present found devices
+  - Step 2 (manual): Enter URL manually
+  - Step 3: Configure interval, groups, and wallbox addon
+- **Network Discovery**: Uses platform-specific detection (ip/ipconfig) to detect actual subnet masks, scans all hosts for `/deviceMessages` endpoint, max 1024 hosts safety limit
+- **Restore State**: `DailyResetFromEntitySensor` uses `RestoreEntity` to persist daily energy totals across HA restarts
+- **Entity Registry**: Disabled entities for unselected groups still appear in HA (can be manually enabled later)
+
+## Common Pitfalls
+
+1. **255 Character Limit**: HA sensors have state string length limits. Always truncate long values or split them (see inverter system state)
+2. **Timestamp Parsing**: Use `ENPAL_TIMESTAMP_FORMAT = "%m/%d/%Y %H:%M:%S"` - Enpal uses MM/DD/YYYY format
+3. **Unit Conversion**: Wh → kWh conversion happens in `normalize_value_and_unit()` to match HA energy dashboard expectations
+4. **Wallbox Status Dependency**: Switch/select entities listen to `sensor.wallbox_status` state changes - ensure sensor exists before enabling controls
+5. **Test Isolation**: Tests don't use pytest parametrize - each test explicitly constructs scenarios for clarity
+
+## Branch Context
+Current branch: `72-bug-sensorinverter_system_state-is-longer-than-255`  
+Working on: Fixing the inverter system state string length issue (this is why `expand_inverter_system_state()` exists)

custom_components/enpal_webparser/button.py

@@ -17,16 +17,15 @@
 # See README.md for setup and usage instructions.
 #
 
-import asyncio
 from functools import cached_property
 import logging
 
 from homeassistant.components.button import ButtonEntity
-from homeassistant.helpers.aiohttp_client import async_get_clientsession
 from homeassistant.helpers.device_registry import DeviceInfo
 from homeassistant.helpers.entity import EntityCategory
 
-from .const import DOMAIN, DEFAULT_WALLBOX_API_ENDPOINT
+from .const import DOMAIN
+from .wallbox_api import WallboxApiClient
 
 
 _LOGGER = logging.getLogger(__name__)
@@ -45,30 +44,40 @@ async def async_setup_entry(hass, config_entry, async_add_entities):
 
     _LOGGER.info("[Enpal] Setting up Wallbox buttons")
 
-    base_url = DEFAULT_WALLBOX_API_ENDPOINT
+    api_client = WallboxApiClient(hass)
 
     buttons = [
-        EnpalWallboxButton(hass, "Ladevorgang starten", f"{base_url}/start", "start"),
-        EnpalWallboxButton(hass, "Ladevorgang stoppen", f"{base_url}/stop", "stop"),
+        EnpalWallboxButton(hass, api_client, "Ladevorgang starten", "start", "start"),
+        EnpalWallboxButton(hass, api_client, "Ladevorgang stoppen", "stop", "stop"),
     ]
 
     for key, label in MODES.items():
         button_name = f"Modus {label} aktivieren"
-        buttons.append(EnpalWallboxButton(hass, button_name, f"{base_url}/set_{key}", f"set_{key}"))
+        buttons.append(EnpalWallboxButton(hass, api_client, button_name, f"set_{key}", f"set_{key}"))
         _LOGGER.debug("[Enpal] Added button for mode: %s", button_name)
 
     async_add_entities(buttons)
     _LOGGER.info("[Enpal] Wallbox buttons successfully added")
 
 
 class EnpalWallboxButton(ButtonEntity):
-    def __init__(self, hass, name, url, unique_id):
+    def __init__(self, hass, api_client: WallboxApiClient, name: str, action: str, unique_id: str):
+        """Initialize the wallbox button.
+        
+        Args:
+            hass: Home Assistant instance
+            api_client: Wallbox API client instance
+            name: Display name for the button
+            action: Action to perform (start, stop, set_eco, set_solar, set_full)
+            unique_id: Unique identifier for the entity
+        """
         self._hass = hass
+        self._api_client = api_client
+        self._action = action
         self._attr_name = f"Wallbox {name}"
-        self._url = url
         self._attr_unique_id = f"enpal_wallbox_button_{unique_id}"
         self._attr_entity_category = EntityCategory.CONFIG
-        _LOGGER.debug("[Enpal] Created button entity: %s (URL: %s)", self._attr_name, self._url)
+        _LOGGER.debug("[Enpal] Created button entity: %s (action: %s)", self._attr_name, self._action)
 
     @cached_property
     def device_info(self) -> DeviceInfo:
@@ -80,29 +89,29 @@ def device_info(self) -> DeviceInfo:
         )
 
     async def async_press(self):
+        """Handle button press."""
         _LOGGER.info("[Enpal] Button pressed: %s", self._attr_name)
-        try:
-            session = async_get_clientsession(self._hass)
-            async with session.post(self._url, timeout=30) as response:
-                if response.status != 200:
-                    text = await response.text()
-                    _LOGGER.warning("[Enpal] Wallbox command failed (%s): %s", response.status, text)
-                else:
-                    _LOGGER.info("[Enpal] Wallbox command successful: %s", self._url)
-        except Exception as e:
-            _LOGGER.exception("[Enpal] Wallbox request failed: %s", e)
-
-        # Wait a moment to allow the wallbox to process the change
-        await asyncio.sleep(2)
-
-        await self._hass.services.async_call(
-            "homeassistant",
-            "update_entity",
-            {
-                "entity_id": [
-                    "sensor.wallbox_lademodus",
-                    "sensor.wallbox_status",
-                ]
-            },
-            blocking=True
+        
+        # Map action to API client method
+        action_map = {
+            "start": self._api_client.start_charging,
+            "stop": self._api_client.stop_charging,
+            "set_eco": self._api_client.set_mode_eco,
+            "set_solar": self._api_client.set_mode_solar,
+            "set_full": self._api_client.set_mode_full,
+        }
+        
+        api_method = action_map.get(self._action)
+        if not api_method:
+            _LOGGER.error("[Enpal] Unknown action: %s", self._action)
+            return
+        
+        # Call API and refresh sensors
+        endpoint = f"/{self._action}"
+        await self._api_client.call_and_refresh_sensors(
+            endpoint,
+            sensor_entities=[
+                "sensor.wallbox_lademodus",
+                "sensor.wallbox_status",
+            ]
         )