Readme and tests

Author: tomquist

CHANGELOG.md

@@ -13,7 +13,7 @@
 ### Added
 - **Added** CT002/CT003 emulation for steering multiple Marstek storage devices over the Marstek CT UDP protocol. Active control is on by default (`ACTIVE_CONTROL = True`): the emulator smooths the grid reading, splits the target across batteries with a 15 W `BALANCE_DEADBAND`, and runs time-weighted saturation detection with handoff — set `ACTIVE_CONTROL = False` for relay mode (raw meter values forwarded, batteries decide). Includes fair-share balancing (`FAIR_DISTRIBUTION`, `BALANCE_GAIN`), manual target override and forced rotation via MQTT, ARP-based consumer discovery, and an opt-in efficiency mode that concentrates power on fewer batteries at low demand (`MIN_EFFICIENT_POWER`, `EFFICIENCY_ROTATION_INTERVAL`, probe-based fades, `SATURATION_GRACE_SECONDS`) ([#283](https://github.com/tomquist/astrameter/pull/283), [#284](https://github.com/tomquist/astrameter/pull/284), [#287](https://github.com/tomquist/astrameter/pull/287), [#289](https://github.com/tomquist/astrameter/pull/289), [#291](https://github.com/tomquist/astrameter/pull/291), [#293](https://github.com/tomquist/astrameter/pull/293), [#294](https://github.com/tomquist/astrameter/pull/294), [#296](https://github.com/tomquist/astrameter/pull/296), [#298](https://github.com/tomquist/astrameter/pull/298), [#301](https://github.com/tomquist/astrameter/pull/301), [#303](https://github.com/tomquist/astrameter/pull/303), [#310](https://github.com/tomquist/astrameter/pull/310), [#311](https://github.com/tomquist/astrameter/pull/311), [#320](https://github.com/tomquist/astrameter/pull/320), [#321](https://github.com/tomquist/astrameter/pull/321)).
 - **Added** MQTT Insights: optional `[MQTT_INSIGHTS]` section publishes internal state (grid power, targets, saturation, consumer topology, EMA poll interval) to MQTT with Home Assistant Device Discovery; per-consumer active/pause + manual target control; Shelly battery offline availability; auto-configured in the HA app when Mosquitto is installed ([#292](https://github.com/tomquist/astrameter/pull/292), [#294](https://github.com/tomquist/astrameter/pull/294), [#297](https://github.com/tomquist/astrameter/pull/297), [#300](https://github.com/tomquist/astrameter/pull/300), [#306](https://github.com/tomquist/astrameter/pull/306)).
-- **Added** Marstek MQTT responder inside MQTT Insights: when `[MARSTEK]` credentials are configured, AstraMeter answers the Marstek CT002/CT003 poll protocol on the MQTT broker using the managed cloud MAC, so combined with [hame-relay](https://github.com/tomquist/hame-relay) the emulator's readings appear as a CT002/CT003 in the Marstek mobile app. Enabled by default; set `MARSTEK_MQTT_ENABLED = false` in `[MQTT_INSIGHTS]` to opt out.
+- **Added** optional Marstek MQTT responder alongside MQTT Insights (HA is the main use case): when `[MARSTEK]` is configured, AstraMeter can answer CT002/CT003 poll traffic on the same broker using the managed cloud MAC; with [hame-relay](https://github.com/tomquist/hame-relay) **≥ 1.3.5** on that broker the Marstek mobile app shows live readings (see README, MQTT Insights). On by default; set `MARSTEK_MQTT_ENABLED = false` in `[MQTT_INSIGHTS]` to disable only this add-on.
 - **Added** opt-in web-based configuration editor (`WEB_CONFIG_ENABLED = True` in `[GENERAL]`) accessible at `http://<host>:52500/config`; supports editing all config sections and keys with type-appropriate inputs, comment preservation, and a Save & Restart button ([#319](https://github.com/tomquist/astrameter/pull/319)).
 - **Added** HomeWizard P1 powermeter via the device WebSocket API, with optional `VERIFY_SSL` ([#231](https://github.com/tomquist/astrameter/pull/231), [#254](https://github.com/tomquist/astrameter/pull/254)).
 - **Added** Enphase IQ Gateway (Envoy) powermeter via the local HTTPS `production.json` API, with optional Enlighten-cloud token acquisition and automatic refresh on 401, and auto-detection of single- vs three-phase readings ([#245](https://github.com/tomquist/astrameter/pull/245)).

README.md

@@ -336,6 +336,7 @@ Optional Marstek cloud auto-registration:
   - Refresh the CT device list after registration (or log out/in if needed). Then select `AstraMeter CT002` / `AstraMeter CT003`, switch battery mode to automatic, and choose that CT. It should be selectable as soon as it appears in the device list.
   - Marstek credentials are only needed for one-time registration. You can remove `MARSTEK.MAILBOX` / `MARSTEK.PASSWORD` immediately after registration succeeds (or if the managed device already exists).
   - If you use Home Assistant app `custom_config`, values from that file take precedence over app UI fields.
+  - **Marstek app (optional):** live CT grid power over MQTT uses the same `[MQTT_INSIGHTS]` broker as [hame-relay](https://github.com/tomquist/hame-relay) **≥ 1.3.5**; see [MQTT Insights](#mqtt-insights) (optional Marstek subsection). HA entities do not depend on this.
 
 ### Value Transformation
 
@@ -861,11 +862,13 @@ CURRENT_POWER_ENTITY = sensor.current_power
 
 ### MQTT Insights
 
-Publishes internal state (grid power per phase, charge targets, saturation, consumer topology) to MQTT with optional Home Assistant auto-discovery.
+**Primary use:** publish CT002/Shelly internal state (grid power, targets, saturation, topology, switches) to MQTT with **optional Home Assistant MQTT Device Discovery** so entities show up in HA.
 
-**Home Assistant App**: When running as an HA app with the Mosquitto broker app installed, MQTT Insights is auto-configured — no manual setup needed. Entities appear automatically in HA.
+**Home Assistant app:** With the Mosquitto add-on installed, MQTT Insights is auto-configured; entities appear without manual `[MQTT_INSIGHTS]` wiring.
 
-**Manual configuration**:
+**Small add-on:** the same broker connection can optionally answer **Marstek CT002/CT003 MQTT polls** so the Marstek mobile app shows live grid power when you use [hame-relay](https://github.com/tomquist/hame-relay) on that broker (see below). You can turn that off with `MARSTEK_MQTT_ENABLED=false` and keep HA publishing unchanged.
+
+**Manual configuration** (when not using the HA app defaults):
 
 ```ini
 [MQTT_INSIGHTS]
@@ -889,9 +892,24 @@ HA_DISCOVERY_PREFIX = homeassistant
 | `BASE_TOPIC` | `astrameter` | Root topic for all published messages |
 | `HA_DISCOVERY` | `true` | Enable Home Assistant MQTT Device Discovery |
 | `HA_DISCOVERY_PREFIX` | `homeassistant` | HA discovery topic prefix |
-| `MARSTEK_MQTT_ENABLED` | `true` | Respond to Marstek CT002/CT003 MQTT polls on this broker (requires `[MARSTEK]`) |
+| `MARSTEK_MQTT_ENABLED` | `true` | Optional: answer Marstek app CT002/CT003 polls on this broker (needs `[MARSTEK]`); set `false` for HA-only |
+| `MARSTEK_MQTT_INTERVAL` | `300` | Optional: seconds between background aggregate publishes for the app; `0` = polls only |
+
+#### Optional: Marstek mobile app (live MQTT)
+
+This is **not** required for Home Assistant. It only helps the **Marstek app** show live CT002/CT003 grid power over the same cloud MQTT path when **[hame-relay](https://github.com/tomquist/hame-relay)** bridges your broker—use **hame-relay ≥ 1.3.5** so poll/replies work reliably. UDP between batteries and AstraMeter is unchanged for control.
+
+**If you want it**
+
+- **`[MARSTEK]`** — Managed fake CT so the **MQTT MAC** matches the cloud device.
+- **Same broker as hame-relay** — `[MQTT_INSIGHTS]` must point at the broker relay uses toward Marstek's cloud.
+
+**Toggles** (defaults in table)
+
+- **`MARSTEK_MQTT_ENABLED`** — `false` = HA MQTT Insights only, no Marstek poll replies.
+- **`MARSTEK_MQTT_INTERVAL`** — Optional periodic aggregate pushes; **`0`** = answer polls only.
 
-**Marstek app visibility**: when `[MARSTEK]` credentials are configured, AstraMeter registers a managed fake CT device in the Marstek cloud. With `MARSTEK_MQTT_ENABLED=true` (default) it also answers the CT's MQTT polls on this broker. Combined with [hame-relay](https://github.com/tomquist/hame-relay) bridging the local broker to the Marstek cloud, the emulator's readings then show up as a CT002/CT003 in the Marstek mobile app. Set `MARSTEK_MQTT_ENABLED=false` to keep MQTT Insights but opt out of this responder.
+Replies follow the usual `hame_energy/…` / `marstek_energy/…` App/device topics for a real CT; AstraMeter matches your CT002/CT003 **type** and **MAC**.
 
 **Published entities** (per CT002 consumer):
 - Grid power (L1/L2/L3/total), charge target (L1/L2/L3), reported power, saturation

config.ini.example

@@ -130,6 +130,9 @@ THROTTLE_INTERVAL = 0
 ## Prefix is fixed to 02b250 (locally administered MAC space) for managed fake devices.
 ## Used for add-device request
 #TIMEZONE = Europe/Berlin
+## Optional: live grid in the Marstek mobile app over MQTT uses [MQTT_INSIGHTS]
+## on the same broker as hame-relay (https://github.com/tomquist/hame-relay) >= 1.3.5
+## (HA discovery does not depend on this).
 
 #[SHELLY]
 #TYPE = 1PM #PLUS1PM #EM #3EM #3EMPRO
@@ -398,9 +401,10 @@ THROTTLE_INTERVAL = 0
 #HAMPEL_MIN_THRESHOLD = 50
 
 ## --- MQTT Insights (optional) ---
-## Publishes internal state (grid power, targets, saturation, consumer topology)
-## to MQTT with Home Assistant Device Discovery support.
+## Mainly for Home Assistant: publishes internal state (grid power, targets,
+## saturation, consumer topology) to MQTT with HA Device Discovery.
 ## When running as a Home Assistant app with Mosquitto, this is auto-configured.
+## Optional add-on: Marstek app poll replies use the same broker (see MARSTEK_* below).
 #[MQTT_INSIGHTS]
 #BROKER = 192.168.1.100
 #PORT = 1883
@@ -418,14 +422,10 @@ THROTTLE_INTERVAL = 0
 #HA_DISCOVERY = true
 ## HA discovery prefix (default: homeassistant)
 #HA_DISCOVERY_PREFIX = homeassistant
-## Respond to Marstek CT002/CT003 MQTT polls on this broker (default: true).
-## Combined with hame-relay (https://github.com/tomquist/hame-relay) the
-## emulator's CT readings surface in the Marstek mobile app. Requires a
-## working [MARSTEK] section: the MAC used in the MQTT topics is the
-## managed fake device AstraMeter registers in the Marstek cloud.
+## Optional Marstek mobile app: answer CT002/CT003 MQTT polls on this broker (default: true).
+## Live readings in the app need hame-relay (https://github.com/tomquist/hame-relay) >= 1.3.5
+## on the same broker; older relays may not forward poll/replies. Requires [MARSTEK] so the
+## topic MAC matches the managed fake device. Set false for HA MQTT Insights only.
 #MARSTEK_MQTT_ENABLED = true
-## Periodic broadcast interval (seconds) for Marstek MQTT responses.
-## Publishes power values for all registered bindings at this cadence
-## so the Marstek app stays up-to-date without relying solely on its own polls.
-## Set to 0 to disable periodic broadcasts (respond only to polls).
+## Seconds between optional Marstek aggregate (cd=1) broadcasts when the app is quiet; 0 = polls only.
 #MARSTEK_MQTT_INTERVAL = 300

src/astrameter/mqtt_insights/mqtt_insights_test.py

@@ -492,6 +492,9 @@ def _make_service(port: int, base_topic: str | None = None) -> MqttInsightsServi
             base_topic=base_topic,
             ha_discovery=True,
             ha_discovery_prefix=f"ha_disc_{_test_counter}",
+            # Broader E2E tests assert poll-only Marstek behaviour; periodic traffic
+            # is covered by test_marstek_periodic_broadcast.
+            marstek_mqtt_interval=0.0,
         )
     )
 

src/astrameter/mqtt_insights/service.py

@@ -77,7 +77,7 @@ class MqttInsightsConfig:
     # Periodic broadcast interval (seconds). When > 0 and marstek_mqtt_enabled,
     # publish power values for every registered binding at this cadence so the
     # Marstek app stays up-to-date without relying solely on its own polls.
-    marstek_mqtt_interval: float = 1
+    marstek_mqtt_interval: float = 300.0
 
 
 @dataclass