Development - #108
Merged
Merged
Conversation
…ogic to be conditional based on if on grid or off grid
Coordinator now bootstraps unified-topic state by publishing
<dongle>/snapshot/request after MQTT subscriptions are set up. Dongle FW
>= 4.3.0 responds with full /input + /hold retained payloads; older
firmware ignores the request and the per-bank topics continue to feed
entities (backward compatible).
No changes to process_message dispatch — the wildcard subscription
<dongle>/# already routes /input and /hold; the per-key entity
lookup in determine_entity_type is bank-agnostic so the same key
landing on either topic updates the same entity.
Tests:
tests/test_unified_topics.py — 9 regression tests covering
per-bank backward compat, unified-topic routing, per-key delta,
same-key-on-both-topics convergence, GridBoss path, flat payload.
conftest.py — minimal coordinator fixture via __new__ bypassing
__init__, real homeassistant package satisfies imports.
pytest.ini — asyncio mode auto, function-scope loop.
Dongle FW >= 4.3.0 publishes <dongle>/setting/updated on every
successful write regardless of who initiated it. Envelope:
{setting, value, reg, from, ts}. Coordinator intercepts this topic
early in process_message and mirrors value to the matching entity
so HA's state converges within ~1 ms instead of waiting for the next
/hold round.
Backward compat: old firmware never publishes the topic, so the
intercept is a no-op for pre-4.3 devices.
Tests:
test_setting_updated_routes_to_entity — entity mirrors new value
test_setting_updated_handles_missing_fields — malformed envelope
is silently ignored, no crash
test_deye_input_routes_canonical_keys — Deye unified /input uses
the same Vpv1/Ppv1/SOC keys as Lux (HA dispatch is brand-agnostic)
test_gridboss_input_routes_state_and_codes — GridBoss unified /input
includes State + FaultCode + WarningCode
test_input_state_treated_as_full_snapshot — input_state envelope
overwrites prior state regardless of input_delta history
14/14 tests pass.
Deye/SunSynk/SolArk/NeoVolta entity support: single "Deye" ENTITIES entry (114
entities, unique_ids 1:1 with the dongle's deye catalog keys) — 84 sensor, 30
number, 6 select, 4 switch. Added "Deye" to the config_flow brand dropdown
(label "Deye / SunSynk / SolArk / NeoVolta"). TimeSlots nested array lands as one
opaque disabled-by-default sensor (per-slot entities = documented TODO, needs
firmware flat keys or a Deye nested-flatten).
Coordinator/entity fixes for the unified-topics dongle (v4.3):
- process_message: treat <dongle>/availability as a plain online/offline string
(LWT), not JSON — was logging "Invalid JSON" on every availability message.
- _create_entities_for_dongle: isinstance(dict) guard — some brand registries
have legacy flat-list entries that crashed .items() ('list' object has no
attribute 'items'), aborting entity creation.
- setting/updated channel: mirror the new value to the matching entity so HA
converges in ~ms instead of waiting for the next /hold cycle.
- switch.py: coerce string "0"/"1" via int() before bool() (bool("0") was True).
Tests: 19 pass (16 original + 3 Deye: catalog-consistency guard, Deye hold-settings
routing, Deye family input routing).
Make the dongle ID prefix in entity_ids optional. Fresh single-dongle installs drop it for cleaner names; existing installs keep it unless the user opts in via the options flow. History (states + long-term statistics) is anchored to unique_id and preserved across the scheme change by renaming through the entity registry (HA migrates history automatically), never by a code-only entity_id change. - const: add CONF_DROP_DONGLE_ID (install-time contextual default) - coordinator: add build_entity_id() as the single source of truth for entity_id construction; drops the dongle prefix only on single-dongle installs (multi-dongle keeps it to avoid collisions) - migration: new async_migrate_entity_ids() renames via the entity registry before platform setup; lookup by config-entry + unique_id doubles as reinstall auto-detection; collision-guarded and per-entity exception-safe - platforms: route all entity_id assignments and cross-entity lookups through build_entity_id (combined/aggregate entities left untouched) - config_flow: required install_kind question (fresh -> drop, reconnect -> keep old scheme so history lines up); single-dongle-only options toggle - translations: strings for install_kind and drop_dongle_id Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Snapshot bootstrap was fired at MQTT-subscription time, before the dongle's firmware version was known and before a fresh-install dongle had published — so FW >= 4.3.0 dongles (which stream change-data only) left entities stuck at 'unknown'. Now driven by the /status payload that actually carries the version. - coordinator: parse the /status `version` (e.g. "4.3.0.111S3"), strip the chip suffix, gate snapshot on >= 4.3.0 (fail-open if unparseable). Request once per session per dongle, and re-request on /availability offline->online recovery. - coordinator: always process /status and /availability even during startup so the version/connection signals that drive the bootstrap aren't dropped. - coordinator: remove the premature blind snapshot publish from start_mqtt_subscription. Dongle replacement: new options-flow action "Replace a dongle" transfers all entities AND their history from an old dongle to a replacement. Rewrites the dongle segment of each entity's unique_id (and entity_id prefix) in place via the entity registry, so HA carries states + statistics across. Both dongles share the config entry's entry_id, so the rewrite is collision-free. The old dongle is swapped out of the entry, preserving its slot and role. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The /status-driven snapshot left an up-to-30s window where entities sat empty on a fresh connect, because /status is a periodic ~30s heartbeat with no on-demand request topic. Fire an eager snapshot request the moment we subscribe (fail-open, harmless on old firmware) so entities populate immediately; the /status and /availability paths remain as reconnect/reboot recovery triggers. Also detect silent reboots: a dongle that restarts without dropping its MQTT session never flips /availability, so its post-reboot change-data baseline would go unnoticed. Track boot.count from /status and force a fresh snapshot when it changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The dongle firmware reports batIndex == 0 for every battery in the /batteries payload (confirmed: a 2-battery payload has both entries at batIndex 0). Both the state-update loop and entity creation keyed on batIndex, so all batteries collided onto a single battery_0_* entity set — extra batteries were invisible and their values overwrote battery 0's. Key batteries by list position instead of the unreliable batIndex, on both the coordinator state-update side and the sensor creation side. Track created entities per (dongle, position) so the creation event can re-fire as the battery count grows without raising "unique id already registered" for existing ones. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Slice 1 changed entity objects to read their values via build_entity_id (which
drops the dongle prefix on single-dongle drop_dongle_id installs), but the
coordinator's data-WRITE paths still wrote to raw sensor.{formatted_dongle_id}_*
keys. Under drop_dongle_id on a single-dongle install the keys mismatched, so
entities would have shown no data — a regression.
Route all coordinator entity-state writes (entity creation, /input + /hold
values, status/uptime blob, fault/warning status, events, firmware update)
through build_entity_id so writers and readers agree. Remove the now-orphaned
formatted_dongle_id locals.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The rich /status payload (firmware version, chip type, MQTT/server connection states, memory, boot/crash counts, SD health, reset reason) was only dumped wholesale as attributes on the single Uptime sensor. Expose the useful fields as their own diagnostic sensors. - StatusFieldSensor: reads one (possibly nested, dotted-path) field out of the shared /status blob the coordinator stores at the dongle's _uptime entity_id — no new MQTT plumbing. Marked EntityCategory.DIAGNOSTIC. - STATUS_DIAGNOSTIC_SENSORS in const: health/connectivity, memory, and reliability-counter fields. Brand- and firmware-agnostic (every dongle publishes /status), so created for every dongle outside the per-brand table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the HTTP POST + WebSocket OTA flow with the dongle's MQTT /admin command
surface (firmware >= 4.3.0). async_install now publishes {"cmd":"ota"} to
<dongle_id>/admin, watches <dongle_id>/ota/progress, and finishes on
<dongle_id>/ota/result — no dongle IP, HTTP, or WebSocket involved.
- Waits for an ACK on <dongle_id>/admin/response before trusting the command;
surfaces a clear error if the dongle is offline or on pre-4.3.0 firmware.
- Tracks prod/beta via a use_beta_firmware config flag (CONF_USE_BETA).
- Strips the chip suffix (S3/C6) from the dongle-reported version so the
installed-vs-latest comparison matches the server's bare version (otherwise an
update always appears available).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The dongle IP is obsolete now that OTA and admin actions run over MQTT. Remove it entirely: all IP fields from every setup type, the "Update dongle IPs" options step, the dongle_ips entry data, and coordinator get_dongle_ip/_dongle_ips. This is a breaking change — firmware updates now require dongle FW >= 4.3.0. - config_flow: strip IP from single/parallel/single_gridboss/dual_gridboss setup and from add/remove/replace dongle options; delete async_step_update_dongles. - Add "Use Beta Firmware" toggle (CONF_USE_BETA) to options settings. - translations: drop all dead *_ip* keys and the update_dongles step strings. - Rewrite the 4.0.0 CHANGELOG to cover the full branch vs main (unified topics, setting/updated channel, Deye support, naming migration, dongle replacement, status diagnostics, OTA-over-MQTT, battery fix, snapshot fixes) and bump manifest to 4.0.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Since firmware 4.3.0 the dongle version is CI-stamped from the build, so HA no
longer specifies a target version in the OTA command — it just asks for the
current build on the prod or beta track. Remove the version arg from the
{"cmd":"ota"} envelope (the dongle picks the build by track). The async_install
version parameter remains in the signature per the UpdateEntity contract but is
unused. Chip-suffix stripping for the installed-vs-latest comparison is retained.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- coordinator: build_entity_id uses getattr for _drop_dongle_id so it's safe on a partially-constructed coordinator (this had broken 17/19 unit tests, which construct the coordinator via __new__). - coordinator: remove undefined `msg.topic` reference in process_status_message's invalid-JSON handler (NameError on any malformed /status payload). - coordinator: don't gate the post-/status coordinator push on _hass_startup_complete so diagnostic sensors populate on the first status. - update.py: ignore retained / out-of-range / wrong-id OTA progress messages so a stale retained progress can't jump the bar; accept "ok" as well as "success" on the OTA result topic. - sensor.py: only mark a battery position created once its entities are actually built, so a partial first /batteries payload doesn't permanently skip it. - sensor.py/const: StatusFieldSensor honours entity_registry_enabled_default; raw-byte heap/PSRAM diagnostics default disabled (recorder bloat) and use the DATA_SIZE device class. All 19 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- tests/test_session_helpers.py: 30 tests covering parse_fw_version, _needs_snapshot, chip-suffix stripping, build_entity_id (incl. the partially-constructed-coordinator regression case), migration._desired_entity_id, and nested /status field extraction. - Remove custom_components/monitormysolar/test.py — a stray dev script (Excel downloader using requests/magic/openpyxl, none of which are integration deps) that was shipping inside the published component. Full suite: 49 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The dongle publishes the snapshot reply on <dongle>/snap/input and <dongle>/snap/hold within ~1s of the request we send in start_mqtt_subscription. But that request fires during async_setup_entry, ~30s before _hass_startup_complete flips true. The dispatcher dropped every non-status/availability message while startup was pending (`elif not self._hass_startup_complete: pass`), so the snap reply was silently discarded and entities stayed empty until the next sparse change-data — which on FW >= 4.3.0 (change-data only) may never come for static values. Result: snap did nothing. Process snap/input and snap/hold unconditionally, like /status and /availability, since the snapshot IS the connect-time bootstrap. tests/test_snapshot_dispatch.py: 4 tests exercising the dispatcher directly with _hass_startup_complete False — snap routes during startup, plain /input stays gated, snap still routes after startup. Full suite: 53 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… paths. Fixes a bug where the new path would fail to unknown
…rposes to avoid entity duplication on update
…rposes to avoid entity duplication on update
…d everything else.
Feature/unified topics
|
Claude finished @zakery292's task in 2m 44s —— View job
Note: this is a large PR (46 files, +6225/-1856). Reviewing core modules (migration.py, coordinator.py, config_flow.py, entity platforms, mqtt handler) in parallel. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.