Merge origin/main: resolve conflicts with rule1 auto modes + negative-price strategies

Conflicts in __init__.py, coordinator.py, select.py resolved by
keeping both sides:
- Our branch: optimization_priority, block_export_on_negative_price,
  battery_cycle_cost_eur_kwh UI entities + upgrade defaults
- Main: charge_to_full_on_negative_price, discharge_to_make_room,
  rule1_time_window, rule1_weekday entities + defaults

block_export_on_negative_price retains our str→bool conversion
(tolerates legacy bool and new "on"/"off" select values).

158 tests pass (10 new from main).

https://claude.ai/code/session_01P9LAQ5ET6SU9dLWULxv2uF

Author: claude

CLAUDE.md

@@ -89,6 +89,36 @@ The integration controls the inverter via **Economic Rule 1** Modbus registers.
 2. Every 10 seconds, the coordinator checks if the current slot is in the schedule
 3. If the desired state differs from current state, it writes registers
 
+**Rule 1 registers always written by `_transition_to_state`**:
+`econ_rule_1_enable` (0/1/2), `_voltage`, `_soc`, `_power`, `_start_day`,
+`_stop_day`.
+
+**Rule 1 registers written only when auto mode is enabled** (via
+`_apply_rule1_auto_settings`, called every cycle, idempotent — only
+writes when the register already differs from the target):
+- `rule1_time_window=auto` → `econ_rule_1_start_time=00:00`,
+  `econ_rule_1_stop_time=23:59` (Felicity's 24-hour convention; the
+  firmware doesn't accept stop=00:00 or stop=24:00).
+- `rule1_weekday=auto` → `econ_rule_1_effective_week=0x7F` (all 7 days).
+
+Both default to `manual` so the integration doesn't touch user-set
+values unless they explicitly opt in.  When still on `manual` and the
+inverter's rule 1 window is restrictive, the inverter silently ignores
+the enable command outside the window — the EMS plans to act, writes
+the register, and nothing happens.  The `rule1_window_warning` check
+surfaces this in the EMS card.
+
+**Rule 1 window warning** (`coordinator._check_rule1_window_conflict`):
+runs every cycle.  Builds the set of intended action slots (auto mode:
+`scheduled_slots` + tomorrow; manual mode: today's remaining slots
+crossing the threshold in the active direction) and checks each against
+the rule 1 time-of-day window and effective-weekday mask read back from
+the inverter.  `start_time == stop_time` is treated as "all day"; a full
+0x7F mask as "all days".  Any mismatch is exposed as the
+`rule1_window_warning` attribute on `schedule_status` and rendered as a
+banner in the EMS card.  Weekday bit mapping: inverter bit0=Sunday..
+bit6=Saturday, mapped from Python via `isoweekday() % 7`.
+
 **Charge deferral** (`coordinator._determine_energy_state`): when the current
 slot is flagged `charge` but a later scheduled charge slot has a cheaper
 price (non-negative slots only), execution returns `idle` instead. The next
@@ -150,6 +180,8 @@ This prevents the SOC prediction from assuming unrealistic charge rates
 | battery_cycle_cost_eur_kwh | 0.0 | Wear cost; added to min sell price |
 | optimization_priority | "cost" | cost / longevity / self_consumption |
 | block_export_on_negative_price | True | Skip sell scheduling at p < 0 |
+| charge_to_full_on_negative_price | False | Schedule every p<0 slot (revenue at p<0); accepts forced PV curtailment |
+| discharge_to_make_room_for_negative_price | False | Pre-emptively discharge before p<0 PV windows so PV can fill the battery |
 
 The coordinator applies a SOH factor to nominal `battery_capacity_kwh`
 before constructing `EMSConfig` — ems.py treats the capacity as
@@ -267,6 +299,49 @@ max_today_slots = floor(headroom / effective_per_slot)
 # SOC validation prunes only when PV alone wouldn't fill the battery.
 ```
 
+### Negative-Price Strategies (charge_to_full / discharge_to_make_room)
+
+Two orthogonal opt-in flags that change how negative-price slots are
+handled.  Both are off by default and compose with all grid modes
+(from_grid, to_grid, both).
+
+**`charge_to_full_on_negative_price`** — acts *during* p<0 slots
+- In `_schedule_from_grid` / `_schedule_both`, after the normal
+  cheapest-slot selection, every negative-price slot in the remaining
+  window is added to the charge set (deduplicated).
+- `_validate_schedule_soc` is called with `keep_all_negative_charges=True`.
+  When set, negative-price slots are exempt from overflow pruning even
+  when PV alone wouldn't fill the battery (the legacy `pv_fills_battery`
+  exemption is broadened to "all negatives").
+- Phantom-charge slots (battery already at capacity entering the slot)
+  are kept too — the inverter may try to charge and the BMS will gate
+  it.  No harm; the schedule reflects user intent.
+- Trade-off: the user accepts some forced PV curtailment in exchange
+  for guaranteed revenue at every p<0 slot.
+
+**`discharge_to_make_room_for_negative_price`** — acts *before* p<0 slots
+- New helper `_select_discharges_for_pv_headroom` runs after charge
+  selection.  Walks the SOC trajectory forward; whenever a negative-
+  price slot with PV surplus would overflow the battery, schedules
+  discharge in the *most expensive* earlier positive-price slot to
+  create headroom.
+- Validity: SOC must never drop below the absolute `min_kwh` floor
+  (hardware safety), and end-of-day SOC must remain >= reserve_target
+  (overnight coverage).  Temporary dips below reserve during the day
+  are allowed — the negative-window PV will refill the battery.
+- Works in `from_grid` mode (which normally never discharges) as well
+  as `to_grid` / `both`.  In `both` mode, the make-room discharges
+  are merged with the regular sell-side selection (sells take
+  precedence on conflicting slots).
+
+**Composition**: when both flags are on, make-room discharges are
+scheduled before negative windows, then charge slots fire during the
+negatives.  Net effect: maximum profit on negative-price days
+(discharge at peak + buy at p<0 + PV fills the cleared battery).
+
+Exposed as off/on select entities (`HA_FelicitySpecialModeSelect`)
+in the UI.
+
 ---
 
 ## Safe Power Management (coordinator.py `_check_safe_power`, ~1307-1424)
@@ -341,6 +416,17 @@ Fetches `energy_state` history from HA API (throttled 60s), shows what actually
 | max_amperage_per_phase | number | 10-63 A | 16 | Grid current limit |
 | voltage_level | number | 48-60 V | 58 | Charge voltage setpoint |
 | discharge_min_voltage | number | 48-55 V | 50 | Discharge voltage floor |
+| charge_to_full_on_negative_price | select | off/on | off | Charge at every p<0 slot (revenue) |
+| discharge_to_make_room_for_negative_price | select | off/on | off | Pre-discharge before p<0 PV windows |
+| rule1_time_window | select | manual/auto | manual | Auto writes rule 1 start=00:00, stop=23:59 |
+| rule1_weekday | select | manual/auto | manual | Auto writes rule 1 effective_week=all days |
+
+All `HA_FelicityInternalNumber` configuration entities render as
+input boxes (`NumberMode.BOX`) so users can type precise values.
+Sliders are awkward for fractional / fine-grained settings like
+`efficiency_factor` (step 0.01) or `arbitrage_price_delta` (step
+0.01 €/kWh).  Pass `mode=NumberMode.SLIDER` to the constructor for
+entities that benefit from scrubbing.
 
 ### Key Sensor Entities
 
@@ -383,8 +469,22 @@ The confidence factor can drop to 0.1 early in the day on partially cloudy morni
 ### 4. Consumption Estimate Sensitivity
 The algorithm uses consumption_est/24 for hourly drain — assumes flat consumption. Houses with evening peaks (cooking, heating) may see under-predicted evening drain.
 
-### 5. Anti-Conflict Guard Only Checks Instantaneous Power
-The 200W grid import check (suppress discharge when importing) triggers on instantaneous reading. Short spikes (kettle, microwave) can briefly suppress profitable discharge.
+### 5. Anti-Conflict Guard Hysteresis — IMPLEMENTED
+Previously the 200W grid import check suppressed discharge on a single
+tick, causing flipper behaviour (discharge → idle → discharge every
+~16s) on short load spikes (kettle, microwave, EV start).  Two writes
+per flip is brutal on the inverter and customers notice.
+
+Now uses thresholded hysteresis:
+- Small/moderate import (200–2000W) must persist for ≥ 2 consecutive
+  cycles (≈ 32s) before suppression triggers.
+- Large import (> 2000W) suppresses immediately (genuine sustained
+  draw like EV charging or oven preheat).
+- After suppression ends, a 60-second cooldown blocks re-suppression
+  so the inverter stabilises before the next decision.
+- Each cycle now logs the grid_power + state decision at DEBUG level
+  so the flipper pattern is easy to spot in retrospect:
+  `State decision: desired=X, current=Y, soc=%, price=, threshold=, grid_power=W`
 
 ### 6. Generator-Port Solar Workaround
 TREX-25/50 with micro-inverters on the generator port need special handling. PV registers read 0, falling back to generator_day_cost_energy. Both backend and frontend handle this but it's fragile.
@@ -398,6 +498,17 @@ hour slot. This was especially painful at midnight when stale
 `state.state` still reports the previous day's stale total, the coordinator
 falls back to the filtered hourly sum.
 
+### 8. Midnight should not force inverter to idle
+The day-rollover block in `_async_update_data` used to unconditionally
+call `_transition_to_state("idle")` and then skip the normal cycle for
+that tick.  This cancelled valid charge/discharge actions that should
+continue across midnight (e.g., a customer selling overnight to clear
+the battery before negative-midday PV refills it next day).  The block
+now does only the bookkeeping (yesterday deficit, daily consumption,
+SOC history reset, slot overrides rotation) and falls through to the
+normal cycle, which re-determines the desired state and only writes a
+transition if the state actually changes.
+
 ---
 
 ## Algorithm Assessment and Improvement Recommendations

Installation description.pdf

@@ -6,6 +6,8 @@
 [![HACS validated](https://img.shields.io/badge/HACS-validated-41BDF5?style=flat-square)](https://github.com/hacs/integration)
 
 Felicity inverter home assistant integration for easy setup and use of the device (via [Modbus](https://www.se.com/us/en/faqs/FA168406/)).
+Additionally includes a full Energy Management System to Buy and Sell electricity on the best moments and guards maximum power use to save guard against overcurrent.
+With the right settings the software makes sure you pay the best energy prices or use no grid if not needed.
 
 For this integration to work you need to have a wired modbus connection to your inverter either [via this USB dongle](https://www.amazon.nl/Industrial-Converter-Lightningproof-Resettable-Protection/dp/B0B87YJLJQ?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A2FQD9ZIAONBLW) or via something [like this](https://www.kiwi-electronics.com/nl/rs485-to-rj45-ethernet-tcp-ip-to-serial-rail-mount-support-20109?country=NL&utm_term=20109&gad_source=1&gad_campaignid=19763718639&gbraid=0AAAAADuMvucKntnrNZrVkZAHDgps81zYC&gclid=Cj0KCQiAx8PKBhD1ARIsAKsmGbeFZaWC_S38eFyu1NtZ0SP4zyLWwMWG70BRz6Ur1nmBymMCxvSR1_kaAmR9EALw_wcB).
 Currently supports IVGM / TREX types: 
@@ -58,8 +60,10 @@ It supports modbus USB dongle and TCP [Modbus](https://www.se.com/us/en/faqs/FA1
 The 3 possible ways are explained in the picture below. At the moment the last part always requires a RS485 connection to the inverter.
 <p align="center">
   <img src="https://github.com/partach/ha_felicity/blob/main/pictures/HA-felicity-connect.png" width="600"/>
-  <br><em>Ways to connect the inverter</em>
+  <img src="https://github.com/partach/ha_felicity/blob/main/pictures/modbus_location_trex10k.png" width="400"/>
+  <br><em>Ways to connect the inverter and TREX10k modbus location</em>
 </p>
+NOTE: when using the USR-D164 wifi module you need to put Pack Interval to 100 (20 causes packet loss)
 
 ## Installation options
 The T-REX 5 and 10K series with HP or HL (High / Low Voltage batteries) with 1 or 3 Phases (P1 or P3) can be selected with selecting
@@ -83,7 +87,9 @@ This can be done via configuration when the intallation is succesfull (device fo
 ## Configuration
 After successfull install the integration can be configured at any time with a few settings. See picture on top for location of the gear icon
 - Update interval (the frequency of refresh of data). For the T-REX 5-10k models keep it on 10 sec minimum due to small baud rate.
-- Monetary override. Nordpool is supported by default but also other monetary integrations as Tibber. The format is that it needs a sensor with attributes about min, max, avg price
+- Setting up Nordpool (For energy prices). Use the HACS version, NOT the default version. (HACS version has 15 min slot information). You need to setup Nordpool for your energy supplier, see the web for examples.
+- Solar Forecast for today and tomorrow. Install an integration that predicts solar power. It should support a Today and Tomorrow sensor showing total expected amount (you need to configure the solar forecast right).
+- Monetary override. If you use Nordpool (recommended) leave this empty! Nordpool is supported by default but also other monetary integrations as Tibber. The format is that it needs a sensor with attributes about min, max, avg price
 If you want use Tibber enter in the override fied: `sensor.tibber_electricity_price` where electricity_price is the sensor with attributes (avg, min, max) and 'tibber' how you named the integration.
 The Felicity integration looks for a variaty of avg_price like fields as attributes and if it finds in the the override sensor, uses that as needed price information. If no information is found, 
 the price information remains unavailable. 
@@ -111,7 +117,7 @@ Note1:
 Note2: The Operating mode **must be set (by user) to Economic mode**. The Energy management feature will not engage in any other mode (Like General).
 
 During setup or with config setting (gear symbol in hub/device overview) you can add a 'Monetary' Home Assistant Device.
-Examples are the Nordpool integration or Tibber. Look at the Nordpool integration details on how to set that up (not covered here).
+Examples are the Nordpool integration (HACS version only, not default) or Another. Look at the HACS Nordpool integration details on how to set that up (not covered here).
 During first setup or during run-time configuration (device gear symbol) it will display a list of installed Monetary integrations to chose from.
 Currently Nordpool and Tibber (via Norpool override field in config) are tested to work.
 
@@ -128,30 +134,33 @@ Example: Max price = 0.30 Euro, Min Price = 0.20 Euro and Avergage Price = 0.25
 When setting the `Price Threshold Level to 5` the Base-Threshold-Price will be 0.25.
 
 **The Grid Mode setting**:
- * If `Grid Mode` <em>(From-grid, To-Grid, Off)</em> is set to From-grid it will allow use of grid power when actual price is <=0.25 Euro
- * If `Grid Mode` <em>(From-grid, To-Grid, Off)</em> is set to To-grid it will allow Battery power to go to grid power when actual price is >=0.25 Euro
+ * If `Grid Mode` <em>(From-grid, To-Grid, Both, Off)</em> is set to From-grid it will allow use of grid power when actual price is <=0.25 Euro (as per given example)
+ * If `Grid Mode` <em>(From-grid, To-Grid, Both, Off)</em> is set to To-grid it will allow Battery power to go to grid power when actual price is >=0.25 Euro (as per given example)
 **Additional variables** are `Battery Charge Max Level` and `Battery Charge Min Level`.
  * In `From Grid mode` it will stop when `Actual Battery Capacity` reaches `Battery Charge Max Level`
  * In `To Grid mode` it will stop when `Actual Battery Capacity` reaches `Battery Charge Min Level`
+ * In `Both` it will sell and charge the battery optimally to make the least amount of cost / use the grid as less as possible
 
-IMPORTANT: The integration is depedent on the Monetary Integration to contiously supply the data.
+IMPORTANT: The integration is depedent on the Monetary Integration and Solar Forecast Integration to contiously supply the data.
 
 ## Dynamic Power Management
 The integration also supports Dynamic Power Management. After instalation, via configuration entities (see above picture), you can set the maximum amperage of your home electricity setup.
 For example if you have a maximum of 16A per group, set the value to 16A. The integration will then make sure the battery loading will be dialed back if the amperage becomes to high.
 (by decreasing the user requested power level, controlled via rule 1 via the integration).
 It will keep monitorning this and will increase the battery loading to requested power levels if the amperage becomes lower.
 
-## Using the card
+## Using the cards
 After installation of the integration you need to first reboot HA.
-The card will be automatically installed and registered by the integration on start up.
+The cards will be automatically installed and registered by the integration on start up.
 To use the card in your dashboard, go to you dashboard, edit, choose `Add card`.
-Choose `Manual`
-Add first line: `type: custom:felicity-inverter-card`
+They can be found at the bottom of the list.
+If they are not visible you can choose `Manual` as card type.
+Add first line: `type: custom:felicity-inverter-card` for the inverter card and `type: custom:felicity-ems-card` for the EMS card.
 Then choose the `visual editor` to continue.
-From the `Device` dropdown chose your felicity inverter install.
+From the `Device` dropdown chose your felicity inverter integration installed.
 <p align="center">
   <img src="https://github.com/partach/ha_felicity/blob/main/pictures/HA-felicity-card-expl.png" width="600"/>
+  <img src="https://github.com/partach/ha_felicity/blob/main/pictures/ems_ui_explanation.png" width="600"/>
   <br><em>Card usage explained</em>
 </p>
 

README.md

@@ -197,6 +197,10 @@ async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
         "battery_cycle_cost_eur_kwh": 0.0,
         "optimization_priority": "cost",
         "block_export_on_negative_price": "on",
+        "charge_to_full_on_negative_price": "off",
+        "discharge_to_make_room_for_negative_price": "off",
+        "rule1_time_window": "manual",
+        "rule1_weekday": "manual",
         CONF_REGISTER_SET: DEFAULT_REGISTER_SET,
         "update_interval": 10,
     }