Skip to content

Adds support for deferrable load groups#784

Merged
davidusb-geek merged 1 commit into
davidusb-geek:masterfrom
JosephSalisbury:deferrable_load_groups
Apr 19, 2026
Merged

Adds support for deferrable load groups#784
davidusb-geek merged 1 commit into
davidusb-geek:masterfrom
JosephSalisbury:deferrable_load_groups

Conversation

@JosephSalisbury
Copy link
Copy Markdown
Contributor

@JosephSalisbury JosephSalisbury commented Apr 11, 2026

Towards #686

Add support for deferrable load groups, using the configuration suggested in #686 (comment).

Summary by Sourcery

Add configuration-driven support for grouping deferrable loads with shared constraints in the optimization model.

New Features:

  • Introduce deferrable load groups configuration allowing grouped loads to share a power budget and/or be mutually exclusive during optimization.

Enhancements:

  • Extend optimization constraints to enforce shared maximum power and mutual exclusion within configured deferrable load groups, including relaxed mode support.
  • Include deferrable load groups in configuration hashing to ensure cache validity when group settings change.

Documentation:

  • Document the new deferrable_load_groups configuration, its fields, constraints, and usage examples in the configuration guide.

Tests:

  • Add optimization and configuration validation tests covering shared power limits, mutual exclusion behavior, empty groups, invalid group definitions, and overlapping group membership.

@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented Apr 11, 2026

Reviewer's Guide

Adds configuration-driven deferrable load grouping, enforcing shared power budgets and optional mutual exclusion in the optimization, with validation, cache-key integration, documentation, and tests.

Updated class diagram for optimization and cache key deferrable load grouping

classDiagram
    class Optimization {
        +dict optim_conf
        +dict vars
        +logger logger
        +_add_deferrable_load_constraints(constraints)
        +_add_deferrable_group_constraints(constraints)
        +_build_results_dataframe(data_opt, ...)
        +_build_objective_function(batt_stress_conf, inv_stress_conf)
    }

    class OptimizationCacheKey {
        +tuple set_deferrable_load_as_timeseries
        +tuple nominal_power_of_deferrable_loads
        +tuple def_load_config_structure
        +tuple deferrable_load_groups
        +bool inverter_is_hybrid
        +bool compute_curtailment
        +float optimization_time_step_s
    }

    class CommandLineConfig {
        +dict optim_conf
        +dict plant_conf
        +dict retrieve_hass_conf
    }

    class ConfigHashBuilder {
        +config_hash(cfg, exclude_keys) str
    }

    OptimizationCacheKey <.. ConfigHashBuilder : builds
    CommandLineConfig <.. ConfigHashBuilder : reads

    class DeferrableLoadGroupConfig {
        +tuple names
        +float max_power
        +bool mutual_exclusion
    }

    Optimization "1" --> "*" DeferrableLoadGroupConfig : uses
    OptimizationCacheKey "1" --> "*" DeferrableLoadGroupConfig : encoded_as_tuple_from
Loading

Flow diagram for deferrable_load_groups validation in build_params

flowchart TD
    Start["Start build_params validation for deferrable_load_groups"] --> GetGroups["Read optim_conf.deferrable_load_groups as groups"]
    GetGroups --> HasGroups{groups is empty?}
    HasGroups -- Yes --> End["Skip group validation"]
    HasGroups -- No --> InitSeen["Initialize seen_indices as empty set"]

    InitSeen --> LoopGroups["For each group gi, group in groups"]

    LoopGroups --> GetNames["names = group.names or []"]
    GetNames --> NamesEmpty{names empty?}
    NamesEmpty -- Yes --> ErrNames["Raise ValueError: names must contain at least 1 deferrable load reference"]
    NamesEmpty -- No --> InitIndices["indices = empty list"]

    InitIndices --> LoopNames["For each name in names"]
    LoopNames --> ParseIdx["Parse idx from name by stripping prefix 'deferrable'"]
    ParseIdx --> ParseOk{parse succeeded?}
    ParseOk -- No --> ErrParse["Raise ValueError: could not parse index from name"]
    ParseOk -- Yes --> CheckRange{idx >= num_def_loads?}
    CheckRange -- Yes --> ErrRange["Raise ValueError: index out of range for configured deferrable loads"]
    CheckRange -- No --> CheckSeen{idx in seen_indices?}
    CheckSeen -- Yes --> ErrSeen["Raise ValueError: load already in another group"]
    CheckSeen -- No --> AddIdx["Append idx to indices"]
    AddIdx --> MoreNames{more names?}
    MoreNames -- Yes --> LoopNames
    MoreNames -- No --> UpdateSeen["seen_indices.update(indices)"]

    UpdateSeen --> ReadPower["max_power = group.max_power"]
    ReadPower --> ReadMutual["mutual_exclusion = bool(group.mutual_exclusion, default False)"]

    ReadMutual --> CheckMaxVal{max_power is not None and max_power <= 0?}
    CheckMaxVal -- Yes --> ErrMaxVal["Raise ValueError: max_power must be positive"]
    CheckMaxVal -- No --> CheckMutualType{mutual_exclusion is bool?}
    CheckMutualType -- No --> ErrMutualType["Raise ValueError: mutual_exclusion must be boolean"]
    CheckMutualType -- Yes --> CheckMaxRequired{max_power is None and mutual_exclusion is False?}
    CheckMaxRequired -- Yes --> ErrMaxReq["Raise ValueError: max_power required when mutual_exclusion is false"]
    CheckMaxRequired -- No --> CheckMutualEnabled{mutual_exclusion is True?}

    CheckMutualEnabled -- No --> NextGroup{more groups?}
    CheckMutualEnabled -- Yes --> LoadSemiCont["semi_cont = optim_conf.treat_deferrable_load_as_semi_cont or []"]

    LoadSemiCont --> LoopIdx["For each idx in indices"]
    LoopIdx --> SemiCheck{"idx < len(semi_cont) and semi_cont[idx] is False?"}
    SemiCheck -- Yes --> ErrSemi["Raise ValueError: mutual_exclusion requires treat_deferrable_load_as_semi_cont=true"]
    SemiCheck -- No --> MoreIdx{more indices?}
    MoreIdx -- Yes --> LoopIdx
    MoreIdx -- No --> NextGroup

    NextGroup -- Yes --> LoopGroups
    NextGroup -- No --> End["Validation complete"]
Loading

File-Level Changes

Change Details Files
Introduce configuration validation for deferrable load groups to ensure well-formed group definitions and compatibility with existing deferrable load settings.
  • Read deferrable_load_groups from optim_conf during parameter building and default to an empty list when absent
  • Validate each group has at least one name and that each name parses to a valid deferrable index within the configured number_of_deferrable_loads
  • Prevent any deferrable load from belonging to more than one group by tracking used indices across groups
  • Enforce that max_power, when provided, is positive and that it is required unless mutual_exclusion is true
  • Require mutual_exclusion to be a boolean and, when true, ensure the referenced loads are configured as semi-continuous via treat_deferrable_load_as_semi_cont
src/emhass/utils.py
Add optimization constraints implementing shared power budget and mutual exclusion behavior for deferrable load groups, including relaxed-mode handling.
  • Introduce _add_deferrable_group_constraints helper on the optimization class to encapsulate group constraint construction
  • For each configured group, derive deferrable load indices from configured names and log the group being processed
  • Add per-timestep shared power budget constraints by bounding the sum of p_deferrable variables in each group by the group max_power when defined
  • Add per-timestep mutual exclusion constraints by bounding the sum of p_def_bin2 binaries in each group to at most 1 when mutual_exclusion is enabled
  • Invoke _add_deferrable_group_constraints in both the main and relaxed optimization model builds, wiring it to the appropriate constraints list
src/emhass/optimization.py
Ensure deferrable load group configuration participates in the optimization cache key so cache entries remain valid when group definitions change.
  • Extend OptimizationCacheKey dataclass to carry a deferrable_load_groups field
  • Populate deferrable_load_groups in config_hash by normalizing each group to a tuple of (names tuple, max_power, mutual_exclusion) from optim_conf
  • Include the normalized group tuple in the overall cache-key computation alongside existing deferrable load structure fields
src/emhass/command_line.py
Document the new deferrable_load_groups configuration and validate behavior with targeted tests.
  • Extend configuration docs with a deferrable_load_groups section describing purpose, schema, constraints, and multiple JSON examples
  • Add optimization tests for shared group power budget, mutual exclusion operation, and backward compatibility when deferrable_load_groups is empty
  • Add async helper to build params with injected deferrable_load_groups and config overrides for validation testing
  • Add async tests that assert ValueError for invalid deferrable load names, mutual_exclusion on non-semi-continuous loads, and overlapping group memberships
docs/config.md
tests/test_optimization.py

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@JosephSalisbury JosephSalisbury force-pushed the deferrable_load_groups branch from 9d6db45 to f62a4c9 Compare April 11, 2026 16:53
@JosephSalisbury JosephSalisbury marked this pull request as ready for review April 11, 2026 16:55
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 3 issues, and left some high level feedback:

  • In _add_deferrable_group_constraints you always add both shared-power and mutual-exclusion constraints, but in the relaxed optimization path the comment says "shared power budget only in relaxed mode"; consider either splitting the helper or passing a flag so that mutual exclusion is not enforced in the relaxed model (and avoid depending on p_def_bin2 there).
  • The deferrable group validation in build_params could be tightened: currently duplicate names within a single group and loads with indices beyond the length of treat_deferrable_load_as_semi_cont silently pass some checks; consider explicitly rejecting duplicates within the same group, validating that semi_cont is defined for all referenced indices, and using a stricter name parser (e.g. enforcing the deferrable prefix and non-negative integer suffix).
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `_add_deferrable_group_constraints` you always add both shared-power and mutual-exclusion constraints, but in the relaxed optimization path the comment says "shared power budget only in relaxed mode"; consider either splitting the helper or passing a flag so that mutual exclusion is not enforced in the relaxed model (and avoid depending on `p_def_bin2` there).
- The deferrable group validation in `build_params` could be tightened: currently duplicate names within a single group and loads with indices beyond the length of `treat_deferrable_load_as_semi_cont` silently pass some checks; consider explicitly rejecting duplicates within the same group, validating that `semi_cont` is defined for all referenced indices, and using a stricter name parser (e.g. enforcing the `deferrable` prefix and non-negative integer suffix).

## Individual Comments

### Comment 1
<location path="src/emhass/utils.py" line_range="2204" />
<code_context>
+                        raise ValueError(
+                            f"deferrable_load_groups[{gi}]: could not parse index from name '{name}'"
+                        )
+                    if idx >= num_def_loads:
+                        raise ValueError(
+                            f"deferrable_load_groups[{gi}]: '{name}' references index {idx}, "
</code_context>
<issue_to_address>
**issue (bug_risk):** Also validate that parsed indices are non-negative to avoid unintended negative indexing

Currently only `idx >= num_def_loads` is checked. A name like `deferrable-1` would parse to `-1`, pass this check, and then be used as a valid negative index (e.g., into `nominal_power_of_deferrable_loads`), silently referencing the last element instead of failing. Please also reject negative indices, e.g. `if idx < 0 or idx >= num_def_loads:`.
</issue_to_address>

### Comment 2
<location path="src/emhass/optimization.py" line_range="2007-2016" />
<code_context>
+    def _add_deferrable_group_constraints(self, constraints):
</code_context>
<issue_to_address>
**issue (bug_risk):** Mutual exclusion logic here is also applied in relaxed mode, which conflicts with the intended "shared power budget only" behavior

This helper always enforces both shared power (`max_power`) and mutual exclusion:

- It unconditionally reads `p_def_bin2` and adds `bin_sum <= 1` when `mutual_exclusion` is true.
- The same helper is then used for `constraints_relaxed`, which is documented as "shared power budget only".

If the relaxed formulation omits `p_def_bin2`, this will either break (missing var) or implicitly introduce binary-style constraints where only the shared power budget is intended.

Consider either:
- Adding a flag (e.g. `enforce_mutual_exclusion`) to `_add_deferrable_group_constraints` and disabling it for the relaxed call, or
- Splitting into two helpers (shared power vs. mutual exclusion) and using only the shared-power helper in relaxed mode.

This would make the behavior match the comment and avoid relying on `p_def_bin2` in the relaxed problem.
</issue_to_address>

### Comment 3
<location path="src/emhass/utils.py" line_range="2232-2193" />
<code_context>
+                constraints.append(group_power_sum <= max_power)
+
+            # Mutual exclusion: at most one load active per timestep
+            if mutual_exclusion:
+                bin_sum = sum(p_def_bin2[i] for i in indices)
+                constraints.append(bin_sum <= 1)
</code_context>
<issue_to_address>
**suggestion:** Mutual exclusion relies on `treat_deferrable_load_as_semi_cont` length, which might silently bypass the check for some indices

When `mutual_exclusion` is enabled, some indices may be assumed semi-continuous without ever being checked if `treat_deferrable_load_as_semi_cont` is shorter than `num_def_loads`. That makes misconfiguration easy to miss. Consider either enforcing `len(treat_deferrable_load_as_semi_cont) >= num_def_loads` when mutual exclusion is used, or treating missing entries as `False` and raising explicitly, so incomplete configs don’t slip through silently.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread src/emhass/utils.py Outdated
Comment thread src/emhass/optimization.py Outdated
Comment thread src/emhass/utils.py
@JosephSalisbury JosephSalisbury force-pushed the deferrable_load_groups branch from f62a4c9 to a432ad9 Compare April 11, 2026 17:02
@JosephSalisbury
Copy link
Copy Markdown
Contributor Author

@davidusb-geek i hit the issue described in #686, so i thought i'd attempt to solve it

i'm not familiar with the project / codebase, so this is all from claude, but it appears to work correctly in my setup:

Screenshot 2026-04-11 at 18 15 09

(deferrable 0 and 1 being a heat pump central heating / hot water modelled as thermal loads, they're not scheduled at the same time)

give a shout if there's anything you'd like to see changed / added etc. thanks!

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 19, 2026

Codecov Report

❌ Patch coverage is 87.71930% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.88%. Comparing base (6c6faf3) to head (a432ad9).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
src/emhass/utils.py 81.08% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #784      +/-   ##
==========================================
+ Coverage   81.82%   81.88%   +0.05%     
==========================================
  Files          10       10              
  Lines        5739     5796      +57     
==========================================
+ Hits         4696     4746      +50     
- Misses       1043     1050       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@davidusb-geek
Copy link
Copy Markdown
Owner

@davidusb-geek i hit the issue described in #686, so i thought i'd attempt to solve it

i'm not familiar with the project / codebase, so this is all from claude, but it appears to work correctly in my setup:

give a shout if there's anything you'd like to see changed / added etc. thanks!

@JosephSalisbury, thanks for contributing and directly to solve an open feature request!
Your PR is quite complete, including a unittest (which is passing fine) and documentation.
There are some minor code format issues that I will deal with myself.
In the future you can deal with these automatically by using the ruff formatter extension in VSCode.
Merging...

@davidusb-geek davidusb-geek merged commit 8a11bc5 into davidusb-geek:master Apr 19, 2026
17 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants