Skip to content

Implement a faster and more memory-efficient version of the flow decomposition algorithm. #186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Implement a faster and more memory-efficient version of the flow decomposition algorithm. #186

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c18504f
Implement a faster and more memory-efficient version of the flow deco…
murgeyseb Apr 14, 2025
223df5d
Fix issue by per uniting sensitivity variable sets
murgeyseb Apr 30, 2025
dd73ec9
Remove config test
OpenSuze Apr 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/flow_decomposition/algorithm-description.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,9 @@ The following matrices are calculated using [sensitivity analysis](inv:powsyblco
- $\mathrm{PTDF}$ is the matrix of the sensitivity of the network element flow to each network injection shift,
- $\mathrm{PSDF}$ is the matrix of the sensitivity of the network element flow to each phase shift transformer tap angle change,

> **_NOTE:_** When fast mode is enabled, those matrices aren't calculated explicitely, and both sensitivity analysis and flow partitioning is done "at once".
> As a result, PTDF and PSDF matrices cannot be reported anymore.

## Flow partitioning

Based on previously calculated elements, flow partitioning can now be calculated as follows:
Expand Down
19 changes: 10 additions & 9 deletions docs/flow_decomposition/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@

## Dedicated parameters

| Name | Type | Default value | Description |
|-------------------------------------------|---------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| enable-losses-compensation | boolean | false | When set to true, adds losses compensation step of the algorithm. Otherwise, all losses will be compensated using chosen power flow compensation strategy. |
| losses-compensation-epsilon | double | 1e-5 | Threshold used in losses compensation step of the algorihm. If actual losses are below the given threshold on a branch, no injection is created in the network to compensate these losses. Used to avoid creating too many injections in the network. May have an impact in overall algorithm performance and memory usage. |
| sensitivity-epsilon | double | 1e-5 | Threshold used when filling PTDF and PSDF matrices. If a sensitivity is below the given threshold, it is set to zero. Used to keep sparse matrices in the algorithm. May have an impact in overall algorithm performance and memory usage. |
| rescale-mode | enum | NONE | Use NONE if you don't want to rescale flow decomposition results. Use ACER_METHODOLOGY for the ACER methodology rescaling strategy. Use PROPORTIONAL for a proportional rescaling. Use MAX_CURRENT_OVERLOAD for a rescaling based on AC current overloads. See [Flow parts rescaling](../flow_decomposition/algorithm-description.md#flow-parts-rescaling) for more details. |
| proportional-rescaler-min-flow-tolerance | double | 1e-6 | Option used from rescale modes PROPORTIONAL and MAX_CURRENT_OVERLOAD. Defines the minimum DC flow required in MW for the rescaling to happen. |
| dc-fallback-enabled-after-ac-divergence | boolean | true | Defines the fallback behavior after an AC divergence Use True to run DC loadflow if an AC loadflow diverges (default). Use False to throw an exception if an AC loadflow diverges. |
| sensitivity-variable-batch-size | int | 15000 | When set to a lower value, this parameter will reduce memory usage, but it might increase computation time |
| Name | Type | Default value | Description |
|-----------------------------------------|---------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| enable-losses-compensation | boolean | false | When set to true, adds losses compensation step of the algorithm. Otherwise, all losses will be compensated using chosen power flow compensation strategy. |
| losses-compensation-epsilon | double | 1e-5 | Threshold used in losses compensation step of the algorihm. If actual losses are below the given threshold on a branch, no injection is created in the network to compensate these losses. Used to avoid creating too many injections in the network. May have an impact in overall algorithm performance and memory usage. |
| sensitivity-epsilon | double | 1e-5 | Threshold used when filling PTDF and PSDF matrices. If a sensitivity is below the given threshold, it is set to zero. Used to keep sparse matrices in the algorithm. May have an impact in overall algorithm performance and memory usage. |
| rescale-mode | enum | NONE | Use NONE if you don't want to rescale flow decomposition results. Use ACER_METHODOLOGY for the ACER methodology rescaling strategy. Use PROPORTIONAL for a proportional rescaling. Use MAX_CURRENT_OVERLOAD for a rescaling based on AC current overloads. See [Flow parts rescaling](../flow_decomposition/algorithm-description.md#flow-parts-rescaling) for more details. |
| proportional-rescaler-min-flow-tolerance | double | 1e-6 | Option used from rescale modes PROPORTIONAL and MAX_CURRENT_OVERLOAD. Defines the minimum DC flow required in MW for the rescaling to happen. |
| dc-fallback-enabled-after-ac-divergence | boolean | true | Defines the fallback behavior after an AC divergence Use True to run DC loadflow if an AC loadflow diverges (default). Use False to throw an exception if an AC loadflow diverges. |
| sensitivity-variable-batch-size | int | 15000 | When set to a lower value, this parameter will reduce memory usage, but it might increase computation time |
| using-fast-mode | boolean | false | When set to true, enable direct computaton of flow decomposition by the loadflow engine without explicitely calculating PTDF and PSDF matrices. More efficient, but does not allow to retrieve intermediate PTDF and PSDF matrices for detailed reporting) |

## Impact of existing parameters

Expand Down
5 changes: 0 additions & 5 deletions flow-decomposition/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,6 @@
<artifactId>jimfs</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-config-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-cgmes-conformity</artifactId>
Expand Down
156 changes: 156 additions & 0 deletions ...composition/src/main/java/com/powsybl/flow_decomposition/FastModeSensitivityAnalyser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
/*
* Copyright (c) 2025, RTE (http://www.rte-france.com)
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package com.powsybl.flow_decomposition;

import com.powsybl.contingency.ContingencyContext;
import com.powsybl.iidm.network.Branch;
import com.powsybl.iidm.network.Network;
import com.powsybl.iidm.network.PhaseTapChanger;
import com.powsybl.iidm.network.PhaseTapChangerStep;
import com.powsybl.loadflow.LoadFlowParameters;
import com.powsybl.sensitivity.*;
import org.jgrapht.alg.util.Pair;

import java.util.*;

import static com.powsybl.flow_decomposition.DecomposedFlow.PST_COLUMN_NAME;

/**
* @author Sebastien Murgey {@literal <sebastien.murgey at rte-france.com>}
*/
public class FastModeSensitivityAnalyser extends AbstractSensitivityAnalyser {
private final Network network;
private final Set<Branch> xnecs;
private final Set<String> flowParts;
private final Map<String, Map<String, Double>> nodalInjectionPartitions;

FastModeSensitivityAnalyser(LoadFlowParameters loadFlowParameters,
SensitivityAnalysis.Runner runner,
Network network,
Set<Branch> xnecs,
SparseMatrixWithIndexesTriplet nodalInjectionsMatrix) {
super(loadFlowParameters, runner);
this.network = network;
this.xnecs = xnecs;
this.flowParts = nodalInjectionsMatrix.colIndex.keySet();
this.nodalInjectionPartitions = nodalInjectionsMatrix.toMap();
}

public Map<String, Map<String, Double>> run() {
List<SensitivityVariableSet> sensitivityVariableSets = new ArrayList<>();
List<Pair<String, String>> sensitivityFactors = new ArrayList<>();
Map<String, Double> nodalInjectionsPartitionSumByFlowPart = new HashMap<>();
for (String flowPart : flowParts) {
String positiveFlowPartName = getPositiveFlowPartName(flowPart);
double positiveFlowPartSum = nodalInjectionPartitions.values().stream().filter(stringDoubleMap -> stringDoubleMap.containsKey(flowPart) && stringDoubleMap.get(flowPart) > 0).mapToDouble(stringDoubleMap -> stringDoubleMap.get(flowPart)).sum();
String negativeFlowPartName = getNegativeFlowPartName(flowPart);
double negativeFlowPartSum = nodalInjectionPartitions.values().stream().filter(stringDoubleMap -> stringDoubleMap.containsKey(flowPart) && stringDoubleMap.get(flowPart) < 0).mapToDouble(stringDoubleMap -> stringDoubleMap.get(flowPart)).sum();
sensitivityVariableSets.add(new SensitivityVariableSet(
positiveFlowPartName,
nodalInjectionPartitions.entrySet().stream().filter(entry -> entry.getValue().containsKey(flowPart) && entry.getValue().get(flowPart) > 0).map(entry -> new WeightedSensitivityVariable(entry.getKey(), entry.getValue().get(flowPart) / positiveFlowPartSum)).toList()));
nodalInjectionsPartitionSumByFlowPart.put(positiveFlowPartName, positiveFlowPartSum);
sensitivityVariableSets.add(new SensitivityVariableSet(
negativeFlowPartName,
nodalInjectionPartitions.entrySet().stream().filter(entry -> entry.getValue().containsKey(flowPart) && entry.getValue().get(flowPart) < 0).map(entry -> new WeightedSensitivityVariable(entry.getKey(), entry.getValue().get(flowPart) / negativeFlowPartSum)).toList()));
nodalInjectionsPartitionSumByFlowPart.put(negativeFlowPartName, negativeFlowPartSum);
}

SensitivityFactorReader factorReader = new FastModeSensitivityFactorReader(flowParts, sensitivityFactors);
Map<String, Map<String, Double>> results = new HashMap<>();
SensitivityResultWriter valueWriter = new FastModeSensitivityResultWriter(sensitivityFactors, results, flowParts, nodalInjectionsPartitionSumByFlowPart);
runSensitivityAnalysis(network, factorReader, valueWriter, sensitivityVariableSets);
return results;
}

private static String getNegativeFlowPartName(String flowPart) {
return "Negative " + flowPart;
}

private static String getPositiveFlowPartName(String flowPart) {
return "Positive " + flowPart;
}

public static double respectFlowSignConvention(double ptdfValue, double referenceFlow) {
return referenceFlow < 0 ? -ptdfValue : ptdfValue;
}

private class FastModeSensitivityResultWriter implements SensitivityResultWriter {

private final List<Pair<String, String>> factors;
private final Map<String, Map<String, Double>> results;
private final Set<String> flowParts;
private final Map<String, Double> nodalInjectionsPartitionSumByFlowPart;

public FastModeSensitivityResultWriter(List<Pair<String, String>> factors, Map<String, Map<String, Double>> results, Set<String> flowParts, Map<String, Double> nodalInjectionsPartitionSumByFlowPart) {
this.factors = factors;
this.results = results;
this.flowParts = flowParts;
this.nodalInjectionsPartitionSumByFlowPart = nodalInjectionsPartitionSumByFlowPart;
}

@Override
public void writeSensitivityValue(int factorIndex, int contingencyIndex, double value, double functionReference) {
if (Double.isNaN(value)) {
return;
}
Pair<String, String> factor = factors.get(factorIndex);
Map<String, Double> flowDecomposition = results.computeIfAbsent(factor.getFirst(), s -> new HashMap<>());
for (String flowPart : flowParts) {
if (factor.getSecond().equals(getPositiveFlowPartName(flowPart))) {
double allocatedFlow = flowDecomposition.getOrDefault(flowPart, 0.0);
flowDecomposition.put(flowPart, allocatedFlow + respectFlowSignConvention(value * nodalInjectionsPartitionSumByFlowPart.get(getPositiveFlowPartName(flowPart)), functionReference));
return;
} else if (factor.getSecond().equals(getNegativeFlowPartName(flowPart))) {
double allocatedFlow = flowDecomposition.getOrDefault(flowPart, 0.0);
flowDecomposition.put(flowPart, allocatedFlow + respectFlowSignConvention(value * nodalInjectionsPartitionSumByFlowPart.get(getNegativeFlowPartName(flowPart)), functionReference));
return;
}
}

PhaseTapChanger phaseTapChanger = network.getTwoWindingsTransformer(factor.getSecond()).getPhaseTapChanger();
Optional<PhaseTapChangerStep> neutralStep = phaseTapChanger.getNeutralStep();
double deltaTap = 0.0;
if (neutralStep.isPresent()) {
deltaTap = phaseTapChanger.getCurrentStep().getAlpha() - neutralStep.get().getAlpha();
}
double pstFlow = flowDecomposition.getOrDefault(PST_COLUMN_NAME, 0.0);
flowDecomposition.put(PST_COLUMN_NAME, pstFlow + respectFlowSignConvention(deltaTap * value, functionReference));
}

@Override
public void writeContingencyStatus(int contingencyIndex, SensitivityAnalysisResult.Status status) {
// We do not manage contingency yet
}
}

private class FastModeSensitivityFactorReader implements SensitivityFactorReader {
private final Set<String> flowParts;
private final List<Pair<String, String>> factors;

public FastModeSensitivityFactorReader(Set<String> flowParts, List<Pair<String, String>> factors) {
this.flowParts = flowParts;
this.factors = factors;
}

@Override
public void read(Handler handler) {
for (Branch xnec : xnecs) {
for (String flowPart : flowParts) {
factors.add(Pair.of(xnec.getId(), getPositiveFlowPartName(flowPart)));
handler.onFactor(SENSITIVITY_FUNCTION_TYPE, xnec.getId(), SensitivityVariableType.INJECTION_ACTIVE_POWER, getPositiveFlowPartName(flowPart), true, ContingencyContext.none());
factors.add(Pair.of(xnec.getId(), getNegativeFlowPartName(flowPart)));
handler.onFactor(SENSITIVITY_FUNCTION_TYPE, xnec.getId(), SensitivityVariableType.INJECTION_ACTIVE_POWER, getNegativeFlowPartName(flowPart), true, ContingencyContext.none());
}

for (String pst : NetworkUtil.getPstIdList(network)) {
factors.add(Pair.of(xnec.getId(), pst));
handler.onFactor(SENSITIVITY_FUNCTION_TYPE, xnec.getId(), SensitivityVariableType.TRANSFORMER_PHASE, pst, false, ContingencyContext.none());
}
}
}
}
}
Loading