Skip to content

Implement a faster and more memory-efficient version of the flow decomposition algorithm. #186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jun 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/flow_decomposition/algorithm-description.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,9 @@ The following matrices are calculated using [sensitivity analysis](inv:powsyblco
- $\mathrm{PTDF}$ is the matrix of the sensitivity of the network element flow to each network injection shift,
- $\mathrm{PSDF}$ is the matrix of the sensitivity of the network element flow to each phase shift transformer tap angle change,

> **_NOTE:_** When *DIRECT_SENSITIVITY_BASED* flow partitioner is used, those matrices aren't calculated explicitly, and both sensitivity analysis and flow partitioning is done "at once".
> As a result, PTDF and PSDF matrices cannot be reported anymore.

## Flow partitioning

Based on previously calculated elements, flow partitioning can now be calculated as follows:
Expand Down
21 changes: 11 additions & 10 deletions docs/flow_decomposition/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,20 @@

## Dedicated parameters

| Name | Type | Default value | Description |
|-------------------------------------------|---------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| enable-losses-compensation | boolean | false | When set to true, adds losses compensation step of the algorithm. Otherwise, all losses will be compensated using chosen power flow compensation strategy. |
| losses-compensation-epsilon | double | 1e-5 | Threshold used in losses compensation step of the algorihm. If actual losses are below the given threshold on a branch, no injection is created in the network to compensate these losses. Used to avoid creating too many injections in the network. May have an impact in overall algorithm performance and memory usage. |
| sensitivity-epsilon | double | 1e-5 | Threshold used when filling PTDF and PSDF matrices. If a sensitivity is below the given threshold, it is set to zero. Used to keep sparse matrices in the algorithm. May have an impact in overall algorithm performance and memory usage. |
| rescale-mode | enum | NONE | Use NONE if you don't want to rescale flow decomposition results. Use ACER_METHODOLOGY for the ACER methodology rescaling strategy. Use PROPORTIONAL for a proportional rescaling. Use MAX_CURRENT_OVERLOAD for a rescaling based on AC current overloads. See [Flow parts rescaling](../flow_decomposition/algorithm-description.md#flow-parts-rescaling) for more details. |
| proportional-rescaler-min-flow-tolerance | double | 1e-6 | Option used from rescale modes PROPORTIONAL and MAX_CURRENT_OVERLOAD. Defines the minimum DC flow required in MW for the rescaling to happen. |
| dc-fallback-enabled-after-ac-divergence | boolean | true | Defines the fallback behavior after an AC divergence Use True to run DC loadflow if an AC loadflow diverges (default). Use False to throw an exception if an AC loadflow diverges. |
| sensitivity-variable-batch-size | int | 15000 | When set to a lower value, this parameter will reduce memory usage, but it might increase computation time |
| Name | Type | Default value | Description |
|------------------------------------------|---------|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| enable-losses-compensation | boolean | false | When set to true, adds losses compensation step of the algorithm. Otherwise, all losses will be compensated using chosen power flow compensation strategy. |
| losses-compensation-epsilon | double | 1e-5 | Threshold used in losses compensation step of the algorihm. If actual losses are below the given threshold on a branch, no injection is created in the network to compensate these losses. Used to avoid creating too many injections in the network. May have an impact in overall algorithm performance and memory usage. |
| sensitivity-epsilon | double | 1e-5 | Threshold used when filling PTDF and PSDF matrices. If a sensitivity is below the given threshold, it is set to zero. Used to keep sparse matrices in the algorithm. May have an impact in overall algorithm performance and memory usage. |
| rescale-mode | enum | NONE | Use NONE if you don't want to rescale flow decomposition results. Use ACER_METHODOLOGY for the ACER methodology rescaling strategy. Use PROPORTIONAL for a proportional rescaling. Use MAX_CURRENT_OVERLOAD for a rescaling based on AC current overloads. See [Flow parts rescaling](../flow_decomposition/algorithm-description.md#flow-parts-rescaling) for more details. |
| proportional-rescaler-min-flow-tolerance | double | 1e-6 | Option used from rescale modes PROPORTIONAL and MAX_CURRENT_OVERLOAD. Defines the minimum DC flow required in MW for the rescaling to happen. |
| dc-fallback-enabled-after-ac-divergence | boolean | true | Defines the fallback behavior after an AC divergence Use True to run DC loadflow if an AC loadflow diverges (default). Use False to throw an exception if an AC loadflow diverges. |
| sensitivity-variable-batch-size | int | 15000 | When set to a lower value, this parameter will reduce memory usage, but it might increase computation time. |
| flow-partitioner | enum | MATRIX_BASED | Use DIRECT_SENSITIVITY_BASED for better performance. However, nodal PTDF aren't explicitely calculated anymore and won't be reported. Use MATRIX_BASED if all detailed node PTDF needs to be reported. |

## Impact of existing parameters

Any implementation of load flow provider and sensitivity analysis provider can be used, as the entire algorithm only
relies on common loadflow API and sensitivity analysis API.

Thus, flow decomposition algorithm relies on [load flow parameters](inv:powsyblcore:*:*#loadflow-generic-parameters) and [sensitivity analysis parameters](inv:powsyblcore:*:*#sensitivity-generic-parameter).
Thus, the flow decomposition algorithm relies on [load flow parameters](inv:powsyblcore:*:*#loadflow-generic-parameters) and [sensitivity analysis parameters](inv:powsyblcore:*:*#sensitivity-generic-parameter).
5 changes: 0 additions & 5 deletions flow-decomposition/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,6 @@
<artifactId>jimfs</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-config-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-cgmes-conformity</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
import com.powsybl.iidm.network.Network;
import com.powsybl.loadflow.LoadFlowParameters;
import com.powsybl.sensitivity.*;
import org.jgrapht.alg.util.Pair;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand All @@ -25,15 +24,15 @@
/**
* @author Hugo Schindler {@literal <hugo.schindler at rte-france.com>}
*/
abstract class AbstractSensitivityAnalyser {
public abstract class AbstractSensitivityAnalyser {
public static final SensitivityFunctionType SENSITIVITY_FUNCTION_TYPE = SensitivityFunctionType.BRANCH_ACTIVE_POWER_1;
public static final List<Contingency> CONTINGENCIES = Collections.emptyList();
private static final Logger LOGGER = LoggerFactory.getLogger(AbstractSensitivityAnalyser.class);
private static final boolean DC_LOAD_FLOW = true;
protected final SensitivityAnalysisParameters sensitivityAnalysisParameters;
protected final SensitivityAnalysis.Runner runner;

AbstractSensitivityAnalyser(LoadFlowParameters loadFlowParameters, SensitivityAnalysis.Runner runner) {
protected AbstractSensitivityAnalyser(LoadFlowParameters loadFlowParameters, SensitivityAnalysis.Runner runner) {
this.sensitivityAnalysisParameters = initSensitivityAnalysisParameters(loadFlowParameters);
this.runner = runner;
}
Expand All @@ -51,21 +50,21 @@ private static LoadFlowParameters enforceDcLoadFlowCalculation(LoadFlowParameter
return dcEnforcedParameters;
}

protected List<Pair<String, String>> getFunctionVariableFactors(List<String> variableList,
protected List<FunctionVariableFactor> getFunctionVariableFactors(List<String> variableList,
List<Branch> functionList) {
List<Pair<String, String>> factors = new ArrayList<>();
List<FunctionVariableFactor> factors = new ArrayList<>();
variableList.forEach(
variable -> functionList.forEach(
function -> factors.add(Pair.of(function.getId(), variable))));
function -> factors.add(new FunctionVariableFactor(function.getId(), variable))));
return factors;
}

protected static SensitivityFactorReader getSensitivityFactorReader(List<Pair<String, String>> factors, SensitivityVariableType sensitivityVariableType, boolean sensitivityVariableSet) {
protected static SensitivityFactorReader getSensitivityFactorReader(List<FunctionVariableFactor> factors, SensitivityVariableType sensitivityVariableType, boolean sensitivityVariableSet) {
return handler -> factors.forEach(
pair -> handler.onFactor(SENSITIVITY_FUNCTION_TYPE,
pair.getFirst(),
pair.functionId(),
sensitivityVariableType,
pair.getSecond(),
pair.variableId(),
sensitivityVariableSet,
ContingencyContext.none()));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
*/
package com.powsybl.flow_decomposition;

import com.powsybl.iidm.network.Country;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;
import org.slf4j.Logger;
Expand Down Expand Up @@ -51,16 +52,16 @@ void export(Path dirPath, String basename, Map<String, DecomposedFlow> decompose
BufferedWriter writer = Files.newBufferedWriter(path, CHARSET);
CSVPrinter printer = new CSVPrinter(writer, FORMAT)
) {
Set<String> allLoopFlowKeys = aggregateAllLoopFlowKeys(decomposedFlowMap);
Set<Country> allLoopFlowKeys = aggregateAllLoopFlowKeys(decomposedFlowMap);
printHeaderRow(allLoopFlowKeys, printer);
printContentRows(decomposedFlowMap, allLoopFlowKeys, printer);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}

private Set<String> aggregateAllLoopFlowKeys(Map<String, DecomposedFlow> decomposedFlowMap) {
return decomposedFlowMap.values().stream().flatMap(decomposedFlow -> decomposedFlow.getLoopFlows().keySet().stream()).collect(Collectors.toSet());
private Set<Country> aggregateAllLoopFlowKeys(Map<String, DecomposedFlow> decomposedFlowMap) {
return decomposedFlowMap.values().stream().flatMap(decomposedFlow -> decomposedFlow.getFlowPartition().loopFlowPerCountry().keySet().stream()).collect(Collectors.toSet());
}

private void failSilentlyPrint(CSVPrinter printer, Object valueToPrint) {
Expand All @@ -79,28 +80,28 @@ private void failSilentlyPrintLn(CSVPrinter printer) {
}
}

private void printHeaderRow(Set<String> loopFlowKeys, CSVPrinter printer) {
private void printHeaderRow(Set<Country> loopFlowKeys, CSVPrinter printer) {
failSilentlyPrint(printer, EMPTY_CELL_VALUE);
failSilentlyPrint(printer, DecomposedFlow.ALLOCATED_COLUMN_NAME);
failSilentlyPrint(printer, DecomposedFlow.INTERNAL_COLUMN_NAME);
failSilentlyPrint(printer, DecomposedFlow.PST_COLUMN_NAME);
loopFlowKeys.stream().sorted().forEach(loopFlowKey -> failSilentlyPrint(printer, loopFlowKey));
loopFlowKeys.stream().sorted().forEach(loopFlowKey -> failSilentlyPrint(printer, NetworkUtil.getLoopFlowIdFromCountry(loopFlowKey)));
failSilentlyPrint(printer, DecomposedFlow.AC_REFERENCE_FLOW_1_COLUMN_NAME);
failSilentlyPrint(printer, DecomposedFlow.AC_REFERENCE_FLOW_2_COLUMN_NAME);
failSilentlyPrint(printer, DecomposedFlow.DC_REFERENCE_FLOW_COLUMN_NAME);
failSilentlyPrintLn(printer);
}

private void printContentRows(Map<String, DecomposedFlow> decomposedFlowMap, Set<String> allLoopFlowKeys, CSVPrinter printer) {
private void printContentRows(Map<String, DecomposedFlow> decomposedFlowMap, Set<Country> allLoopFlowKeys, CSVPrinter printer) {
decomposedFlowMap.forEach((xnecId, decomposedFlow) -> printContentRow(xnecId, decomposedFlow, allLoopFlowKeys, printer));
}

private void printContentRow(String xnecId, DecomposedFlow decomposedFlow, Set<String> allLoopFlowKeys, CSVPrinter printer) {
private void printContentRow(String xnecId, DecomposedFlow decomposedFlow, Set<Country> allLoopFlowKeys, CSVPrinter printer) {
failSilentlyPrint(printer, xnecId);
failSilentlyPrint(printer, decomposedFlow.getAllocatedFlow());
failSilentlyPrint(printer, decomposedFlow.getInternalFlow());
failSilentlyPrint(printer, decomposedFlow.getPstFlow());
allLoopFlowKeys.stream().sorted().forEach(loopFlowKey -> failSilentlyPrint(printer, decomposedFlow.getLoopFlows().getOrDefault(loopFlowKey, NO_FLOW)));
allLoopFlowKeys.stream().sorted().forEach(loopFlowKey -> failSilentlyPrint(printer, decomposedFlow.getFlowPartition().loopFlowPerCountry().getOrDefault(loopFlowKey, NO_FLOW)));
failSilentlyPrint(printer, decomposedFlow.getAcTerminal1ReferenceFlow());
failSilentlyPrint(printer, decomposedFlow.getAcTerminal2ReferenceFlow());
failSilentlyPrint(printer, decomposedFlow.getDcReferenceFlow());
Expand Down
Loading