Document #451

- Improve RELEASE_NOTES text.
- Fix and add internal references and reference targets.
- Document cap_comm option to Scenario.solve(), MESSAGE(), etc.

Author: khaeru

RELEASE_NOTES.rst

@@ -4,12 +4,21 @@ Next release
 All changes
 -----------
 
-- Extend the auxiliary variable :ref:`_commodity_balance` to include input and output flows based on CAP and CAP_NEW variables (:pull:`451`).
-
-  When loading a Scenario created with `message_ix` version 3.11.0 or earlier,
-  these items will be initialized (and left empty),
-  using at most one call to :meth:`~message_ix.Scenario.commit`.
-  See :meth:`.MESSAGE.initialize`.
+- Add representation of commodity flows associated with construction and retirement of technology capacity (:pull:`451`).
+
+  - New parameters
+    |input_cap|,
+    |input_cap_new|,
+    |input_cap_ret|,
+    |output_cap|,
+    |output_cap_new|, and
+    |output_cap_ret|.
+    When loading a Scenario created with :mod:`message_ix` version 3.11.0 or earlier,
+    these items are automatically initialized (and left empty),
+    using at most one call to :meth:`~message_ix.Scenario.commit`.
+    See :meth:`.MESSAGE.initialize`.
+  - Revise :ref:`equation_commodity_balance_aux` to include input and output flows based on |CAP| and |CAP_NEW|.
+  - New :class:`.MESSAGE` / :meth:`.Scenario.solve` option :py:`cap_comm=True` to enable this representation.
 
 - Document the :ref:`minimum version of Java <install-java>` required for :class:`ixmp.JDBCBackend <ixmp.backend.jdbc.JDBCBackend>` (:pull:`962`).
 - Improve type hinting (:pull:`963`).

doc/api.rst

@@ -249,6 +249,18 @@ Model classes
    :exclude-members: defaults
    :show-inheritance:
 
+   MESSAGE accepts the following *model_options* to control its behaviour:
+
+   - **cap_comm** (:class:`bool` or equivalent :class:`int` or :class:`float`):
+     If :any:`True`, include **comm** odity flows associated with **cap** acity changes of technologies
+     in :ref:`equation_commodity_balance_aux`.
+     If :any:`False` (the default), these flow are omitted,
+     and values of the associated parameters have no effect.
+     This corresponds to the GAMS compile-time variable ``MESSAGE_CAP_COMM``.
+
+     .. note:: For some models, this can significantly increase the linear program (LP) size
+        and thus the solve time.
+
    .. autoattribute:: items
       :no-value:
 
@@ -263,10 +275,10 @@ Model classes
    The MACRO class solves only the MACRO model in “standalone” mode—that is, without MESSAGE.
    It is also invoked from :class:`.MESSAGE_MACRO` to process *model_options* to control the behaviour of MACRO:
 
-   - **concurrent** (:class:`int` or :class:`float`, either :py:`0` or :py:`1`).
+   - **concurrent** (:class:`bool` or equivalent :class:`int` or :class:`float`).
      This corresponds to the GAMS compile-time variable ``MACRO_CONCURRENT``.
-     If set to :py:`0` (the default), MACRO is solved in a loop, once for each node in the Scenario.
-     If set to :py:`1`, MACRO is solved only once, for all nodes simultaneously.
+     If :any:`False` (the default), MACRO is solved in a loop, once for each node in the Scenario.
+     If :any:`True`, MACRO is solved only once, for all nodes simultaneously.
 
    .. autoattribute:: items
       :no-value:
@@ -279,7 +291,8 @@ Model classes
    MESSAGE_MACRO solves the MESSAGE and MACRO models iteratively, connecting changes in technology activity and resource demands (from MESSAGE) to changes in final demands and prices (from MACRO).
    This iteration continues until the solution *converges*; i.e. the two models reach a stable point for the values of these parameters.
 
-   MESSAGE_MACRO accepts three additional *model_options* that control the behaviour of this iteration algorithm:
+   MESSAGE_MACRO accepts all of the *model_options* recognized by :class:`.MESSAGE` and :class:`.MACRO`,
+   plus additional options that control the behaviour of this iteration algorithm:
 
    - **max_adjustment** (:class:`float`, default 0.2): the maximum absolute relative change in final demands between iterations.
      If MACRO returns demands that have changed by more than a factor outside the range (1 - `max_adjustment`, 1 + `max_adjustment`) since the previous iteration, then the change is confined to the limits of that range for the next run of MESSAGE.

doc/conf.py

@@ -67,6 +67,7 @@
 
    <abbr title="International Institute for Applied Systems Analysis">IIASA</abbr>
 
+.. |y0| replace:: :math:`y_0`
 .. |yA| replace:: :math:`y^A`
 .. |yV| replace:: :math:`y^V`
 """
@@ -77,13 +78,29 @@
 # sometimes multiple targets appear above a single heading, and that heading text would
 # be automatically used for the link text.
 for name in (
+    "ACT",
+    "CAP",
+    "CAP_NEW",
+    "LAND",
+    "STOCK_CHG",
     "duration_period",
     "duration_period_sum",
     "growth_new_capacity_up",
     "historical_new_capacity",
     "initial_new_capacity_up",
+    "input",
+    "input_cap",
+    "input_cap_new",
+    "input_cap_ret",
+    "output",
+    "output_cap",
+    "output_cap_new",
+    "output_cap_ret",
+    "land_input",
+    "land_output",
     "map_tec_lifetime",
     "remaining_capacity",
+    "technical_lifetime",
 ):
     rst_prolog += f"\n.. |{name}| replace:: :ref:`{name} <{name}>`"
 
@@ -222,8 +239,9 @@ def local_inv(name: str, *parts: str) -> Optional[str]:
 macros = {}
 macros.update(
     {
-        k.replace("_", ""): r"\text{{{k}}}"
+        k.replace("_", ""): rf"\text{{{k}}}"
         for k in """ACT
+COMMODITY_BALANCE
 STORAGE
 STORAGE_CHARGE
 duration_time_rel

message_ix/core.py

@@ -755,7 +755,8 @@ def solve(self, model="MESSAGE", solve_options={}, **kwargs):
             See the :class:`.MESSAGE` class and :obj:`.DEFAULT_CPLEX_OPTIONS`.
         kwargs :
             Other options control the execution of the underlying GAMS code; see the
-            :class:`.MESSAGE_MACRO` class and :class:`.GAMSModel`.
+            *model_options* described for :class:`.MESSAGE`, :class:`.MACRO`,
+            :class:`.MESSAGE_MACRO`, and :class:`.GAMSModel`.
         """
         super().solve(model=model, solve_options=solve_options, **kwargs)
 

message_ix/model/MESSAGE/model_core.gms

@@ -28,6 +28,13 @@
 *
 * Decision variables
 * ^^^^^^^^^^^^^^^^^^
+*
+* .. _ACT:
+* .. _CAP:
+* .. _CAP_NEW:
+* .. _LAND:
+* .. _STOCK_CHG:
+*
 * =============================================================== ====================================================================================
 * Variable                                                        Explanatory text
 * =============================================================== ====================================================================================
@@ -554,19 +561,25 @@ RESOURCE_HORIZON(node,commodity,grade)$( SUM(year$map_resource(node,commodity,gr
 * Constraints on commodities and stocks
 * ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 *
-* .. _commodity_balance:
+* .. _equation_commodity_balance_aux:
+*
+* Equation COMMODITY_BALANCE_AUX
+* """"""""""""""""""""""""""""""
+*
+* This equation sets the value of variable :math:`\COMMODITYBALANCE`,
+* which is then constrained by :ref:`commodity_balance_gt` and :ref:`commodity_balance_lt`.
+*
+* For each :math:`(n, c, l, y, h)`, this variable includes:
 *
-* Auxiliary COMMODITY_BALANCE
-* """""""""""""""""""""""""""
-* For the commodity balance constraints below, we introduce an auxiliary variable called :math:`COMMODITY\_BALANCE`.
-* This is implemented as a GAMS ``$macro`` function.
-* This variable also includes the material flows upon construction and retirement of the technology capacities.
-* For the material flows, the formulation is valid for the model years that are different than the first model year.
-* The formulation for the first model year requires the use of the ``historical_new_capacity`` parameter
-* instead of the ``CAP`` variable,
-* and depends on conditions for ``remaining_capacity`` to determine
-* whether the end of a technology's technical lifetime falls within the duration of a model period
-* (|duration_period|) or not.
+* - Net |input| minus |output| of commodities based on technology activity (|ACT|).
+* - Net |land_input| minus |land_output| of commodities based on |LAND|.
+* - Inter-period transfers via |STOCK_CHG|.
+* - If the :class:`MESSAGE` option :py:`cap_comm=True` is given,
+*   flows of commodities (e.g. ‘materials’)
+*   associated with construction and retirement of technology capacity (|CAP|).
+*   For |y0|, this representation requires |historical_new_capacity| parameter values,
+*   and depends on conditions for |remaining_capacity|, |duration_period|, and |technical_lifetime|
+*   to determine when a technology is retired and account for the relevant flows.
 *
 *  .. math::
 *     \sum_{\substack{n^L,t,m,h^A \\ y^V \leq y}} output_{n^L,t,y^V,y,m,n,c,l,h^A,h}
@@ -587,7 +600,7 @@ RESOURCE_HORIZON(node,commodity,grade)$( SUM(year$map_resource(node,commodity,gr
 *         \cdot CAP_{n^L,t,y^V,y} & \\
 *     + \ STOCK\_CHG_{n,c,l,y,h} + \ \sum_s \Big( land\_output_{n,s,y,c,l,h} - land\_input_{n,s,y,c,l,h} \Big) \cdot & LAND_{n,s,y} \\[4pt]
 *     - \ demand\_fixed_{n,c,l,y,h}
-*     = COMMODITY\_BALANCE_{n,c,l,y,h} \quad \forall \ l \notin (L^{RES}, & L^{REN}, L^{STOR} \subseteq L)
+*     = \COMMODITYBALANCE_{n,c,l,y,h} \quad \forall \ l \notin (L^{RES}, & L^{REN}, L^{STOR} \subseteq L)
 *
 * The commodity balance constraint at the resource level is included in the `Equation RESOURCE_CONSTRAINT`_,
 * while at the renewable level, it is included in the `Equation RENEWABLES_EQUIVALENCE`_,
@@ -698,7 +711,7 @@ $ENDIF
 * This constraint ensures that supply is greater or equal than demand for every commodity-level combination.
 *
 *  .. math::
-*     \text{COMMODITY_BALANCE}_{n,c,l,y,h} \geq 0
+*     \COMMODITYBALANCE_{n,c,l,y,h} \geq 0
 *
 ***
 COMMODITY_BALANCE_GT(node,commodity,level,year,time)$(
@@ -722,7 +735,7 @@ COMMODITY_BALANCE_GT(node,commodity,level,year,time)$(
 * is (exactly) equal to demand.
 *
 *  .. math::
-*     \text{COMMODITY_BALANCE}_{n,c,l,y,h} \leq 0
+*     \COMMODITYBALANCE_{n,c,l,y,h} \leq 0
 *
 ***
 COMMODITY_BALANCE_LT(node,commodity,level,year,time)$(

message_ix/model/MESSAGE/parameter_def.gms

@@ -190,6 +190,10 @@ Parameter
 * Input/output mapping, costs and engineering specifications
 * ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 *
+* .. _input:
+* .. _output:
+* .. _technical_lifetime:
+*
 * .. list-table::
 *    :widths: 25 60
 *    :header-rows: 1
@@ -233,6 +237,13 @@ Parameter
 *    * - emission_factor
 *      - ``node_loc`` | ``tec`` | ``year_vtg`` | ``year_act`` | ``mode`` | ``emission``
 *
+* .. _input_cap:
+* .. _input_cap_new:
+* .. _input_cap_ret:
+* .. _output_cap:
+* .. _output_cap_new:
+* .. _output_cap_ret:
+*
 * Input/output mapping related to technology capacities
 * ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 *
@@ -724,6 +735,9 @@ Parameters
 * (provided by another model) to derive supply of commodities (e.g., biomass) and emissions
 * from agriculture and forestry. The parameters listed below refer to the assigned land scenario.
 *
+* .. _land_input:
+* .. _land_output:
+*
 * .. list-table::
 *    :widths: 25 75
 *    :header-rows: 1