Switching data between being a math parameter, decision variable, or global expression

[brynpickering]

I'm moving this discussion from #771 to here so that it isn't lost in a sea of closed PRs when it's merged.

The content of this post was originally written by @irm-codebase with minor edits by me.

This discussion pertains to switching a referenced math component between being a parameter, decision variable, or global expression.

Example

This is useful when switching between a capacity expansion model (flow_cap is a decision variable) and a dispatch model (flow_cap is a parameter).
Most constraints don't need to change when this switch is made - flow_out is still limited by flow_cap whether that's a variable or fixed value - so we can keep the reference to flow_cap in them and separately handle whether that math component is a parameter or a decision variable.

1. Switching through a new Model instantiation

Suggested by @brynpickering with a PoC in #783

In this approach, parameters and variables are two distinct entities and users must actively 'swap' between them in code by initialising another model. All model inputs / schemas and so on are re-validated against parameter / dimension definitions.

It could also allow us to clean up the math so that it is all dealt with at model instantiation, including e.g. defining the model "mode".

# Expected shape of a YAML math file
parameters:
variables:
global_expressions:
constraints:
piecewise_constraints:
objectives:

# expected pydantic schemas
class Parameter(MathIndexedComponent):
    unit: str = ""
    default: NumericVal = float("nan")
    type: Literal["int", "float"] = Field(default="float")

class Variable(MathIndexedComponent):
    unit: str = ""
    default: NumericVal = float("nan")
    domain: Literal["real", "integer"] = Field(default="real") # this could be aligned with `parameter.type`.
    bounds: Bounds,  # max / min

# expected code sequence
model = calliope.from_yaml(plan_model)
model.build()
model.solve()

new_inputs = model.inputs.assign(model.results[var] for var in capacity_variables)
new_model = calliope.Model(
    inputs=new_inputs,
    attrs=model.all_attrs.update({"config.init.mode": "operate"})
    #  or **kwargs to allow updates, i.e. `mode=operate`
)  

Embedding parameter / variable 'switching' within the YAML math

suggested by @irm-codebase

In this approach, we combine Variable and Parameter into Numeric, and specify the direction (input/parameter or result/variable) in a role field. The math files then define their expected role for each numeric value.

Will likely benefit from keeping everything in model.data.

# Expected shape of a YAML math file
numeric:
global_expressions:
constraints:
piecewise_constraints:
objectives:

# expected pydantic schema
class Numeric(MathIndexedComponent):
    unit: str = ""
    default: NumericVal | None = None
    domain: Literal["real", "integer"] = Field(default="real")
    bounds: Bounds,  # max / min
    role: Literal["parameter", "variable"]

# expected code sequence
model.build()  # base math sets number.flow_cap.role = "variable"
model.solve()
model.build(mode="operate", force=True)  # operate.yaml sets numeric.flow_cap.role = "parameter"

[irm-codebase]

@brynpickering fits our proposals, yeah. As discussed, let's go with option 1 (#783).

The main problem is that we kind of let go of easy 'switching', but the code simplifications are worth it.