Inconsistencies in getting component data

[clyman23]

There are currently a number of ways to retrieve component data from a solved pyomo model (Concrete Model) using the latest released version of pyomo (v5.7.3). This was noticed when upgrading from 5.6.x to 5.7.x with the introduction of the .data() method for some pyomo components.

There are many ways to retrieve the values of component data that vary based on the component type. The following is my current understanding of how to retrieve information for each component type:

• Set set_name
  • Need to call set_name.data()
  • set_name.value has been deprecated
  • pe.value(set_name) will fail with TypeError: 'OrderedSimpleSet' object is not callable
• Indexed Param param_name indexed by set set_name
  • Simply call param_name[set_value] where set_value is some value in set_name to get the param value at that index
  • Can also call pe.value(param_name[set_value]) and get same result
  • param_name[set_value].value and param_name[set_value].data() will both fail with AttributeError (understandably, since the return of param_name[set_value] is just the value of the param at that index)
  • Calling param_name.value and param_name.data() both return AttributeError
• Simple Param param_name
  • Call pe.value(param_name) or param_name.value
  • param_name.data() raises AttributeError
  • Can also call pe.value(param_name[None]) or param_name[None].value
• Indexed Var var_name indexed by set set_name
  • Need to call pe.value(var_name[set_value]) or var_name[set_value].value
  • Unlike an indexed param, cannot just call var_name[set_value]
  • Using .data() returns AttributeError
• Simple Var var_name
  • Same things as Indexed Var when using set_value=None
  • Can also just call var_name.value or pe.value(var_name)
  • Using .data() returns AttributeError

All of this can make it very confusing for developers to extract solved model component values; the developer is required to use different methods to extract values if the component is a set, a param, or a var.

I would propose expanding the functionality of pe.value() method to be able to extract values of all component types. This would streamline the process and make deployment of large pyomo projects much simpler.

[blnicho]

@clyman23 could you give more context about your intended use-case? There are lots of examples in the Pyomo code base of how to programmatically interact with and extract results from a Pyomo model. Many of the methods that could help with this are not fully documented but if you give us an example of what you would like to do we can point you to methods that you might not know about.

For example, if you want to print out the values of every Param and Var you could do:

# Display values of all Params and Vars
for c in model.component_objects((pe.Param, pe.Var)):
	c.display()

# Programmatically extract values of all Params and Vars
for c in model.component_objects((pe.Param, pe.Var)):
	for k, v in c.items():
		print(c.name, '[',k,'] = ', pe.value(v))

[clyman23]

@blnicho thank you for the response; this is definitely helpful. My main concern is the inconsistency for extracting Param/Var values vs extracting Set values (and that pe.value() fails when passed a Set).

There is the additional inconsistency with accessing values of Params and Vars:

import pyomo.environ as pe
model = pe.ConcreteModel()
model.x = pe.Param([1, 2], initialize=100)
model.y = pe.Var([1,2], initialize=0)

print(model.x[1]) # Returns value 100
print(model.y[1]) # Returns _GeneralVarData object

print(pe.value(model.x[1])) # Returns value 100
print(pe.value(model.y[1])) # Returns value 0

This second inconsistency isn't a huge problem, but it is a little curious that the Param and Var objects respond differently to being directly indexed like this.

[jsiirola]

The second inconsistency is due to the Param data model. By default, Params are immutable, which means that their values are not allowed to change after the Param is constructed. This promise to never change the value enables certain efficiencies (both memory and performance), including the ability to directly return the value when being accessed by access methods like model.x[1]. This allows consumers (the expression system) to perform certain optimizations (e.g., removing constant terms like 0*m.myvar). However, if you want the Params to be changable (i.e., you declared them with mutable=True), then you would get _ParamData objects. That is:

import pyomo.environ as pe
model = pe.ConcreteModel()
model.x = pe.Param([1, 2], initialize=100, mutable=True)

print(model.x[1]) # Returns _ParamData object
print(pe.value(model.x[1])) # Returns value 100

The recommended way to get the value of a Var/Param/expression is to use the value() function (exactly because of the challenge of working with immutable Param objects).

I agree that value() should support Set objects. We should add that functionality.

(BTW: Set.value is deprecated because it returned the set values as a python set, which is inherently unordered in all versions of Python. Set.data() returns a tuple, which at least preserves the ordered-ness of the underlying Set)

[clyman23]

@jsiirola thank you for the insight! Very helpful.

Agreed on value() needing to support Set objects

[dimili]

Another inconsistency I noticed is when you fix a variable value with model.varname.fix(). Then
value(model.varname) or model.varname.value returns <pyomo.core.base.param.SimpleParam at 0x9e45368> while
value(model.varname)() or model.varname.value() returns the fixed value.