Remove use of enums when matching models to writers

[whart222]

One of the nice features of Pyomo is that it is easy to walk the expression tree to generate different file formats.

I've had the occasion to do this several times recently, but I've noted that it's not easy to integrate these writers into pyomo in a manner that makes the model.write() logic work seamlessly.

Specifically, the enum types in pyomo.opt.base.formats need to be modified appropriately. For example, see the following fork of Pyomo where a JSON problem writer is prototyped: https://github.com/whart222/pyomo/blob/json_writer/pyomo/opt/base/formats.py

To make Pyomo extensible, I think we need to move away from enum types, and instead have a global list of formats that are associated with registered writers and readers.

[michaelbynum]

I completely agree about not using enums. However, why not just have functions to call or classes to create and use? Something like:

from pyomo.writers import JsonWriter
writer = JsonWriter()
writer.write(model)

or

from pyomo.writers import write_json
write_json(model)

[jsiirola]

@michaelbynum I agree. Plus we already have a WriterFactory that supports creating writers by name. The enums that @whart222 is referring to are an artifact of the old "meet in the middle" approach that Pyomo uses for implicitly mapping models to solvers. Through various discussions we have had, my understanding was that we were going to move away from that model to something that relied on the solver either the directly instantiating the writer (like you suggest) or using a factory interface to directly instantiate the writer. This issue is writing down that goal, and extending it to update how the model.write() method works.

[whart222]

FWIW, I also think it's reasonable to make our writers directly accessible in a natural manner @michaelbynum . But I'd like to preserve the magic of model.write("foo.<fmt>") where the user specifies the format via the filename. This requires a registration of writers.

[carldlaird]

@whart222 Can you discuss the benefits of that magic? I am skeptical, but might be convinced.

Also, I fear that it is ambiguous anyway, and might be better handled by the mechanism @michaelbynum describes.

Thoughts?

[whart222]

If we have a tool, like the pyomo commandline, which allows the user to specify an output file "foo.nl", then there is an implied global knowledge somewhere of all the writers.

Rather than embed that global knowledge in some random location for each tool like this, it is sensible to put that global knowledge in a common location that can be easily extended. So, when I create my "blah" file format, I can still use the pyomo commandline to write to it.

This doesn't preclude the usefulness of organizing Pyomo to include a pyomo.writers module for all the standard writers.

[carldlaird]

@whart222 Good catch. Can/should the logic be in the command-line tool to support that capability?

[whart222]

IMHO, the logic should be core to Pyomo. I like to use factories to create writers/solvers/etc in my scripts, so it would be weird to have that be tied to the command-line tool.

[carldlaird]

@whart222 are we discussing factories, or auto-detection of writer from filename?

[whart222]

You could have a pyomo.globals for this type of stuff. I don't think this is a good idea, but it is explicit.

[whart222]

Yes, to both.

[carldlaird]

OK, but a factory is explicitly selecting a particular writer - auto-detecting writer from filename is ambiguous... that was my major concern. (No problem with factories here...)

[whart222]

If the writer logic is done correctly, wouldn't this be unambiguous? If we have multiple *.json writers, then I think they'd need to be disambiguated to register them (e.g. writers for *.foo.json and *.bar.json files). And if the user said write my.json then the writer would complain.

[michaelbynum]

I agree that factories are convenient. However, why not just have names associated with the writers just like we do for the solvers. Then, you don't need a global list anywhere. For example, we have lots of interfaces to Gurobi:

opt = SolverFactory('gurobi')
opt = SolverFactory('gurobi_direct')
opt = SolverFactory('gurobi_persisent')

We could do something similar for writers.

@writer_factory.register('json')
class JsonWriter(object):
    ...

@writer_factory.register('different_json')
class MySpecialJsonWriter(object):
    ...