MCP-0034 Ternary

[henrikt-ma]

Opening a draft pull request for MCP-0034 in order to create a forum for discussing the MCP at the stage of having a prototype implementation, hoping that there will be a clear direction to follow when writing specification changes. Such preliminary discussion, combined with just a short delay for writing specification changes, will then provide MCP reviewers with fresh background that should help reaching the under evaluation stage in a timely manner.

[henrikt-ma]

Language group at 101th design meeting:

Poll: Scope of Ternary:

• Full-fledged built in type, similar to Boolean: 6
• Defined as proposed, but then restricted in various ways to only be allowed inside annotations: 4
• Defined in a different way with meaning only inside annotations: 0

Conclusion: No change for now.

Poll: How to define Ternary:

• In the same way as Boolean (using TernaryType etc): 4
• Similar to Clock, that is, not as a reserved name: 3
• Define as pre-defined operator record: 2
• Abstain: 1

Conclusion: No change for now.

Poll: How to refer to unknown:

• unknown as keyword: 5
• unknown as identifier: 3
• Ternary.unknown without Ternary.true and Ternary.false: 2
• Ternary.unknown with Ternary.true and Ternary.false: 1
• consensus(false, true): 0
• Abstain: 1

Conclusion: No change for now.

Define how to map Ternary when exporting to FMI:

• Yes, but only for FMI 2 and later, as enumeration type with enumerators {false = -1, unknown = 0, true = 1}: 5
• Yes, and also for FMI 1, but without mapping to integer values for FMI 1: 2
• No, don't define it: 6
• Add non-normative comment about possibility to map to enumeration: 4

Conclusion: No change for now.

Poll: Make specification changes and start collecting reviews:

• Yes: 5
• No, revisit at another design meeting: 0
• No, give up: 0
• Abstain: 4

Conclusion: Make spec changes and collect reviews.

[HansOlsson]

From phone-meeting:
Ternary MCP-0034 -> Henrik?

• Gerd: Should have time to implement

[HansOlsson]

@gkurzbach status?

[gkurzbach]

@gkurzbach status?
Will start this month.

[henrikt-ma]

The widespread "shoudPass" typo (in the TernaryTest library) mentioned in yesterday's phone meeting has now been corrected.

[henrikt-ma]

Design meeting:

Questions:

• Does unknown really need to be keyword?
• Can external C interface use enumeration?
• Inconsistent to allow explicit conversion from Boolean when there is also implicit conversion?
• Are we making sure that we are explaining what ternary logic is?

Does unknown really need to be keyword?

Alternatives:

• unknown as keyword
  • Compact notation and always clear what unknown means
  • Analogous to false and true being keywords
• Ternary.unknown
  • No new keyword
  • Mentally similar to enumeration
  • Similar to StateSelect.never
  • Cannot easily also have Ternary.true and Ternary.false due to true and false being keywords
  • Could make true and false constants in top level scope instead of keywords
• unknown as built-in constant in top level scope
  • Won't break models using unknown as identifier
  • Can deprecate use as identifier, for making it keyword in the future
  • Can use .unknown to avoid shadowing problems, but won't automatically work if unknown becomes keyword
  • Could make true and false constants in top level scope instead of keywords
• Ternary()
  • No problem with shadowing.
  • No clear sign of this being the unknown value.

Two step solution with different end results:

1. Introduce unknown as top level constant and deprecate shadowing it (possibly also deprecate accessing it as .unknown). Allow Ternary() for the odd case of suffering from shadowing during deprecation period.
2. One of the following directions:

• Make all of false, unknown, and true keywords.
• Make all of false, unknown, and true top level constants.

Language group:

• Do 1 now, and postpone deciding 2.

Can external C interface use enumeration?

Alternatives:

• -1, 0, 1
• 1, 2, 3 (similar to Modelica enumerations)
• typedef enum{MODELICA_FALSE, MODELICA_UNKNOWN, MODELICA_TRUE} Modelica_ternary; in ModelicaUtilities.h

Nobody wants the typedef with its clutter in the global namespace.

Majority in favor of 1, 2, 3.

Inconsistent to allow explicit conversion from Boolean when there is also implicit conversion?

• Reasons for having it:
  • Makes code more clear when present, useful to express what was originally implicit in later intermediate representations such as "Flat Modelica".
  • Resembles how enumeration is cast to Integer (even though this can't be done implicitly).
  • Not hard to implement.
• Reasons for not having it:
  • Consistency with Integer→Real and enumeration→Integer conversions: explicit conversion if and only if there is no implicit conversion.

Majority wants to have possibility to convert explicitly.

Are we making sure that we are explaining what ternary logic is?

Yes, we should make sure this makes it from the rationale documents to the specification changes.

Next steps

1. Update test implementation
2. Get language group support for continuing with making specification changes.

[henrikt-ma]

Of the Next steps in #2477 (comment), the first has been completed. This brings us to the next item, which I'd like to put on the agenda for the upcoming phone meeting:

• Get language group support for continuing with making specification changes.

[henrikt-ma]

I have received local feedback on the proposal that I'm forwarding to the language group. It is the same old question about the external C mapping to int, and with this mapping also being expected to show up in other places such as result files and table data files, there's a fear of users mixing up Boolean and Ternary data.

By skipping 1 in the mapping, one could reliably detect when a Ternary table column is populated with Boolean data, even when all values are Ternary(true). From the point of view of the test implementation, it would mean no added complexity to use the mapping 2 (Ternary(false)), 3 (unknown), 4 (Ternary(true)) instead of 1, 2, 3. I'd appreciate if the language group would like to comment on the pros and cons of these two mappings, and conclude with a poll:

• Mapping A: 1 (Ternary(false)), 2 (unknown), 3 (Ternary(true))
• Mapping B: 2 (Ternary(false)), 3 (unknown), 4 (Ternary(true))

[christoff-buerger]

I have received local feedback on the proposal that I'm forwarding to the language group. It is the same old question about the external C mapping to int, and with this mapping also being expected to show up in other places such as result files and table data files, there's a fear of users mixing up Boolean and Ternary data.

By skipping 1 in the mapping, one could reliably detect when a Ternary table column is populated with Boolean data, even when all values are Ternary(true). From the point of view of the test implementation, it would mean no added complexity to use the mapping 2 (Ternary(false)), 3 (unknown), 4 (Ternary(true)) instead of 1, 2, 3. I'd appreciate if the language group would like to comment on the pros and cons of these two mappings, and conclude with a poll:

* Mapping A:  1 (`Ternary(false)`), 2 (`unknown`), 3 (`Ternary(true)`)

* Mapping B:  2 (`Ternary(false)`), 3 (`unknown`), 4 (`Ternary(true)`)

I really do not understand why we do not use 0 for false. I see your point to avoid mixup with _Bool, but I honestly think there is no chance to avoid such -- anything non-zero is true in C; and any assumption about which values are used in practice by a C compiler and linked libraries is false.

But using a non-zero value for false just adds boilerplate code to C based implementations, with some funny intrinsic consequences. To represent false by 0 in C is so deeply build in the language design, it is related to pointer arithmetics, implementation of control-flow instructions on hardware side, compiler optimizations etc. I am afraid a non-zero false is just fighting the language and all the ecosystem around it.

All of this is of course, if we are concerned of C as implementation language at all -- it is not like the world is build around C. My understanding of your question is that we are concerned.

[henrikt-ma]

My take of the long discussion behind #2477 (comment) was that we really didn't want a mapping that could easily be confused with the mapping of Boolean; the desirable property we got from {1, 2, 3} was that we at least didn't map any value the same as the Boolean false, so that mistakes could be detected by noticing that none of the Ternary values would have the effect of false if accidentally being treated as a Boolean in the external function type mapping.

With Ternary(false) being mapped to 0, I see a big risk of not detecting when unknown and Ternary(true) are mixed up in the external functions, especially if there are not many unknown values in the test data.

Also note that with the Ternary values mapping to {2, 3, 4}, implementations will almost certainly detect when the external function makes the mistake of returning a value with the encoding for Boolean, while the mapping {1, 2, 3} would mean that a function always returning true, incorrectly encoded as 1 (meaning false), could not be detected by the external language interface. Of course, chances of detecting problems in return values would be drastically reduced by the mapping {0, 1, 2}, where both false and true would map to valid values for a Ternary. With {2, 3, 4}, the only way a truth value could be incorrectly returned as a Boolean without being detected is when true has the numeric value 4 in the C code, and is returned without canonicalization to 1; in practice extremely unlikely to happen.

[gwr69]

From reading this, I get the impression that detecting an accidental misuse of Boolean where Ternary was called for is the most important aspect to consider in choosing a mapping onto a numerical range. But what about avoiding an error (minimize the probability for an unintended return value) or minimizing error effects? Given that a mapping {unknown, false, true} onto {-1, 0, 1} is rather widespread and that the range {0, 1, 2} nicely aligns with ternary digits and that from the interpretation of undecidable the range {0, 0.5, 1} lends itself, we see that in all (!) of these common mappings false and true will be correctly interpreted, e.g., automatic conversion from Boolean to Ternary. The only value that is different when returned, e.g., -1 or 2 or 0.5, can safely be assumed to correspond to unknown making the probability of errors lower—not higher—imho? The range {2, 3, 4} appears highly unusual...

[HansOlsson]

I can see the benefit of the mapping to 1, 2, 3; since that makes Ternary close to a built-in enumeration type (and we could possibly extend enumeration types later so that it is just a built-in enumeration type).

Obviously one can argue that Modelica should be closer to C for enumerations in general, which would argue for 0, 1, 2 - but that would be a breaking change.

[gwr69]

I can see the benefit of the mapping to 1, 2, 3; since that makes Ternary close to a built-in enumeration type (and we could possibly extend enumeration types later so that it is just a built-in enumeration type).

Obviously one can argue that Modelica should be closer to C for enumerations in general, which would argue for 0, 1, 2 - but that would be a breaking change.

But adjusting 1-indexing to 0-indexing is easy to do, the question remains then, whether it will be {false, true, unknown} or {false, unknown, true}. Even if you decide to go for 1, 2, 3, the likeliness for a misrepresentation might be lower if unknown were to be outside the range and not inside and thus unknown === 3?

[henrikt-ma]

I think a very strong reason to avoid mapping Ternary(false) to 0 in C is that it like a trap: as long as the value is Ternary(false), you get away with the Boolean interpretation of an int in C, but as soon as you do so you will typically also interpret any non-zero value as true, and the error won't be detected until the day someone passes the value unknown to the function and the user actually realizes that it is being misinterpreted as Ternary(true).

For those who prefer a 1-based mapping similar to enumerations, this can be combined with a safe mapping to {2, 3, 4}, by internally using an enumeration corresponding to:

type Ternary_enum = enumeration(Ternary_error, Ternary_false, Ternary_unknown, Ternary_true);

When receiving values from an external C function, one can treat it as an enumeration, but instead of verifying that the value is in 1..4, one just modifies the test or adds an extra test that will reject Ternary_error.

[HansOlsson]

Looking at the original motivation this seems to have become way too complicated compared to the problem it was intended to solve, i.e., handling a tool-specific default for visible as a third-value - such that a user can say something like visible=if x then true else <keep tool-specific default>.

To implement that we need:

• A built-in tree-valued type
• Some kind of overloading - so that visible can be Boolean or three-valued.
• Possibility to define handling of and/or (will return to).

The first part is to check what we currently have and in MSL we have 4-valued (Modelica.Electrical.Digital.Interfaces.UX01) and 9-valued logic (Modelica.Electrical.Digital.Interfaces.Logic), and built-in in the language we have one type that is sort of 5-valued logic (StateSelect - similarly as visible it allows a tool to have a default for a binary choice and the user to override it to force the underlying binary choice to be true or false). I believe the reason we missed the 4-valued and 9-valued logic in the original discussion is that we unfortunately didn't consider language and standard library as one entity.

As you might guess they are all enumerations (a concept that was added together with StateSelect in 2.0).

So, why not accept that Ternary is another built-in enumeration (as already suggested). Since we cannot name it Ternary.false call it Ternary.ForceFalse or Ternary.never (following StateSelect). Nothing special for conversion in expressions - similarly as for StateSelect one will have to write: stateSelect=if enforceStates then (if useQuaternions then StateSelect.always else StateSelect.never) else StateSelect.avoid).

Note that for visible the common cases will be: if x then Ternary.always else Ternary.default and if x then Ternary.never else Ternary.default (or possibly some other name than Ternary); and to me not having a conversion between Boolean and Ternary actually makes this more readable. If the name is good enough we might even deprecate using a Boolean for visible - thinking that visible=Ternary.always is clearer than visible=true.

The benefits of this are:

• The language and library stays consistent for users, and doesn't add much to the mental model. Basically having 3-, 4-, 9-valued (and kind of 5-valued) logic handled in a similar manner reduces the threshold for new users. This also means that the external interface just works like any other enumeration (if anyone actually uses them in external C-functions). If we make any simplifications it should be across all enumerations.
• Consistent user-interface reducing the burden for users and developers: e.g., code completion, plotting etc, can work consistently between 3-, 4-, 9-valued logic.
• Shorter time to add to MSL. There are blocks for handling operations on 4- and 9-valued logic; with connectors as inputs (and outputs) - users expect the same for 3-valued. By having it as a built-in enumeration we can add it to MSL-repo as a top-level class together with blocks using 3-valued logic that will work right now, without waiting for a new ModelicaSpec (and having it used in MSL).

Note that this means that and/or will be blocks similarly as for 4- and 9-valued logic. (How they are implemented is of less importance - internally they can use min/max for 3-valued logic or tables as the 4- and 9-valued logic.)

Note that Boolean is kind of the exception here, one reason is that Boolean expressions can be used to control if an equation is present or not - a binary decision that doesn't have a real counter-part in the other logics. Another is that we for some reason started enumeration-counting at 1 not 0.

We can then later build on this - if needed; I guess there's a reason most computer languages don't have 3-valued logic built in.
For the C-interface we might, similarly as for records, consider how to extend the mapping to specific C-enumerations.

[henrikt-ma]

Let's not forget that the initial reason for creating this MCP was that you blocked any progress on dialog visibility control due to the lack of a ternary type in the language. Back then, it meant that there was a lot of work that had to be done for such a simple thing as a ternary visibility flag, but the idea of a Ternary in the language seemed useful enough that I considered it worthwhile trying to get this sort of obstacle out of the way once and for all. In endless design meetings since the beginning of 2020, the language group has guided the design towards the current state of the MCP. I must say I find too late to all of a sudden request a total change of direction now, but instead of discussing whether it's too late or not, let me give some good reasons for sticking with the current design.

To break the current separation of language and standard library design by making them cyclically dependent would be a disaster in my opinion. It is bad enough that ModelicaServices isn't defined in the specification, but at least the rest of the specification doesn't depend upon ModelicaServices, so let's not put Ternary in ModelicaServices either.

Unlike Ternary, it seems right to me that the domain-specific enumerations for digital circuits in Modelica.Electrical.Digital are defined within the library. For example, I really don't want to see Digital.Interfaces.UX01 being used in the standard language annotations. The Ternary of this MCP, on the other hand, is not a domain-specific type. It serves a much more basic and generic purpose in the language, and as we know from the background of this MCP, it is the kind of thing we would like to make use of in standard language annotations.

A Ternary with implicit conversion from Boolean is very user-friendly compared to a built-in enumeration, as the user doesn't have to keep track of which true or false to use when expressing truth values.

The implicit conversion from Boolean also means that functions and annotations can be enhanced in a backwards compatible manner by changing function inputs or annotation flags from Boolean to Ternary. Existing code where Boolean expressions are used will still work, but a function input can be made optional with more intelligent behavior than just defaulting to a Boolean expression in the other inputs, and an annotation can explicitly default to unknown instead of having special rules applying when no value is given (which no expression that could also be true or false can represent).

Similar to the lack of blocks for StateSelect in the MSL, I don't see a strong need for Ternary blocks in the MSL, and it might even be desirable to not introduce such blocks until we have identified a block-oriented modeling domain where Ternary is a good model of the data in connectors.

[HansOlsson]

(Apologies if duplicated.)

Let's not forget that the initial reason for creating this MCP was that you blocked any progress on dialog visibility control due to the lack of a ternary type in the language. Back then, it meant that there was a lot of work that had to be done for such a simple thing as a ternary visibility flag, but the idea of a Ternary in the language seemed useful enough that I considered it worthwhile trying to get this sort of obstacle out of the way once and for all. In endless design meetings since the beginning of 2020, the language group has guided the design towards the current state of the MCP. I must say I find too late to all of a sudden request a total change of direction now, but instead of discussing whether it's too late or not, let me give some good reasons for sticking with the current design.

To me it sometimes makes sense to step back and reconsider, and simplify the design based on previously considered variants.
This MCP seems to have taken some basic ideas and run far ahead with them - without considering what already exists.

A Ternary with implicit conversion from Boolean is very user-friendly compared to a built-in enumeration, as the user doesn't have to keep track of which true or false to use when expressing truth values.

That is not generally true, but depends on the tool support (or lack thereof).

For a declared enumeration Ternary Dymola will currently:

• Have drop-down boxes in the parameter dialog (with optional description texts).
• Allow code completion so that you can just type Ternary. and hit Ctrl-Space to get a list of elements.

In that case an implicit conversion is in fact less user-friendly, unless something is added for that as well.

Obviously, it would be possible to support both - but it will still add to the cost in tool and to the user's mental model; and I'm not sure if it will ever become even as user-friendly as an enumeration.

Adding new features, because the current ones (such as enumeration) aren't convenient enough, isn't a good way forward.

Similar to the lack of blocks for StateSelect in the MSL, I don't see a strong need for Ternary blocks in the MSL, and it might even be desirable to not introduce such blocks until we have identified a block-oriented modeling domain where Ternary is a good model of the data in connectors.

If we don't see a major need for Ternary blocks (or functions) in MSL (or other similar libraries), then I'm not sure if there is sufficient use of Ternary outside of annotations to make it more convenient.

I'm not saying that there's no use for Ternary in model, just that I don't see it as significant enough to motivate the added complexity.

[HansOlsson]

Language group:
Markus: Need operations, don't think a lot of people will use in C-code.
Gerd: Gave up implementing (too much effort)

[henrikt-ma]

Markus: Need operations, don't think a lot of people will use in C-code.

It is not the expectation that there will be lots of Ternary inputs and outputs of external functions that is the main motivation for specifying the mapping to integers. A much more important motivation is that the external language mapping is the de facto convention for data interchange outside the scope of the specification. For example, how to store simulation results, or how to represent a table on file for use with the MSL tables. If the de facto convention leaves the tools without any guidance when it comes to Ternary, tools will inevitably have to come up with their own tool-specific mappings, and this would cause a large amount of unnecessary debate and headache the day we regret that we didn't agree on a mapping form the beginning.

That the external function value mapping is the de facto convention for other kinds of data interchange also means that if we agree on a mapping which makes it easy to detect mistakes when using external functions, it will also become easy to detect mistakes when dealing with things like simulation results and table data. In the end, these could be more important reasons to support easy error detection than the error detection in external functions. The less overlap with the mappings of Boolean and Ternary, the better.

Once we have agreed upon a mapping, there are good reasons to actually specify that this mapping should be used by the external language interface:

• We don't need to come up with another way of stating what the convention is.
• It avoids leaving the specification with a strange gap in the external language type mappings.
• The little implementation effort is very small.

[henrikt-ma]

I also heard from @qlambert-pro that there were some questions in the meeting about the completeness of our test implementation. The easiest way to assess this is to look at the test library https://github.com/henrikt-ma/TernaryTest, which is fully supported. For those who find it boring to just look at the test library code, I can give a live demo where we can both look at the examples in the test library, as well as other cases brought up by the audience.

Please react with 👍 in case you are interested!

[HansOlsson]

Markus: Need operations, don't think a lot of people will use in C-code.

It is not the expectation that there will be lots of Ternary inputs and outputs of external functions that is the main motivation for specifying the mapping to integers. A much more important motivation is that the external language mapping is the de facto convention for data interchange outside the scope of the specification. For example, how to store simulation results, or how to represent a table on file for use with the MSL tables. If the de facto convention leaves the tools without any guidance when it comes to Ternary, tools will inevitably have to come up with their own tool-specific mappings, and this would cause a large amount of unnecessary debate and headache the day we regret that we didn't agree on a mapping form the beginning.

And we should take the logical next step, and see if we can avoid even having to ask those questions.

We currently have the following kinds of types in Modelica:

• Non-primitives:
  • record
  • array
• Primitive:
  • Real
  • Integer
  • String
  • Boolean
  • enumerations (including StateSelect, 4- and 9-valued logic in MSL)

Adding a new kind of primitive type has proven to be a substantial effort (based on experience with adding enumerations), and thus doing it needlessly would be an unnecessary hurdle for all Modelica users and implementers especially as Ternary can naturally fit into one of these existing categories; as an enumeration. (Alternatively it could be a record for "option type" with two booleans, but that would be messier.)

Apart from not increasing the burden internally in tools (including code completion, drop-down menus for parameters, plotting) that also removes the need to consider the external C-interface, exporting to FMI, as all of that is already defined for the existing categories, and simplifies the user's mental model.

Supporting a reasonable Ternary that solves the original issue shouldn't take more code than this comment. Going back to the original issue I also realize that we don't have consider compatibility for the new Ternary visible (or active?) as it doesn't replace an existing Boolean visible (or active?).

That's why the lack of second implementation should have been a major warning flag.

[DagBruck]

Just to make sure I understand this: Ternary could be an enumeration, but that would not allow you to reuse the current keywords true and false, and convert from Boolean to Ternary. Is that the gist of it?

[henrikt-ma]

Just to make sure I understand this: Ternary could be an enumeration, but that would not allow you to reuse the current keywords true and false, and convert from Boolean to Ternary. Is that the gist of it?

In addition to that, it would be strange to provide semantics for operators (unless the language introduced a general mechanism for operators acting on enumerations).

As I see it, it would be a major drawback to go for a solution without implicit conversion from Boolean, as this means that every change from Boolean to Ternary would break backwards compatibility. For example, pretend we wanted to leverage the Ternary expressiveness for the Dialog.enable to say one of force enable, tool decides, or force disable. To avoid breaking backward compatibility, we would need to leave the Boolean Dialog.enable, introduce a parallel Ternary Dialog.tenable, and specify weird semantics for how the two should interact. With implicit conversion from Boolean, on the other hand, we could just have Ternary Dialog.enable, and all existing code would remain valid, while new code could take advantage of the third truth value.

Similarly, without implicit conversion from Boolean, functions currently taking a Boolean argument, but where one would like to use a Ternary instead, would need to be duplicated in order to avoid breaking backward compatibility.

[HansOlsson]

Just to make sure I understand this: Ternary could be an enumeration, but that would not allow you to reuse the current keywords true and false, and convert from Boolean to Ternary. Is that the gist of it?

Yes, for the first part (not reuse), and more complicated for the second (convert).

With Ternary as an enumeration the enumeration members cannot be named true and false, since they are reserved words; and thus we need different names for them - similarly as for StateSelect and for 4-valued and 9-valued logic.

Having Ternary as an enumeration doesn't prevent us from having a conversion from Boolean even if a bit unusual. However, without the underlying enumeration it would be practically necessary.

[DagBruck]

As I see it, it would be a major drawback to go for a solution without implicit conversion from Boolean, as this means that every change from Boolean to Ternary would break backwards compatibility.

Yes, I can see that. But doesn't that suggest the need for a more fundamental construct to allow conversion to and from user-defined types? And such an extension would not only allow a good implementation of the type Ternary, but also the next type of similar characteristics.

[henrikt-ma]

As I see it, it would be a major drawback to go for a solution without implicit conversion from Boolean, as this means that every change from Boolean to Ternary would break backwards compatibility.

Yes, I can see that. But doesn't that suggest the need for a more fundamental construct to allow conversion to and from user-defined types? And such an extension would not only allow a good implementation of the type Ternary, but also the next type of similar characteristics.

I'm afraid that would be so complicated that we'd never complete it. The proposal for Ternary has already been carefully designed over numerous design meetings, been test implemented, and is accompanied by a thorough test library.