Surprising external transitions behavior for parallel states

[Andarist]

Consider this codesandbox with a machine such as:

createMachine({
  type: "parallel",
  states: {
    s1: {
      initial: "a",
      states: {
        a: {
          entry: () => console.log("entry a")
        },
        b: {
          id: "b",
          entry: () => console.log("entry b")
        }
      },
      on: {
        FOO: "#b"
      },
      exit: () => console.log("exit s1")
    },
    s2: {
      exit: () => console.log("exit s2")
    }
  }
})

This currently doesn't behave correctly in terms of SCXML because the defined transition is external and for external transitions, the LCCA state of the source and target states acts as the transition's domain. LCCA stands for Least Common Compound Ancestor.

What does it mean? It means that any descendant of the LCCA gets exited and reentered (if it is in the machine's configuration prior to transition). This is a very surprising behavior in this example because the source state is s1 and it's a direct child of a parallel state. This means that its LCCA is actually the implicit root state (as parallel state cant act as LCCA) which in turn means that s2 should be exited and reentered as well. I highly doubt that this is intuitive behavior and it's pretty easy to get tripped by this - I would expect us to at least considering switching the default type of transitions to internal because it seems that this is what people usually expect and it should lead to a better "pit of success" feeling for our users.

It seems to me that even a simpler example that doesn't involve any parallel states is rather surprising: https://codesandbox.io/s/silly-buck-68c26?file=/src/index.js

[johnyanarella]

Documenting for a broader audience some of the discussion you and I have previously had about this topic, along with some new thoughts:

XState's magic default for relative transition targets

I'm going to quote the machine declaration portion from your second sandbox here, as it is a succinct example of another dimension of the problem at hand:

createMachine({
  entry: () => console.log("entry root"),
  exit: () => console.log("exit root"),
  on: {
    FOO: "b"
  },
  initial: "a",
  states: {
    a: {
      entry: () => console.log("entry a")
    },
    b: {
      id: "b",
      entry: () => console.log("entry b")
    }
  }
})

The SCXML specification for <transition> defines a default of "external" for the transition's type. XState introduces a different default of "internal" when you specify a relative target.

i.e. as things stand today, for the FOO transition above:

Transition Target	XState's Default Transition Type
b	external
.b	internal

NOTE: I would still expect that an SCXML export from XState of this machine would end up explicitly adding a type="internal" to the corresponding <transition> tag.

XState's configuration object as a DSL

In previous discussions, we ruled out the idea of changing the default to "internal" because it would deviate from the SCXML spec's default for the <transition> tag.

If we're being totally honest here, the JS API's configuration object has already taken a number of liberties in service of terse expressiveness in JavaScript. Ex. on and its children's various potential shapes; the assumption that the keyed values in on dictionary are ordered (valid for ES6 but not for the JSON spec); shorthand like onDone; and most notably, the idea of relative targets.

Any discussion as to whether we're deviating from the SCXML spec must first acknowledge that the JS API's configuration object is not a 1:1 mapping to SCXML tags. It is a DSL all its own, designed in the context of JavaScript specific syntax, idioms and conventions. (Which is also why that configuration object can not serve as a platform-agnostic interchange format, but SCXML can.)

So, there is an argument to be made that this leaves open the option of changing the default for a transition when defined using that DSL. In point of fact, XState already does in order to introduce a sensible default for relative targets (see above).

SCXML's choice of external as default, and the impact of LCCA behavior

With regard to the first example, and the astonishing consequences of the default external transition in that machine, I just wanted to document what you and I already discovered:

1. the language in SCXML spec is very specific about this LCCA behavior; and
2. it appears that this was done in order to match prior art (such as UML statecharts) - see: https://lists.w3.org/Archives/Public/www-voice/2013JanMar/0059.html

SCXML <transition type="internal"> behavior

Curiously, the SCXML spec defines some interesting behavior around "internal":

...if the transition has 'type' "internal", its source state is a compound state, and all its target states are proper descendants of its source state, the exit set consists of all active states in C that are proper descendants of its source state. (If a transition has 'type' of "internal", but its source state is not compound or its target states are not all proper descendants of its source state, its exit set is defined as if it had 'type' of "external". The exit set of a set of transitions is the union of the exit sets of the individual transitions.)

tl;dr: if you specify "internal" for a transition where that is not valid, it acts like you specified "external" instead.

🤯

From this clue, I get the sneaky suspicion they might have entertained the idea of "internal" being the default at one point, especially since that default behavior would generally do what we've been thinking is the intuitive thing to do.

To be clear, I don't like this at all, though. I wish they had introduced an "auto" option instead, and made explicitly specifying "internal" where it was invalid throw an error instead of silently doing the opposite.

Choosing the best among bad options...

For the purposes of interoperability across interpreters and other tools in the SCXML ecosystem, whatever the SCXML spec says is sacrosanct.

Most of the places where XState's DSL has deviated are generally different enough to avoid confusion with any corresponding SCXML tag.

However, in the situation where the DSL has a thing that is also called "transition" that has a "type" property with the same set of options as the SCXML <transition> tag's type attribute... deviation from the SCXML spec's default for a transition type has the potential to cause confusion.

Consequently, I think that the current relative-target-specific default is on shaky ground. Look at how drastically different the behavior is in your example above based on an easily omitted "." character. I don't think it is intuitive for one configuration object property's value to change the default of another property.

I think a more apparent and flexible resolution to these concerns would be to introduce an "auto" option (as the default) in the DSL.

I would expect this "auto" option to apply the most intuitive default behavior regardless of how you specify the target ('#b' vs 'b' vs '.b'). The transition type would be declared solely by the transition's type property rather than by syntax in its target. Additionally, this could potentially address your first example, where "internal" would be the sensible default for "auto" there.

With this change, the DSL has the opportunity to push users into the pit of success by default.

To be clear, "auto" would be strictly a feature of the DSL. The visualization would render transitions as external or internal, and the SCXML export would output transitions with the resulting "internal" or "external" type.

I'm not yet fully advocating the "auto" option above - just throwing this out there as another potential approach to resolving our shared concerns.

[johnyanarella]

Just realized "intuitive" is really underspecified above.

Scenarios I recall from our previous conversations:

1. common handlers that have been moved up to a shared common parent or ancestor state (to avoid duplication in all child or descendant states) and which transition within that region; or, more generally:
2. transitions solely to child or descendant states.

For both, XState's existing examples typically steer you toward declaring a relative target, where you get a default transition type of internal.

Given that, is this our heuristic for unsurprising default transition types? (i.e. the transition types that would be applied for "auto")

• transitions that target only child or descendant states are reasonably expected to be internal
• otherwise, are reasonably expected to be external
• regardless of whether the specified target was expressed as a symbol, id or relative path.

[Andarist]

I agree with @johnyanarella that XState is already its own DSL - heavily inspired by SCXML and we plan to maintain the ability to convert one to another as much as possible when it makes sense but at the end of the day it's a separate "language". More often than not people will use XState without prior knowledge of SCXML or UML or statecharts in general - I think we are not bound to preserve all defaults of the SCXML. And for people actually knowing about the prior art, it should be much easier to grasp the difference and learn about a different default from our docs. More than that - even Jim Barnett states that:

That said, it makes perfect sense for your GUI to default to 'internal'.

So it doesn't look to me that such a "change" would be fundamentally wrong - it would be a pragmatic choice.

---

Re: internal behaving like external when targeting states outside of the source state.

While maybe odd as it doesn't lead to people grasping both concepts as easily, it actually would be quite convenient for us to just use this as "auto". We wouldn't have to introduce any new terms to our own DSL (and code to handle it and other cases) which seems quite convenient as our goal is not to create as much to create a new language as it is to provide intuitive tools to solve problems. transition.type would be just left for SCXML-compatibility and advanced use cases.

[johnyanarella]

Yeah, in the end, that ship sailed already, when it comes to the existing DSL. 😁

The API already does auto

(I've been so deep in reading SCXML papers and thinking about alternative DSLs for it, that I forgot that the JavaScript DSL exposes an optional internal Boolean rather than an explicit type parameter.)

Agreed. In the docs, there's a long list of terse JavaScript DSL syntax that automatically become internal or external transitions—there's already an implied "auto" whenever internal is omitted.

Syntax	Transition Type	Auto vs Explicit
EVENT: '.foo'	internal transition to child	auto
EVENT: { target: '.foo' }	internal transition to child (starts with '.')	auto
EVENT: undefined	forbidden transition	auto
EVENT: { actions: [ ... ] }	internal self-transition	auto
EVENT: { actions: [ ... ], internal: true }	internal self-transition, same as above	explicit
EVENT: { target: undefined, actions: [ ... ] }	internal self-transition, same as above	auto
EVENT: { target: 'foo' }	all transitions to siblings are external	auto
EVENT: { target: '#someTarget' }	all transitions to other nodes are external	auto
EVENT: { target: 'same.foo' }	external transition to own child node	auto
EVENT: { target: '.foo', internal: false }	external transition to child node	explicit
EVENT: { actions: [ ... ], internal: false }	external self-transition	explicit
EVENT: { target: undefined, actions: [ ... ], internal: false }	external self-transition	explicit

from:

• https://xstate.js.org/docs/guides/transitions.html#internal-transitions
• https://xstate.js.org/docs/guides/transitions.html#external-transitions

The docs need some clarification

Contradicting this, are these statements:

• "By default, all transitions with a specified target are external." (https://xstate.js.org/docs/guides/transitions.html#self-transitions)
• "By default, transitions are external, ..." (https://xstate.js.org/docs/guides/transitions.html#external-transitions)
• "All transitions are external by default." (https://xstate.js.org/docs/guides/actions.html#actions-on-self-transitions)

[johnyanarella]

I don't think it is a good idea to default to internal and rely on the algorithm to behave as external whenever internal is invalid. Regardless of what the DSL chooses to capture or how it conveys "auto", I think the resulting transition nodes will need to hold the resolved internal vs external result.

Otherwise, this is going to come back to bite you when:

• you are debugging a statechart and looking at the transition node in your debugger to see the result ("why is this performing an external transition? it says internal right there!")
• you are visualizing a statechart in the inspector and want to depict external vs internal visually ("why is this drawn as an internal transition? when it runs it performs an external transition!")

[johnyanarella]

I think you can address the astonishing behavior for both "common handler" examples without introducing anything new to the DSL, by tweaking the "auto" behavior with some additional heuristics that inspect the states' relationships rather than just the target syntax.

However, there are two obstacles:

• .b vs b or #b or get target() { return m.states.b } currently behave differently
  • changing the "auto" behavior here to consider whether the target resolves to a child would be a breaking change
  • some of the documented examples would need to change
• actually performing those heuristics gets more difficult
  • startsWith(stateNode.machine.delimiter) won't be enough anymore
  • even traversing the children of the current state definition won't be enough (because the final composed node ids for states that omit id include ancestor keys)
  • you'd have to resolve the target against a lookup from the completed machine node hierarchy

[Andarist]

you are debugging a statechart and looking at the transition node in your debugger to see the result ("why is this performing an external transition? it says internal right there!")

Based on what observable behavior this could trip the user?

you are visualizing a statechart in the inspector and want to depict external vs internal visually ("why is this drawn as an internal transition? when it runs it performs an external transition!")

This can actually just be drawn differently based on the relation of source and target states to avoid confusion (and not based on the internal/type property)

[johnyanarella]

The first does presuppose that a developer has a bug in their machine (entry and exit actions are unexpectedly occurring on what they thought should be an internal transition, they're so confused now, and totally ready to flip their table over...) and they are using a debugger to inspect the state node properties of machine.states.*.

Granted, the XState inspector would (hopefully) be a better tool to debug this situation. Good warnings / lint errors would be even better.

So, yes, you could derive the actual transition type as needed, but I thought we were all about making states explicit? 😉

"So much complexity in software comes from trying to make one thing do two things." – Ryan Singer

By not wrangling that complexity (user's stated intent: auto vs internal vs external) at its source, it will propagate throughout the codebase.

• 🙂 "This transition definition is explicitly marked as internal, but it can't be internal, so we should log a warning."
• 😜 "This transition definition automatically defaulted to internal, but will really be external, and we don't want to warn when the user didn't explicitly say internal is true."
• 🤪 "This transition node says it is internal, but when selecting candidate transitions we need to ignore what it says and treat it as external because of the source/target state relationship."
• 😯 "This transition node says it is internal, but in this separate tool we should actually render it as external because of the source/target state relationship."
• 😳 ...on and on, spiraling further away from the original declaration.

Anyway, that's all internal implementation details, and you are way more familiar with the maintenance impacts and trade-offs in the JS engine implementation than me—this is all just a suggestion that might not make as much sense there.

[Andarist]

Well - I see how maybe one could be a little bit confused when it comes to exit actions, entry actions on the other hand are rather hard to be confused as they would just be from a different part of the document.

I don't mind surfacing "incorrect" internal transitions early - it makes sense to me. I'm only not sure if this is that much beneficial to include "auto" anywhere explicitly, it would require documentation and explanation for people that actually know SCXML/UML semantics. It's a new term to be learned. Wouldn't just treating lack of the internal property as "auto" suffice? In such scenario we'd only have to surface "incorrect" internal transitions when internal: true would be explicitly configured for a particular transition.

[johnyanarella]

Intriguing... to your point about the DSL not necessarily requiring an understanding of SCXML semantics:

What if the DSL exposed reenter instead of internal? The DSL would now be talking in terms of observable behavior (“will entry/exit actions fire again?”). Granted, it flips the Boolean. Omitted, it would default to false.

(New name might ease migration to new behavior, too.)

Assuming the default behavior is now to be an internal transition when targeting oneself or children, and external for anything else, this would just be used for those edge cases where you do want to re-enter yourself or a child (use external instead) or explicitly not re-enter an ancestor (use internal instead).

(Is that sound logic - can you use internal for an ancestor? Sorry, just woke up and no coffee yet!)

And you could throw helpful errors if it was used for an invalid source / target configuration.

The relationship to internal and external in SCXML could be explained in the SCXML section at the end of the relevant doc, as elsewhere.

[Andarist]

What if the DSL exposed reenter instead of internal? The DSL would now be talking in terms of observable behavior (“will entry/exit actions fire again?”). Granted, it flips the Boolean. Omitted, it would default to false.

I like it quite a bit!

Assuming the default behavior is now to be an internal transition when targeting oneself or children, and external for anything else, this would just be used for those edge cases where you do want to re-enter yourself or a child (use external instead) or explicitly not re-enter an ancestor (use internal instead).

Hmm, I would be for removing internal completely - in favor of reenter. IMHO better to just keep a single prop that could influence this behavior. The algorithm would still mirror the one described in the SCXML spec, just slightly adjusted to use a different prop. We could surface "invalid" reenter: true to users in a form of dev warning that this doesn't really change the behavior of the given transition. This would basically leave us with type: 'auto' | 'external' | 'internal' (just using a different prop). I would probably not change reenter: false with "invalid" combination of source & target states to be an error though - it's easier to treat this as SCXML-compatible if we don't throw errors in those situations.

[johnyanarella]

Ah, yeah, was referring to the resulting transition behavior in the second quote.

Exactly, suggesting “internal” as an attribute in the JS API config object goes away.

[johnyanarella]

Good point that you want invalid <transition type=“internal” ... /> originating from SCXML to remain silent (no error).

I’d only suggest throwing an error for invalid use of the DSL “reenter” property. But, I definitely see why it gets tricky to distinguish, as you’re likely well past the origin of the value at the point where you’d have enough info to do that validation.

[johnyanarella]

...or maybe it’s always a warning.

Edit: oh, you said that! Yes, total agreement, then!