Proposal: guiding principle for when `void` should be allowed

[stereotype441]

Background

The language spec currently has an "allow list" indicating the circumstances in which an expression is allowed to have a static type of void; any other circumstance in which an expression ever has a static type of void is considered to be a compile-time error.

Although the spec's allow list is quite long, it doesn't cover all the situations in which the implementations allow void. For example:

• the front end and analyzer allow the branches of a switch expression to have type void; the spec's allow list doesn't say anything about switch expressions.
• the front end and analyzer allow the RHS of a pattern assignment to be void. The spec doesn't allow this.

There are also several cases where the front end and analyzer disagree about whether void should be allowed:

• The front end allows the receiver of [] and []= to have a static type of void (assuming a suitable extension exists that defines these operators, e.g. extension on Object? { operator [](index) => 0; }). The spec and the analyzer do not allow this.
• The front end allows the scrutinee of a switch statement to have a static type of void. The spec and the analyzer do not allow this.
• The front end doesn't allow the index of a compound index set to be void (e.g., foo[voidValue]++), even if the corresponding operator [] and operator []= declarations define the type of their index parameter to be void. The spec and the analyzer do allow this.

Part of the reason for the discrepancies is that the spec's "allow list" approach doesn't correspond very closely with how the "void can't be used here" errors are implemented in the analyzer and the front end:

• In the front end, the code for analyzing a subexpression accepts an isVoidAllowed parameter that defaults to false (so the exceptions to this default do, in effect, constitute an allow list), but there are a large number of cases where true is passed to isVoidAllowed, even though void is actually not allowed, in order to avoid issuing redundant error messages. For example, the visitDoStatement method passes isVoidAllowed: true when analyzing the condition of the do/while loop, because the invalid use of void is caught by a later call to ensureAssignableResult (which makes sure the loop condition is a boolean). Because of this approach, there's a danger than when adding new features to the language, we will accidentally copy isVoidAllowed: true to places where it doesn't belong. (This appears to have happened during the implementation of patterns, for example.)
• In the analyzer, the code for reporting invalid uses of void exists in many ad hoc places in the resolver, one for each circumstance in which an expression is not allowed to have a static type of void. Essentially this acts as a "deny list"; this has the effect that when adding new features to the language, we are liable to forgot to add the appropriate void checks. (This also appears to have happened during the implementation of patterns.)

Also possibly of interest: project https://github.com/orgs/dart-lang/projects/111/views/1

After talking with several folks about the situation, I believe we would be better served if we agreed upon a general guiding principle for when an expression is allowed to have static type void. The implementations should implement that general principle directly, rather than having a bunch of "allow" or "deny" cases. The spec should also state the general principle directly.

Proposal

Here's my proposal for what the general principle could be:

• Every place where an expression may appear in the grammar is considered to be an assigned expression, a dispatched expression, a discarded expression, or a relayed expression, based on the runtime semantics of the containing construct:
  • When the runtime semantics of the containing construct are to assign the value of the expression to a variable or parameter, or store it in a data structure, the expression is considered an assigned expression. Examples include the initializer of a variable declaration, the argument to a method call, the RHS of a user-definable binary operator, and the elements of a collection literal. Assigned expressions are always associated with an explicit assignability check. They are permitted to have a static type of void if and only if the destination type in the assignability check is void. So, for example, in void x = foo;, foo may have a static type of void, but in Object? x = bar;, bar may not.
  • When the runtime semantics of the containing construct depend on the specific value of the expression (or its type), the expression is considered a dispatched expression. Examples include the receivers of method invocations, the boolean conditions of flow control constructs, and the operands of is, as, and await expressions. Dispatched expressions may never have a static type of void.
  • When the runtime semantics of the containing construct are to ignore the value of the expression, the expression is considered a discarded expression. Examples include expression in an expression statement, and the updaters in a for loop. Discarded expressions are permitted to have a static type of void (in the absence of other errors of course).
  • When the runtime semantics of the containing construct are for the value of the (sub)expression to become the value of the containing expression, the subexpression is considered a relayed expression. Examples include the subexpression of a parenthesized expression, the branches of a switch or conditional expression, and the RHS of ??. Relayed expressions are permitted to have a static type of void (in the absence of other errors). For example, x ?? voidExpr; is ok, but Object? y = x ?? voidExpr; is not (because x ?? voidExpr has type void).

I'm circulating this to get an initial impression from the language team of whether this sounds like a good idea before embarking on additional work. Before going through with the idea, I would want to prototype it and run it through google3 to make sure it's not too breaking.

[stereotype441]

CC @dart-lang/language-team

[leafpetersen]

Thanks! Great writeup.

Minor comment:

• but in int x = bar;, bar may not.

Probably Object? x = bar or dynamic x = bar would be a better example, since the original example is an error for other reasons?

The proposal itself seems reasonable to me, modulo potential breakage.

[lrhn]

Seems reasonable, if we can put every expression into precisely one of those four boxes.

• So, cascades! The receiver of a cascade expression is a sub-expressions whose value becomes the value of the entire expression, so it qualifies as a relayed expression, but it is also a receiver, so a dispatched expression. Should be a "two yes, one no" decision. It's a receiver, receivers cannot be void. In general, if an expression matches ine category that disallows void, that wins.

-And the end of a cascade selector chain is discarded.

• assignments have the RHS as both relayed and assigned. Both can accept void depending on context (and only one if the contexts is the context type). Treat as assignment, then propagate the void type out as normal for a relayed expression.

• Operators. Need to look at each one. The first operand is always a receiver.

  • ==, !=: does a null check on the second operand, so it's dispatched.
  • +, -, etc. user definable: can treat second operand as assignment.
  • !(prefix), &&, ||: moot, must be bool.
  • ??: as mentioned, first operand is dispatched, second is relayed.
  • +=, etc: second operand/RHS is assignment.
    • Plus(!) if the operator returns void, that value is an assignment. There is no single expression which has that value.
  • ??=: same
  • ?/:: first operand dispatched, and bool, rest relayed.
  • suffix!: operand is dispatched.
  • [], []=: receiver is receiver, index and value are assignment.
  • ...[...]+=: same as += above, the value of the binary operator is assigned to the setter parameter.
  • interpolation expression: dispatched.
  • is, as: dispatched.

• pattern expressions. Depends on the pattern?

  • expr or const expr: is a receiver for ==.
  • == expr, != expr, as operand of == (dispatched).
  • > e, ...<= e: as assignment to operator parameter of matched value.

• Pattern matched value expressions: switch expressions, expression before case of if, initializer expression of pattern declaration, or RHS of pattern assignments,... it depends on the pattern.

  • A pattern treats it's matched value as one of the four kinds, and it's an error if the matched value type is void and the matched value is dispatched.
  • _ discards
  • var id assigns ()
  • Type id assigns if Type is supertype of the MVT (required for ), dispatches if not. So error if MVT is void and Type is not.
  • record, list or map pattern dispatches.
  • object pattern dispatches if not supertype of MVT or if it has any property extractions.
  • id- variable pattern in pattern assignment, assign.
  • as, ! and ? patterns dispatch.

• Elements of literals:

  • expression elements are assignment, to the elegant type (or key/value type of maps)
  • expressions of spread, null-aware and null-aware spread elements dispatch.
  • ecpressions in if and for elements work the same as for statements, their nested elements are relayed.

The point here is that it's not just expressions that are not allowed to be void, it's values.
There are values that we recognize, but which do not have a source expression producing them. There are operations performed on objects without having an expression at all:

• e1 += e2, the result of invoking + is a value which is assigned to e.
• a pattern Foo(bar: p), the results of invoking Foo.bar on a value is a new value. It's context, which determines how it's used,t is the pattern p.
• for (v in e)....the value of e.iterator.current is a value which is assigned to v.

[stereotype441]

@leafpetersen

Minor comment:

• but in int x = bar;, bar may not.

Probably Object? x = bar or dynamic x = bar would be a better example, since the original example is an error for other reasons?

Good point, thanks. I've changed the proposal to say Object? x = bar.

[stereotype441]

We discussed this in today's language meeting and decided to move forward with an experimental implementation to measure potential breakage.

Additional notes from the discussion:

• Consider using "position" nomenclature, e.g. "an expression in an assigned position / a dispatched position / etc."
• A nice intuitive way of understanding the "assigned position" and "relayed position" categories is that the runtime semantics pass the value through "without looking at it", therefore allowing void to pass through doesn't break any contract.
• Don't forget to think about cascades:
  • Is the expression before the first .. in a relayed position (because the value of the cascade is the same as the value of the expression before the first ..)? Or is it in a dispatched position (because the behavior of selectors after each .. depends on the runtime type of the value)?
  • The end of each selector chain that follows .. is in a discarded position.
• Don't forget to think about complex forms like e1 += e2 (see @lrhn's comment above). In general we don't like to specify things in terms of desugaring, but it probably makes sense to think about desugaring when we figure out how to apply our general principles.
• Don't forget to also think about what things are allowed to flow into void (we should be careful not to change these rules by accident). Consider in particular the rules for return; and return e; statements in:
  • sync functions with return type void vs. non-void.
  • async functions with return type void vs. Future<void> vs. other return types.
• We discussed how the operand of an as expression should be treated. We didn't reach a consensus, but the following points were raised:
  • It's unclear how the proposal should apply to an as expression: in x as T, the position of x has both the properties of a dispatched position (in that x as T behaves differently depending on the type of x) and of a relayed position (since the value of x as T is the same as the value of x).
  • As a practical matter, x as Object? is the obvious "escape hatch" to allow "voidness" to be removed from a type that was incorrectly classified as void; @eernstg expressed concern about taking this away.
    • Note that there are other escape hatches available, since a cast can be removed from any compound type using as. But none of them are as straightforward as a simple cast. For example, if we prohibit x as Object? from casting void to Object?, a user could always do something like (<void>[x] as List<Object?>).first instead.
  • On the other hand, @lrhn suggested that anyone using this escape hatch is probably doing the wrong thing: either they own code that produced the void value, in which case they can always change it to have a different type, or they don't own the code, in which case they are breaking the interface contract if they try to inspect the value. So maybe if we make it more difficult to cast from void to Object?, that's a good thing.
  • I said I would run an experiment to see how breaking this particular part of the proposal is, and then bring that data to the language team so we can discuss further.

[stereotype441]

Regarding the question of whether to allow the operand of as to be void, I ran an experiment in Google3. TL/DR, I now think it would be better to continue allowing the operand of as to be void.

Details

I made a lint that fires if the operand of `as` has a static type of `void`; that lint was tripped by 217 places in Google3 code, spanning a variety of projects. Of these 217, 9 were in open source code.

Of those 9, two were especially interesting to look at:

• https://github.com/google/dart_cli_script/blob/7f5d1dbb68e2ed87c9813332e80c01da69ee2688/lib/src/script.dart#L701. In this example, the code uses Future.wait to wait for two futures to complete, one of which has type Future<void> and the other of which is Future<String>. Because of LUB rules, this results in a Future<List<void>>, which is then awaited to produce List<void>. But the user knows that one of the elements of the list is a String, so they extract it and cast it using as. This seems like a legit use of the "escape hatch" to me: the type system can't tell which elements of the list are safe to access, so to be safe it considers it to be a List<void>. But the user knows that one list entry is a String, and thus safe to access, so they gain access to it using as.
  • Note that since Dart 3.0, we have offered various extension methods that do the same job as Future.wait but with better type safety (example); this could could be rewritten to use one of those extension methods and it wouldn't need to do the cast anymore.
• https://github.com/flutter/flutter/blob/a443575b65a2f2fa4867f8275968f7b8047cfce9/packages/flutter/lib/src/widgets/framework.dart#L1199. This example is from the setState method in the flutter framework. The implicit contract of setState is that the callback passed in by the user needs to perform the state updates synchronously. There's no 100% reliable way to check that contract using the type system, but we can at least check that the user didn't accidentally mark their callback function as async by verifying that it didn't return a Future. To verify that, we need to cast away the "voidness" of the callback's return type. This also seems to me like a legit use of the "escape hatch".

In all, 6 of the 9 places in open source code that tripped the lint seemed to me like legit uses of the "escape hatch".

Then I did a random sampling of 10 of the places in closed source that tripped the lint. Of those, 6 of them were instances of the aforementioned Future.wait pattern, and thus IMHO legit.

That's enough legit use cases to convince me that it's not worth trying to close this loophole in the type system.

[lrhn]

The Flutter setState use is heuristic, assuming that a void function, which makes no promises about the actual value returned, won't ever return a Future without it being bug.

That's probably a pretty good assumption. Futures should be handled, unless you know they don't need it. A Future being returned from a void function is definitely not intended to be handled, so likely it's a bug.

At least in the case of a function that must not be asynchronous, and shouldn't schedule anything for later. (Otherwise a void foo() async {…} can be a fine way to schedule something for later, without the caller waiting for it.)

So ... a mostly reasonable use of type-checking a void value. Not to use it, that would be bad, but to detect a common mistake.

That's using is, not as. It's not really much difference if it promotes.
(Could we make void not be promotable? Then allow is, but not as?)

The wait uses sounds like something that could have been fixed in any number of better ways.
Even just Future.wait<dynamic>([voidFuture, stringFuture]) could have been used all along, without using the new, and better typed, alternatives. Even if its "legit", meaning not trying to circumvent void protection, but just reinstating a lost non-void type, it's was never a good pattern.

The most convincing thing here is actually that people don't try to circumvent void often.
The benefits of disallowing it would be equally small.
(But there are also workarounds, like (v,) as (Object?,) is (Future,), and it's not that many places that would need them.)

[stereotype441]

The Flutter setState use is heuristic, assuming that a void function, which makes no promises about the actual value returned, won't ever return a Future without it being bug.

That's probably a pretty good assumption. Futures should be handled, unless you know they don't need it. A Future being returned from a void function is definitely not intended to be handled, so likely it's a bug.

At least in the case of a function that must not be asynchronous, and shouldn't schedule anything for later. (Otherwise a void foo() async {…} can be a fine way to schedule something for later, without the caller waiting for it.)

So ... a mostly reasonable use of type-checking a void value. Not to use it, that would be bad, but to detect a common mistake.

That's using is, not as. It's not really much difference if it promotes. (Could we make void not be promotable? Then allow is, but not as?)

Today, the operand of is is not allowed to be void. So what's actually happening in setState is that it's using as to cast from void to Object?, so that it can use is later.

I suppose we could change things so that the operand of is is allowed to be void (and further, make void not promotable, as you suggest). Then we could rewrite this setState logic so that it didn't use as. But I don't love it, because (a) it's a lot of churn for code that's already working, and (b) I feel like the current state of affairs where we permit void as T is easy to explain: as is what you use when you know what the runtime type is, and the static type analysis isn't serving you; casting away a void fits perfectly into this pattern. Whereas I can't think of a justification for allowing void is T that seems convincing to me.

[munificent]

SGTM, ship it!