Add 'optional' imports/exports

Author: lukewagner

design/mvp/Binary.md

@@ -232,7 +232,9 @@ instancedecl  ::= 0x00 t:<core:type>                      => t
                 | 0x04 ed:<exportdecl>                    => ed
 importdecl    ::= na:<nameattributes> et:<externtype>     => (import na et)
 exportdecl    ::= na:<nameattributes> et:<externtype>     => (export na et)
-externtype    ::= 0x00 0x11 i:<core:typeidx>              => (core module (type i))
+externtype    ::= 0x10 t:<someexterntype>                 => (optional t)
+                | t:<someexterntype>                      => t
+someexterntype ::= 0x00 0x11 i:<core:typeidx>             => (core module (type i))
                 | 0x01 i:<typeidx>                        => (func (type i))
                 | 0x02 b:<valuebound>                     => (value b) 🪙
                 | 0x03 b:<typebound>                      => (type b)
@@ -287,6 +289,8 @@ Notes:
   `none` case of an optional immediate.)
 * 🔧 for fixed-sized lists the length of the list must be larger than 0 to pass
   validation.
+* ❓ validation of `optional` rejects everything except `func`, `instance` and
+  `resource` types (this may be relaxed later).
 
 
 ## Canonical Definitions
@@ -541,6 +545,7 @@ named once.
   subset of `canonopt`s.
 * Add optional `shared` immediate to all canonical definitions (explicitly or
   via `<canonopt>`) when shared-everything-threads (🧵②) is added.
+* Improve `externtype` opcodes for `optional`
 
 
 [`core:byte`]: https://webassembly.github.io/spec/core/binary/values.html#binary-byte

design/mvp/CanonicalABI.md

@@ -68,6 +68,7 @@ specified here.
   * [`canon thread.yield-then-resume`](#-canon-threadyield-then-resume) 🧵
   * [`canon thread.suspend-then-promote`](#-canon-threadsuspend-then-promote) 🧵
   * [`canon thread.yield-then-promote`](#-canon-threadyield-then-promote) 🧵
+  * [`canon optional.test`](#-canon-optionaltest) ❓
   * [`canon error-context.new`](#-canon-error-contextnew) 📝
   * [`canon error-context.debug-message`](#-canon-error-contextdebug-message) 📝
   * [`canon error-context.drop`](#-canon-error-contextdrop) 📝
@@ -2567,6 +2568,7 @@ possible, for example if the index was actually a `borrow` or if the `own`
 handle is currently being lent out as borrows.
 ```python
 def lift_own(cx, i, t):
+  ;; TODO: assert(not isinstance(t, Optional))
   h = cx.inst.handles.remove(i)
   trap_if(not isinstance(h, ResourceHandle))
   trap_if(h.rt is not t.rt)
@@ -2586,6 +2588,7 @@ from the source handle, leaving the source handle intact in the current
 component instance's `handles` table:
 ```python
 def lift_borrow(cx, i, t):
+  ;; TODO: assert(not isinstance(t, Optional))
   assert(isinstance(cx.borrow_scope, Subtask))
   h = cx.inst.handles.get(i)
   trap_if(not isinstance(h, ResourceHandle))
@@ -3025,10 +3028,12 @@ elements in the current component instance's `handles` table. The increment of
 ensures that all borrowed handles are dropped before the end of the task.
 ```python
 def lower_own(cx, rep, t):
+  ;; TODO: assert(not isinstance(t, Optional))
   h = ResourceHandle(t.rt, rep, own = True)
   return cx.inst.handles.add(h)
 
 def lower_borrow(cx, rep, t):
+  ;; TODO: assert(not isinstance(t, Optional))
   assert(isinstance(cx.borrow_scope, Task))
   if cx.inst is t.rt.impl:
     return rep
@@ -3556,14 +3561,16 @@ performed for a component. These are defined as:
 
 For a canonical definition:
 ```wat
-(canon lift $callee:<funcidx> $opts:<canonopt>* (func $f (type $ft)))
+(canon lift $callee:<core:funcidx> $opts:<canonopt>* (func $f (type $ft)))
 ```
 
 In addition to [general validation of `$opts`](#canonopt-validation) the additional
 validation is performed:
 
+TODO: can `$ft` be `optional`?
+
 * `$callee` must have type `flatten_functype($opts, $ft, 'lift')`
-* `$f` is given type `$ft`
+* `$f` is given type `$ft` (TODO: must be non-`optional` `func` type)
 * if a `post-return` is present, it has type `(func (param flatten_functype($opts, $ft, 'lift').results))`
 * requires options based on [`lift(param)`](#canonopt-validation) for all parameters in `ft`
 * requires options based on [`lower(result)`](#canonopt-validation) if `ft` has a result
@@ -3789,8 +3796,10 @@ For a canonical definition:
 ```
 
 In addition to [general validation of `$opts`](#canonopt-validation), additional
-validation is performed where `$callee` has type `$ft`:
+validation is performed where `$callee` has type `$ft'`:
 
+* `$ft'` must either be of the form `(func ...)` or `(optional (func ...))`
+  * let `$ft` be the `(func ...)` (sub)expression of `$ft'`
 * `$f` is given type `flatten_functype($opts, $ft, 'lower')`
 * requires options [based on `lower(param)`](#canonopt-validation) for all parameters in `ft`
 * requires options [based on `lift(result)`](#canonopt-validation) if `ft` has a result
@@ -3814,9 +3823,11 @@ handles are not dropped before `Subtask.deliver_resolve` is called (below).
 def canon_lower(callee, ft, opts, flat_args: list[CoreValType]) -> list[CoreValType]:
   thread = current_thread()
   trap_if(not thread.task.inst.may_leave)
+  trap_if(callee is None)
   subtask = Subtask()
   cx = LiftLowerContext(opts, thread.task.inst, subtask)
 ```
+TODO: mention that `callee is None` can only happen if `$ft'` was `optional`.
 
 The next chunk makes the call to `callee` using the `opts` immediates of the
 `canon lower` definition to configure `lift_flat_values` and `lower_flat_values`
@@ -5097,6 +5108,19 @@ prepared to propagate cancellation, they can omit `cancellable` so that
 cancellation is instead delivered at a later `cancellable` call.
 
 
+### ❓ `canon optional.test`
+
+For a canonical definition:
+```wat
+(canon optional.test $sortidx (core global $present))
+```
+validation specifies:
+* `$sortidx` must ... and be `optional`
+* `$optional.test` is given type `(global i32)`
+
+TODO: describe
+
+
 ### 📝 `canon error-context.new`
 
 For a canonical definition:

design/mvp/Explainer.md

@@ -21,12 +21,14 @@ more user-focused explanation, take a look at the
     * [Specialized value types](#specialized-value-types)
     * [Definition types](#definition-types)
     * [Declarators](#declarators)
+    * [Optional types](#optional-types)
     * [Type checking](#type-checking)
   * [Canonical definitions](#canonical-definitions)
     * [Canonical ABI](#canonical-abi)
     * [Canonical built-ins](#canonical-built-ins)
       * [Resource built-ins](#resource-built-ins)
       * [Concurrency built-ins](#-concurrency-built-ins)
+      * [Feature-test built-in](#-feature-test-built-in)
       * [Error Context built-ins](#-error-context-built-ins)
   * [Value definitions](#-value-definitions)
   * [Start definitions](#-start-definitions)
@@ -69,6 +71,7 @@ subsequent WASI Developer Preview minor releases:
 * 🐘: [memory64]
 * 🗺️: the `map` type
 * 🏷️: `implements` annotations for plain-named interface imports/exports
+* ❓: `optional` imports and exports
 
 
 ## Grammar
@@ -607,9 +610,9 @@ valtype       ::= <typeidx>
 keytype       ::= bool | s8 | u8 | s16 | u16 | s32 | u32 | s64 | u64 | char | string 🗺️
 resourcetype  ::= (resource (rep i32) (dtor <core:funcidx>)?)
                 | (resource (rep i64) (dtor <core:funcidx>)?) 🐘
-functype      ::= (func async? (param <labellit> <valtype>)* (result <valtype>)?)
+functype      ::= (func optional? async? (param <labellit> <valtype>)* (result <valtype>)?)
 componenttype ::= (component <componentdecl>*)
-instancetype  ::= (instance <instancedecl>*)
+instancetype  ::= (instance optional? <instancedecl>*)
 componentdecl ::= <importdecl>
                 | <instancedecl>
 instancedecl  ::= core-prefix(<core:type>)
@@ -624,9 +627,8 @@ externtype    ::= (<sort> (type <u32>) )
                 | <componenttype>
                 | <instancetype>
                 | (value <valuebound>) 🪙
-                | (type <typebound>)
-typebound     ::= (eq <typeidx>)
-                | (sub resource)
+                | (type (eq <typeidx>))
+                | (type optional? (sub resource))
 valuebound    ::= (eq <valueidx>) 🪙
                 | <valtype> 🪙
 
@@ -1009,6 +1011,14 @@ definitions:
 Note that the inline use of `$G` and `$U` are syntactic sugar for `outer`
 aliases.
 
+#### Optional types
+
+TODO:
+* `alias export` produces `optional` unless `eq`
+* `with` checks ...
+* ascription checks ...
+
+
 #### Type Checking
 
 Like core modules, components have an up-front validation phase in which the
@@ -1091,6 +1101,9 @@ and `$C1` is a subtype of `$C2`:
 checking and so any `external-id`, `implements` or `versionsuffix` attribute can
 be added to any of the `func`s above without changing the result.
 
+TODO: also mention `optional`
+
+TODO: no `typebound`
 When we next consider type imports and exports, there are two distinct
 subcases of `typebound` to consider: `eq` and `sub`.
 
@@ -1298,6 +1311,9 @@ replaced by `$R` when validating the instantiations of `$c1` and `$c2`. These
 type-checking rules for instantiating type imports mirror the *elimination*
 rule of [universal types]  (∀T).
 
+TODO: if don't provide type, substitute incompatible type; prevent functions
+from being supplied...
+
 Importantly, this type substitution performed by the parent is not visible to
 the child at validation- or run-time. In particular, there are no runtime
 casts that can "see through" to the original type parameter, avoiding
@@ -1545,6 +1561,7 @@ canon ::= ...
         | (canon thread.yield-then-resume cancellable? (core func <id>?)) 🧵
         | (canon thread.suspend-then-promote cancellable? (core func <id>?)) 🧵
         | (canon thread.yield-then-promote cancellable? (core func <id>?)) 🧵
+        | (canon optional.test <sortidx> (core global <id>?)) ❓
         | (canon error-context.new <canonopt>* (core func <id>?)) 📝
         | (canon error-context.debug-message <canonopt>* (core func <id>?)) 📝
         | (canon error-context.drop (core func <id>?)) 📝
@@ -2331,6 +2348,11 @@ For details, see [`canon_thread_available_parallelism`] in the Canonical ABI
 explainer.
 
 
+##### ❓ Feature-test built-in
+
+TODO: describe `canon optional.test`
+
+
 ##### 📝 Error Context built-ins
 
 ###### 📝 `error-context.new`

design/mvp/WIT.md

@@ -308,6 +308,8 @@ world union-my-world {
 }
 ```
 
+TODO: add `optional` to examples
+
 ### De-duplication of interfaces
 
 If two worlds share an imported or exported [interface name], then the union of
@@ -339,6 +341,8 @@ world union-my-world-b {
 }
 ```
 
+TODO: mention merging of `optional` and non-`optional`
+
 ### Name Conflicts and `with`
 
 When two or more included Worlds have the same name for an import or export
@@ -438,6 +442,7 @@ world invalid-union-world {
 
 ### A Note on Subtyping
 
+TODO: update this
 In the future, when `optional` export is supported, the world author may explicitly mark exports as optional to make a component targeting an included World a subtype of the union World.
 
 For now, we are not following the subtyping rules for the `include` statement. That is, the `include` statement does not imply any subtyping relationship between the included worlds and the union world.
@@ -563,6 +568,8 @@ interface my-interface {
 }
 ```
 
+TODO: `use` from all-`optional`-imports/export --> `optional`
+
 #### Top-level `use`
 
 If a package being referred to has a version number, then using the above syntax
@@ -1448,9 +1455,11 @@ import-item ::= external-id? 'import' id ':' extern-type
 
 external-id ::= '@external-id' '(' string-literal ')' 🏷️
 
-extern-type ::= func-type ';'
-              | 'interface' '{' interface-items* '}'
-              | use-path ';' 🏷️
+extern-type      ::= 'optional' some-extern-type
+                   | some-extern-type
+some-extern-type ::= func-type ';'
+                   | 'interface' '{' interface-items* '}'
+                   | use-path ';' 🏷️
 ```
 
 🏷️ While `id`s are restricted to kebab-case (to enable idiomatic bindings for
@@ -1508,6 +1517,8 @@ world my-world {
 }
 ```
 
+TODO: note `optional` and what it means
+
 [`componenttype`]: Explainer.md#type-definitions
 
 ## Item: `include`