wasmtime_wasi call_run() return type is weird and undocumented and doesn't provide access to the exit code

[Timmmm]

I'm running a WASM module like this:

let result_result = command.wasi_cli_run().call_run(&mut store).await;

call_run() is unfortunately completely undocumented ("Run the program" doesn't add anything), and the return type is very unobvious: Result<Result<(), ()>>. Expanding the Results it's actually anyhow::Result<std::result::Result<(), ()>>.

So I think there are two sources of possible error from this run, which probably explains the nested Results:

1. Some kind of error loading the module or with the bytecode or whatever.
2. The module returns a non-zero exit code.

I assumed that the inner Result<(), ()> would handle the latter case, but actually non-zero exit codes result in the outer anyhow::Result<> containing the error.

That's very surprising and kind of awkward because there's no way to access the exit code either. I would have expected a return type of anyhow::Result<ExitStatus> surely?

[Timmmm]

Actually you can get the exit code like this, but it's still very weird.

    let run_result = command.wasi_cli_run().call_run(&mut store).await;

    match run_result {
        Ok(res) => res.map_err(|_| anyhow!("Unknown error"))?,
        Err(error) => {
            if let Some(exit) = error.downcast_ref::<I32Exit>() {
                info!("Call failed with exit code {:?}", exit.0);
                return Ok(false);
            }
            return Err(error);
        }
    };

[alexcrichton]

You'll want to keep in mind that the API you linked is generated code, procedurally from a WIT file using the bindgen! macro. If you're interested I think it'd make sense to add docs to the top-level wasmtime-wasi crate, but it's not currently possible to add extra Rust-specific docs to each function.

As for the actual specifics of what's going on here, it is:

• Ok(Ok(())) - the guest executed successfully and returned an "ok" return status from main
• Ok(Err(())) - the guest executed successfully and returned an "err" return status from main
• Err(e) - the guest aborted execution via a trap of some kind or a host error
  • As a sub-case of this, as you've found, the I32Exit structure can be used to detect explicit exit codes from the guest.

If you're interested in adding extra helpers to this type that'd be most welcome! We can have custom impl Guest blocks within the wasmtime-wasi crate to add more methods to bindgen!-generated methods.

[Timmmm]

What's the difference between an "err" status returned from main and a non-zero exit code?

And that's unfortunate about the auto-generation. Seems like Run the program. comes from a .wit file so it probably doesn't make sense to add docs about Rust there. Is there a way to inject extra documentation somehow?

[primoly]

The run function simply returns result (shorthand for result<_, _>), so it just indicates success or failure.¹ To return an exit code, a command component has to call exit-with-code inside run.² As the docs say, exit-with-code will not return from the component’s point of view, so wasmtime-wasi instead returns Err(I32Exit<status_code>) to propagate the error code up to the caller of call_run.³

Example of a component exiting with status code 77

(component
  (type (;0;)
    (instance
      (type (;0;) (func (param "status-code" u8)))
      (export (;0;) "exit-with-code" (func (type 0)))
    )
  )
  (import "wasi:cli/exit@0.2.5" (instance (;0;) (type 0)))
  (core module (;0;)
    (type (;0;) (func (param i32)))
    (type (;1;) (func (result i32)))
    (import "cm32p2|wasi:cli/exit@0.2" "exit-with-code" (func $exit-with-code (;0;) (type 0)))
    (export "cm32p2|wasi:cli/run@0.2|run" (func $run))
    (func $run (;1;) (type 1) (result i32)
      i32.const 77
      call $exit-with-code ;; This will never return.
      unreachable
    )
  )
  (alias export 0 "exit-with-code" (func (;0;)))
  (core func (;0;) (canon lower (func 0)))
  (core instance (;0;)
    (export "exit-with-code" (func 0))
  )
  (core instance (;1;) (instantiate 0
      (with "cm32p2|wasi:cli/exit@0.2" (instance 0))
    )
  )
  (type (;1;) (result))
  (type (;2;) (func (result 1)))
  (alias core export 1 "cm32p2|wasi:cli/run@0.2|run" (core func (;1;)))
  (func (;1;) (type 2) (canon lift (core func 1)))
  (component (;0;)
    (type (;0;) (result))
    (type (;1;) (func (result 0)))
    (import "import-func-run" (func (;0;) (type 1)))
    (type (;2;) (result))
    (type (;3;) (func (result 2)))
    (export (;1;) "run" (func 0) (func (type 3)))
  )
  (instance (;1;) (instantiate 0
      (with "import-func-run" (func 1))
    )
  )
  (export (;2;) "wasi:cli/run@0.2.5" (instance 1))
)

On the topic of the outer Result: Maybe it would make sense for bindgen generated functions to return something like Result<T, ComponentExecutionError> making it clearer what it’s for and how it differs from the returned value of the component function (result or not).

Footnotes

1. https://github.com/WebAssembly/wasi-cli/blob/main/wit/run.wit ↩

2. https://github.com/WebAssembly/wasi-cli/blob/main/wit/exit.wit ↩

3. https://github.com/bytecodealliance/wasmtime/blob/9d40c6eb67d0d791c68fae46a2059647ca0e5dbc/crates/wasi/src/p2/host/exit.rs#L16-L18 ↩

[alexcrichton]

Can indeed confirm that as @primoly said the difference is returned-from-main vs exited-with-wasi-function.

Currently there's no way to edit the documentation of generated methods. There's workarounds to prepend documentation to generated types but that's all I've figured out how to do at least.

[Timmmm]

Ok even weirder, if you exit(0) then it still calls exit-with-code and you get an Err(I32Exit(0)) which is really surprising. So actually I need to do:

    let run_result = command.wasi_cli_run().call_run(&mut store).await;

    match run_result {
        Ok(res) => res.map_err(|_| anyhow!("Unknown error"))?,
        Err(error) => {
            if let Some(exit) = error.downcast_ref::<I32Exit>() {
                if exit.0 != 0 {
                  info!("Call failed with exit code {:?}", exit.0);
                  return Ok(false);
               }
            } else {
              return Err(error);
            }
        }
    };

That's... unfortunate!

[pchickey]

Yes, it is unfortunate. Wasi provides exit only because unwinding (via the now Phase 4 exception handling proposal) was not available at the time that Wasi Preview 0, 1 and 2 were written. Since we are providing special machinery outside of the Wasm VM to make up for a feature the Wasm VM doesn't have, this is a really awkward feature to implement. We hope that it will be better in the future. Wasmtime doesn't yet have exception-handling implemented, but its high on Chris Fallin's list for this summer. Wasi Preview 3 may be able to include a change that eliminates exit if all of the runtimes and guests are able to switch to exception handling in time, otherwise it may have to wait for Preview 4.

[bjorn3]

How would the exception handling proposal help here? exit(N) is supposed to immediately exit the process without running cleanup as would happen when you throw exceptions. This is explicitly documented in rust's libstd:

https://doc.rust-lang.org/stable/std/process/fn.exit.html

This function will never return and will immediately terminate the current process. The exit code is passed through to the underlying OS and will be available for consumption by another process.

Note that because this function never returns, and that it terminates the process, no destructors on the current stack or any other thread’s stack will be run.

[pchickey]

As I understand it, "running cleanup" is defined to the code that LLVM emits into catch blocks, and rustc would elect not put any sort of Drop impl destructors in there. I don't know how this would interop with C++ code using exceptions over the FFI, though, and if in practice it won't, I guess we are stuck with treating exit specially forever - unless someone adds it explicitly to core wasm, which I sincerely doubt.

[bjorn3]

As I understand it, "running cleanup" is defined to the code that LLVM emits into catch blocks, and rustc would elect not put any sort of Drop impl destructors in there.

With -Cpanic=unwind -Ctarget-feature=+exception-handling, rustc will emit wasm that uses catch_call + rethrow for running Drop impls on exceptions: https://rust.godbolt.org/z/q17TPb364