guest-async component build error

[xdlin]

Hi @dicej I'm very interested in the async component function, thanks for your great work! When I tried to build it locally, but I have this error:

wasm-tools component new --adapt wasi_snapshot_preview1.reactor.wasm \  
    target/wasm32-wasi/release/guest_async.wasm -o guest-async.wasm

I got this error:

error: failed to encode a component from module

Caused by:
    0: failed to validate component output
    1: `task.return` requires the component model async feature (at offset 0x1b527)

I confirmed that I'm using the wasm-tools you provided here: cargo install --locked --git https://github.com/dicej/wasm-tools --branch async wasm-tools. Is there anything I missed?

[dicej]

Hi @xdlin; thanks for reporting this. I've moved the contents of this repo into this PR as part of my effort to get everything upstreamed, so the code here (and the wasm-tools and wit-bindgen forks it points to) is no longer being maintained. Sorry for the confusion. I'll update the README.md to indicate the status.

Meanwhile, my wit-bindgen and wasm-tools forks have been merged upstream, so the latest releases of those crates have full async/future/stream support. However, part of that upstreaming work required adding a new, non-default wasmparser::WasmFeatures::COMPONENT_MODEL_ASYNC feature which is required to validate components which use any of the new async features. And since there's currently no way to specify extra features to wasm-tools component new, you have to do something like this instead:

wasm-tools component new --skip-validation module.wasm -o component.wasm
wasm-tools validate --features all component.wasm

[xdlin]

Thank you for your quick response, I keep watching this PR as well :) I'm grateful for the info, I'll try with your PR later.

Previously I build my project which support Async Host <--> WASM <--> Async Guest with fp-bindgen, but it's inactive now, and I believe component mode with async is the future, so I'd like to switch to component mode once it's ready.

[xdlin]

wasm-tools component new --skip-validation module.wasm -o component.wasm

I guess this is a necessary step for 'wasm-wasip1'?

sync WIT 1st try

I first tried with sync code, and with --target wasm32-wasip1, and run command:

wasm-tools component new --skip-validation ../target/wasm32-wasip1/debug/guest.wasm  -o component.wasm

I got this error:

error: failed to encode a component from module

Caused by:
    0: failed to decode world from module
    1: module was not valid
    2: failed to resolve import `wasi_snapshot_preview1::clock_time_get`
    3: module requires an import interface named `wasi_snapshot_preview1`

sync WIT with adapter

wasm-tools component new --skip-validation ../target/wasm32-wasip1/debug/guest.wasm  -o component.wasm --adapt ./wasi_snapshot_preview1.reactor.wasm 

After downloading the adapter and ran with this command, the sync code compiles

async WIT

When I change the WIT to async mode: generate!({async: true, ...}), I got this error:

error: failed to encode a component from module

Caused by:
    0: failed to decode world from module
    1: module was not valid
    2: failed to resolve import `[export]$root::[task-return]hello`
    3: module requires an import interface named `$root`

with target wasm32-wasip2

If I build with target wasm32-wasip2, I got a identical error like above:

...
= note: error: failed to encode component
          
          Caused by:
              0: failed to decode world from module
              1: module was not valid
              2: failed to resolve import `[export]$root::[task-return]hello`
              3: module requires an import interface named `$root`
          

error: could not compile `guest` (lib) due to 1 previous error

So the error happens with async code only, am I missed something?

my working env

• rustc: 1.84.0 (9fc6b4312 2025-01-07)
• wasm-tools: 1.223.0
• wit-bindgen v0.37.0
• wit-bindgen-rt v0.37.0

[dicej]

Hmm, it seems to working for me. Here's what I did:

cargo install wasm-tools
mkdir -p foo/wit foo/guest/src foo/host/src
cd foo
cat >wit/round-trip.wit <<EOF
package local:local;

interface baz {
  foo: func(s: string) -> string;
}

world round-trip {
  import baz;
  export baz;
}
EOF
cat >guest/Cargo.toml <<EOF
[package]
name = "guest"
version = "0.1.0"
edition = "2021"

[dependencies]
futures = "0.3.30"
once_cell = "1.19.0"
wit-bindgen = "0.37.0"
wit-bindgen-rt = "0.37.0"

[lib]
crate-type = ["cdylib"]
EOF
cat >guest/src/lib.rs <<EOF
mod bindings {
    wit_bindgen::generate!({
        path: "../wit",
        world: "round-trip",
        async: true,
    });

    use super::Component;
    export!(Component);
}

use bindings::{exports::local::local::baz::Guest as Baz, local::local::baz};

struct Component;

impl Baz for Component {
    async fn foo(s: String) -> String {
        format!(
            "{} - exited guest",
            baz::foo(&format!("{s} - entered guest")).await
        )
    }
}
EOF
cargo build --manifest-path guest/Cargo.toml --release --target wasm32-wasip1
curl -OL https://github.com/bytecodealliance/wasmtime/releases/download/v28.0.1/wasi_snapshot_preview1.reactor.wasm
wasm-tools component new --adapt wasi_snapshot_preview1.reactor.wasm --skip-validation guest/target/wasm32-wasip1/release/guest.wasm -o guest.wasm
wasm-tools validate --features all guest.wasm
cat >host/Cargo.toml <<EOF
[package]
name = "host"
version = "0.1.0"
edition = "2021"

[dependencies]
anyhow = { version = "1.0.81", features = ["backtrace"] }
tokio = { version = "1.36.0", features = ["fs", "macros", "rt-multi-thread", "time"] }
wasmtime = { git = "https://github.com/dicej/wasmtime", branch = "async", features = ["component-model-async"] }
wasmtime-wasi = { git = "https://github.com/dicej/wasmtime", branch = "async" }
EOF
cat >host/src/main.rs <<EOF
use {
    anyhow::Result,
    std::{env, future::Future, time::Duration},
    tokio::fs,
    wasmtime::{
        component::{self, Component, Linker, PromisesUnordered, ResourceTable},
        Config, Engine, Store, StoreContextMut,
    },
    wasmtime_wasi::{WasiCtx, WasiCtxBuilder, WasiView},
};

mod round_trip {
    wasmtime::component::bindgen!({
        trappable_imports: true,
        path: "../wit",
        world: "round-trip",
        concurrent_imports: true,
        concurrent_exports: true,
        async: true,
    });
}

struct Ctx {
    wasi: WasiCtx,
    table: ResourceTable,
}

impl WasiView for Ctx {
    fn table(&mut self) -> &mut ResourceTable {
        &mut self.table
    }
    fn ctx(&mut self) -> &mut WasiCtx {
        &mut self.wasi
    }
}

impl round_trip::local::local::baz::Host for Ctx {
    type Data = Ctx;

    #[allow(clippy::manual_async_fn)]
    fn foo(
        _: StoreContextMut<'_, Self>,
        s: String,
    ) -> impl Future<
        Output = impl FnOnce(StoreContextMut<'_, Self>) -> wasmtime::Result<String> + 'static,
    > + Send
           + 'static {
        async move {
            tokio::time::sleep(Duration::from_millis(10)).await;
            component::for_any(move |_: StoreContextMut<'_, Self>| {
                Ok(format!("{s} - entered host - exited host"))
            })
        }
    }
}

#[tokio::main]
async fn main() -> Result<()> {
    let mut config = Config::new();
    config.wasm_component_model(true);
    config.wasm_component_model_async(true);
    config.async_support(true);

    let engine = Engine::new(&config)?;

    let make_store = || {
        Store::new(
            &engine,
            Ctx {
                wasi: WasiCtxBuilder::new().inherit_stdio().build(),
                table: ResourceTable::default(),
            },
        )
    };

    let component = Component::new(&engine, &fs::read(env::args().nth(1).unwrap()).await?)?;

    let mut linker = Linker::new(&engine);

    wasmtime_wasi::add_to_linker_async(&mut linker)?;
    round_trip::RoundTrip::add_to_linker(&mut linker, |ctx| ctx)?;

    let mut store = make_store();

    let round_trip =
        round_trip::RoundTrip::instantiate_async(&mut store, &component, &linker).await?;

    // Start three concurrent calls and then join them all:
    let mut promises = PromisesUnordered::new();
    for _ in 0..3 {
        promises.push(
            round_trip
                .local_local_baz()
                .call_foo(&mut store, "hello, world!".to_owned())
                .await?,
        );
    }

    while let Some(value) = promises.next(&mut store).await? {
        println!("{value}");
    }

    Ok(())
}
EOF
cargo run --manifest-path host/Cargo.toml --release -- guest.wasm

Does the above work for you? If so, maybe you can modify that to suit your needs. Otherwise, please let me know where it fails on your end.

[xdlin]

Wow, your version works~ And after comparing the code, I find the reason, I exported the function in world directly:

package local:local;

world round-trip {
  export foo: func(s: string) -> string;
}

And the related guest code:

mod bindings {
    wit_bindgen::generate!({
        path: "../wit",
        world: "round-trip",
        async: true,
    });

    use super::Component;
    export!(Component);
}

use bindings::{Guest as Baz};

struct Component;

impl Baz for Component {
    async fn foo(s: String) -> String {
        format!(
            "{} - exited guest",
            s
        )
    }
}

With this direct export function in world, I can reproduce the error

[xdlin]

Thanks for your help and your great work on this feature, I can continue my experiment without any blockers after changing the WIT syntax a little bit

[dicej]

Ah interesting; I know I tested direct world exports in WAT, but maybe I'm missing a test for that which uses wit-bindgen. I'll make sure to add one and fix any issues that pop up. Thanks for letting me know!

[dicej]

FYI, here's the fix for world-level exports: bytecodealliance/wasm-tools#1979

[xdlin]

hi @dicej when I tried to integrate this feature into my own project, it was OK before, but with recent change, I got thie compile error:

   Compiling wasmtime-environ v30.0.0 (https://github.com/dicej/wasmtime?branch=async#3d6cd202)
   Compiling cranelift-frontend v0.117.0 (https://github.com/dicej/wasmtime?branch=async#3d6cd202)
   Compiling cranelift-native v0.117.0 (https://github.com/dicej/wasmtime?branch=async#3d6cd202)
error[E0026]: variant `wasmparser::CanonicalFunction::TaskReturn` does not have a field named `result`
   --> /home/linxiangdong/.cargo/git/checkouts/wasmtime-a7bebf70e035e99d/3d6cd20/crates/environ/src/component/translate.rs:636:69
    |
636 |                         wasmparser::CanonicalFunction::TaskReturn { result } => {
    |                                                                     ^^^^^^
    |                                                                     |
    |                                                                     variant `wasmparser::CanonicalFunction::TaskReturn` does not have this field
    |                                                                     help: `wasmparser::CanonicalFunction::TaskReturn` has a field named `type_index`

error[E0027]: pattern does not mention field `type_index`
   --> /home/linxiangdong/.cargo/git/checkouts/wasmtime-a7bebf70e035e99d/3d6cd20/crates/environ/src/component/translate.rs:636:25
    |
636 |                         wasmparser::CanonicalFunction::TaskReturn { result } => {
    |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ missing field `type_index`
    |
help: include the missing field in the pattern
    |
636 |                         wasmparser::CanonicalFunction::TaskReturn { result, type_index } => {
    |                                                                           ~~~~~~~~~~~~~~
help: if you don't care about this missing field, you can explicitly ignore it
    |
636 |                         wasmparser::CanonicalFunction::TaskReturn { result, type_index: _ } => {
    |                                                                           ~~~~~~~~~~~~~~~~~
help: or always ignore missing fields here
    |
636 |                         wasmparser::CanonicalFunction::TaskReturn { result, .. } => {
    |                                                                           ~~~~~~

Some errors have detailed explanations: E0026, E0027.

I'm not sure whether this is the proper place to report this, if necessary, I can put this in another repo with full context.

[xdlin]

It looks like the definition of TaskReturn was changed in wasmparer v0.225, but wasmtime async branch somehow still points to v0.224

[dicej]

@xdlin we've shifted development over to the main branch of the https://github.com/bytecodealliance/wasip3-prototyping repository, so you should no longer use my branch. Sorry for the confusion. That branch should be compatible with the latest version of wasmparser.

Also, please keep in mind that all of this code is very experimental and won't be supported until it's been merged into the main branch of the bytecodealliance/wasmtime repo. Feel free to report bugs on the wasip3-prototyping repo as you run into them, but I can't guarantee we'll address them promptly. And note that more ABI-breaking changes are expected, so binaries built today won't necessarily run on future versions of wasip3-prototyping, and it won't always be obvious which versions/forks/branches of wasm-tools, wit-bindgen, and wasmtime are compatible with each other.

[xdlin]

@dicej Thank you so much for your response, I'm totally understand that this feature is in early stage and not production ready yet. While since I'm so eager for this feature, and would like to integrate that into my own project as soon as possible, so I'm happy to take the ABI breaking risk, and hopefully I can provide some valuable feedback