Component: Unclear how to access exported resource returned by guest

[tliron]

The exported resources example shows how to create a guest resource. Unfortunately it doesn't show how to actually send that resource to an exported client function.

But, to this issue, it doesn't show how to access a resource returned by a call to the guest.

The guest returns ResourceAny, and the documentation does make it clear that this is expected. However, now what?

It cannot be converted to a Resource, because try_into_resource only works on host resources. (That is also not clear in the documentation, I had to delve into the source code to figure that out.)

Also, it's also unclear to me if I must call resource_drop on the value returned by the guest. Or is that necessary just for host resources?

The documentation could be more specific, and the example is not especially useful. In any case, I do not know how to proceed.

[bjorn3]

A resource is an opaque id. I think all the host can do with a guest resource is to pass it as argument when calling a guest function or to call the drop function of the resource.

[tliron]

@bjorn3 , actually I figured it out and it is possible. The example code is the hint. Here's how it's done:

1. First, create the dispatcher. From the example: let logger = guest.logger();
2. Then, do not call call_constructor. We already have the ResourceAny! In fact, call_constructor returns a ResourceAny, too.
3. So now just call the resource functions via the dispatcher on the ResourceAny that you got from the guest, e.g. logger.call_log(&mut store, my_returned_value, Level::Debug, "hello!")?;
4. I'm pretty sure you need to drop_resource, too, when you're done. Well, at least it doesn't return an error when I call it.

I wish the example actually showed this. Instead, the example doesn't have any exported functions in the interface, which doesn't seem to me to be a very common scenario.

[alexcrichton]

@tliron do you have a suggestion for what WIT you'd like to see in the example? it sounds like you figured things out otherwise, but I'd be happy to help update the example to be more useful to you.

[tliron]

I mostly figured things out, but I'm sorry, I doubt that others would, too.

Also, I'm still not sure if I have to manually resource_drop.

1. I suggest that the "exported resources example" show a resource being both sent as an argument to an exported function, and also returned from it. So users can see how to create/send a resource and how to deal with one handed to them by the guest.

2. I think the ResourceAny documentation is confusing and possibly wrong. It says that it can represent either a guest or host resource, but then blends both uses together. It mentions try_from_resource, but that only works on host resources. Generally it is unclear why you would ever need to create a ResourceAny for a host resource in the first place. I mean no disrespect to the author, but I would recommend rewriting that entire section to make it clear why ResourceAny exists for the guest, why it exists for the host, and separate the rules for those two use cases clearly.

On that note, why does wasmtime::component::bindgen emit code that uses ResourceAny, forcing us to unpack and deal with it ourselves? Wouldn't it have been more ergonomic to already handle the conversion to Resource<T> for us? Is there an efficiency concern here?

Again, going back to my suggestion 1, a full example of dealing with this would make these challenges easier to see.

[alexcrichton]

Hm I'm a bit confused, and while I agree we can improve docs I'm going to try to drill in here to be a bit more specific. Basically I'm not sure where the disconnect is and understanding that'll be important to improve the documentation.

---

Also, I'm still not sure if I have to manually resource_drop.

The example linked ends with:

    // The `ResourceAny` type has no destructor but when the host is done
    // with it it needs to invoke the guest-level destructor.
    my_logger.resource_drop(&mut store)?;

and the documentation states:

Note that it is required to call resource_drop for all instances of ResourceAny: even borrows. Both borrows and own handles have state associated with them that must be discarded by the time they’re done being used.

So I'm curious to understand more where you're left with an ambiguity of what to do? It should be the case that all ResourceAny needs to be dropped via resource_drop.

---

I suggest that the "exported resources example" show a resource being both sent as an argument to an exported function, and also returned from it

The example has this code:

    let my_logger = logger.call_constructor(&mut store, Level::Warn)?;
    assert_eq!(logger.call_get_max_level(&mut store, my_logger)?, Level::Warn);

where call_constructor is returning a resource (my_logger: ResourceAny) and call_get_max_level is taking the resource as an argument. Do you feel that one of these isn't satisfying what you were looking for, and if so how come?

It's impossible for the host to create a ResourceAny out of nothing. It's required to be created by the guest and returned back, so is that perhaps a possible source of confusion?

---

I think the ResourceAny documentation is confusing and possibly wrong.

I definitely agree that some examples of using ResourceAny for host resources would be useful! It's relatively niche and thus would be good to document.

Could you clarify which part you think is wrong though? I skimmed over and it looks accurate (albeit not as clear as it could be) to me.

---

On that note, why does wasmtime::component::bindgen emit code that uses ResourceAny, forcing us to unpack and deal with it ourselves? Wouldn't it have been more ergonomic to already handle the conversion to Resource for us? Is there an efficiency concern here?

These are good questions! Unfortunately though it's not possible to do this. The reasons for this touch on the design of the component model itself and how it interacts with instantiation and static types. Basically it's impossible to statically rule out runtime type errors here. Now that doesn't mean the situation couldn't be improved with a type parameter, but even if that were the case there'd still be the possibility for a runtime type error. The current design is intended to reflect that a runtime type error is always possible.

[tliron]

Let's take this slowly.

I think the current example is the niche one. Perhaps there's a misunderstanding on the use cases for resources. Here's a snippet from one of my WITs to give you an idea:

package acme:acme;

interface dispatcher {
    variant value {
        null,
        integer(s64),
        unsigned-integer(u64),
        float(f64),
        boolean(bool),
        text(string),
        bytes(list<u8>),
        value-list(value-list),
        value-map(value-map),
    }

    resource value-list {
        constructor(values: list<value>);
        get: func() -> list<value>;
        length: func() -> u64;
    }

    resource value-map {
        constructor(kv-pairs: list<tuple<value, value>>);
        get: func() -> list<tuple<value, value>>;
        length: func() -> u64;
    }

    dispatch: func(name: string, arguments: list<value>) -> result<value, string>;
}

world functions {
    import host; // not shown here, but also uses resources as arguments
    export dispatcher;
}

The point is to show you that resources can be used as arguments and return values for functions, indeed in complex scenarios where they are nested in lists, records, or variants. I believe that's their true power.

In the currently existing example the resource creation is initiated by the host, so it's clear that the host owns it and must drop it. It includes no exported or imported functions at all in the interface. Actually, it does make resources seem rather useless as they don't do much that can't be done with just exported functions (with a little extra "logger-name" argument), so I understand why it looks "niche" to you.

Back to my example, the concept of ownership is unclear. Who owns a resource sent as an argument? Who owns a resource returned by a function? And who is responsible for dropping? The documentation mentions "borrows", but to be honest I don't understand what is "borrowed" here at all. The arguments and return value are all pass-by-value, intending to pass ownership, too. How does one "borrow" a resource in my example? Are there any "borrows" you can point to?

My OP issue very specifically was about accessing, at the host, the result returned by the exported dispatch function above. I managed to figure out that I had to do it like this (pseudo-snippet dealing specifically with a nested value-list):

pub fn get_returned_list(&mut self, value: dispatcher::Value) -> Result<Vec<dispatcher::Value>> {
        match value {
            dispatcher::Value::ValueList(resource) => {
              let value_list = self.functions.acme_acme_dispatcher().value_list();
              let vector = value_list.call_get(&mut self.store, resource).unwrap();
              resource.resource_drop(&mut self.store)?; // do I need this?
              Ok(vector)
           }
           ....
       }
}

My initial challenge was that it was unclear that I had to explicitly call functions.acme_acme_dispatcher().value_list() in order to gain access to that returned (and nested) resource. The documentation says that I should use ResourceAny::try_from_resource, but that's wrong for this scenario, as that function works only on "host-defined resources". The documentation doesn't make that clear, and indeed doesn't tell me what to do, instead, with guest-defined resources. I figured that out by myself by poring over Wasmtime code.

The side issue (not the main reason I opened this issue): In the currently existing example, the drop is indeed obvious because in it you are constructing the resource yourself, so of course you would have to drop it when you're done using it. Nobody else would. But it's that last drop in my code that is unclear to me.

Did the guest pass ownership to me? Does that mean I really have to drop the resource?

What about passing resources as arguments in the call to dispatch? Who owns those and who is supposed to drop them? The guest? How does the guest drop them? Can we see an example of that?

And here's why this has me worried and I'm making a big deal out of it: If indeed the host has to drop the returned resource then, well, what happens if it doesn't? What happens if I never handle the return value the way I did above? Will this be a memory leak? If that's true, then that's a very big deal that needs to be carefully documented and made very prominent by an example.

Consider that in my case, because the resources can be nested (recursively), such a cleanup would involve more than one drop. That's a lot of responsibility put on my code. I can do it, it's just not clear to me that I must.

And I'm sorry but the current documentation and example have not helped me understand much about this situation.

[alexcrichton]

Thanks for writing that out! To answer some questions more quickly:

• Ownership is conveyed through the WIT signature. If you take a value-list that's by ownership. If you take borrow<value-list> that's now owned. If you return value-list then the host is now responsible for dropping that. Returning borrow<value-list> isn't allowed by WIT. Basically tl;dr; in your example ownership is always required everywhere, so passing arguments to the guest is always relinquishing ownership and receiving results is always acquiring ownership.
• If the host does not call resource_drop, yes, that's a leak. It's a leak until the Store<T> is deallocated, though, so it's not a permanent leak in the sense of you called malloc and forgot to call free.

It includes no exported or imported functions at all in the interface.

For this the example has export logging; which means that the entire interface, and all functions, and all resource methods/functions, are all exported. For example example:exported-resources/logging/logger/get-max-level is an exported function. Could you clarify more what you're looking for in an exported function? Are you looking for example for a free funtion within interface logging?