wasip3: Unclear how to implement `poll` for UDP sockets

[alexcrichton]

For TCP/stdio/etc the expectation is that poll is routed to zero-length reads/writes on streams, but UDP doesn't have a stream and only has async functions for senc/recv. I don't believe that the same trick of zero-length reads/writes will work here, so currently I don't believe that UDP sockets can get hooked up to poll.

cc @badeend

[badeend]

I haven't given it a lot of thought yet, but naively I envisioned a non-blocking POSIX send to translate to something like:

• start an async call to udp-socket::send
• if it didn't complete immediately, stash the task handle on the socket fd.
• poll awaits that task handle, if any. No stashed task handle == ready
• next time when send is called and there is still a pending task from a previous send -> return EWOULDBLOCK

and similar for recv

[alexcrichton]

For send, where would it send to? Because I believe that would result in sending a valid, but empty, datagram somewhere, right?

For recv, what I'm worried about there is that it requires actually receiving something and buffering it in the guest. If the guest waits for readiness but never actually reads it then that can in theory result in a confusing situation where something was actually read and ends up getting lost.

For TCP the zero-length reads/writes solve this because they don't actually transmit any data, but for UDP there's no way to do a zero-length operation that doesn't do I/O I believe

[badeend]

For send, where would it send to?

Nowhere, poll would never issue a call to udp-socket::send. All poll does is await a previous unfinished send (which was initiated by the user with actual data)

[alexcrichton]

That's a good point yeah, and in a sense this sort of matches what we'll have to do with TCP when zero-length I/O ops actually work from a readiness perspective.

So perhaps this is how send would work:

• If there's an operation in-flight, return EWOULDBLOCK
• Otherwise a send is started with the provided buffer
• If that completes immediately, we're done.
• If that blocks, the operation is cancelled and the input buffer is copied to socket-internal storage and the send is re-issued.
• For poll that would report readiness when there's no in-flight operation.

For receive it would:

• If there's a buffered datagram, that's returned.
• If there's no I/O operation in flight, EWOULDBLOCK is returned.
• Otherwise an I/O operation is started which stores in socket-internal storage.
• For poll that would kick off I/O if it's not started yet and otherwise watch the in-progress operation.

This does require buffering/storage in wasi-libc and can result in lost datagrams, but upon reflection we're also doing that as a fallback with TCP. Given that my concern is probably overblown about trying to avoid buffering data.

[badeend]

That sounds about right, yes. 👍

This does require buffering/storage in wasi-libc

Correct. That's the cost of trying to emulate a readiness-based API on top of a completion-based Component Model. Zero-length read/write probe on streams are a heuristic; not a guarantee. As you already pointed out, I think buffering is unavoidable in the general case.

can result in lost datagrams

If the socket is dropped while there was still a pending packet in the buffer that would indeed get lost. Do you know if there any other ways for wasi-libc to drop data?

One upside is that UDP is unreliable in nature, so applications are typically already written to tolerate packet loss.

[badeend]

If that blocks, the operation is cancelled and the input buffer is copied to socket-internal storage and the send is re-issued.

On second thought, I'm not so sure that's a good idea. This assumes cancellation is atomic. I think wasmtime actually is because there's currently only one await point, but that's more a coincidence than a deliberate design choice. Guests shouldn't depend on that.

It also potentially has the same problem as: WebAssembly/component-model#617

[alexcrichton]

My impression at least is that cancellation is indeed atomic insofar as it returns a status of what happened. Completion can still race with cancellation, but if that happens the guest is still notified that the operation completed instead of being cancelled, which should be fine.

Given that I don't know of any atomicity-related downsides, but are you referring to something else or something different perhaps?

[badeend]

Let's say some fictitious host has an implementation like this:

pub async fn send(data: Vec<u8>, addr: IpAddr) {
    await check_remote_addr(addr);
    await the_actual_send(data, addr);
    await log(data, addr); // <---- Cancelled during this await point
}

and the future is canceled at the third await point. In that case, the guest will get a Canceled status code. Yet, the actual send syscall was already completed at that point. Re-issuing the same send call from the guest will result in duplicate data being sent out.

[alexcrichton]

That's a good point yeah, and arguably a limitation of Wasmtime's support for cancellation in the host today. Wasmtime's model of cancellation is purely "just drop the future" since that's all Rust has as a mechanism for cancellation. Personally I'd chalk that up to a Wasmtime bug, however, as opposed to something that wasi-libc or others would have to work around.

[badeend]

I'd chalk that up to a Wasmtime bug

I think Wasmtime actually illustrates the core issue: guests shouldn't make any assumptions about the atomicity of a send call. I’d prefer adjusting the interface rather than requiring atomicity in the spec.

Alternatives:

• Update wasi-libc to unconditionally copy the input into a socket-internal buffer, so it never depends on the host call being atomic.
• Or augment the wasi-sockets interface with a "try" variant of send and receive, similar to this idea. For example:

// Current API:
send: async func(data: list<u8>, remote-address: option<ip-socket-address>) -> result<_, error-code>;

// New "try" API:
try-send: func(data: list<u8>, remote-address: option<ip-socket-address>) -> result<result<_, error-code>, future>;

Here, try-send is synchronous from the caller's perspective: the buffer is immediately returned. If the socket isn't ready, the returned future resolves when the operation can be retried.

[alexcrichton]

In a sense try-send touches on WebAssembly/component-model#617 and WebAssembly/component-model#618, and I think they're all sort of inter-related. I think it's reasonable in the meantime to unconditionally copy to an internal buffer given the lack of atomicity, yeah. It seems reasonable to me to have a "redux" at some point about async I/O and try-* functions and idioms and such.

[lukewagner]

One issue with try-send is that, at least in virtualization scenarios, it does the whole O(n) list<u8> copy even when it fails. IIRC, this is the motivation for 0.2's wasi:io/streams.check-write and, I'm guessing, wasi:udp/outgoing-datagram-stream.check-send, so I guess it could be fixed the same way.

This might be a naive question but: in the same way that wasi:io/output-stream turned into stream<u8>, could wasi:udp/outgoing-datagram-stream turn into stream<outgoing-datagram>? They both seem to follow the same API pattern in 0.2. Then, if we added stream.try-{read,write}, those and 0-length reads/writes could be used to address the issues mentioned above.

[badeend]

it does the whole O(n) list copy even when it fails

Correct. But its already a lot better than always needing an intermediate copy. With lazy lowering, the final gap can be closed.

could wasi:udp/outgoing-datagram-stream turn into stream?

That was my initial approach and is also the reason why the 0.2 interface was already aligned to look stream-ish.

However during the actual 0.2 -> 0.3 interface conversion, I quickly ran into the complication that every send/receive may fail/succeed independently.
I spent some time looking into how to model this in WIT. For example using separate "data" and "result" streams. Then the question becomes how to correlate the I/O operations with the results. io_uring uses the user_data field for this, where the caller generates & provides a unique identifier on the submission queue and gets that same identifier back in the completion queue.

In the end it became awfully complex, so I landed on the current design instead.

[lukewagner]

@badeend Ahh, that all makes sense; thanks for explaining (and for giving streams a good try). (I think perhaps the potential 0.3.x stream<T,U,V>/future<T,V> extension, where V flows in the "opposite" direction as the T/Us, could potentially address this need for per-stream-element results, but that should be a discussion for a separate issue). In that case, try-send does make sense, perhaps complemented by some ready: async func() (to take the place of 0-length write)?

Separately, thinking more about your astute comment above, I think @alexcrichton is right that this should be considered a (temporary) limitation of host bindings that could be fixed by some addition that allowed the host send() function in your example to (1) be notified of cancellation in an async context so that it can await if need be, (2) even once notified of cancellation, still return a value indicating that the I/O has in fact happened (and thus guest wasm will see that the subtask reach the RETURNED state and not a CANCELLED_BEFORE_(STARTED|RETURNED) state. If send() was implemented as a component, this would be analogous to the send() task being notified of cancellation but then calling task.return instead of task.cancel, which is allowed. Would this resolve the atomicity issue?