[Draft] Idea: add tokio-workers compatibility shim crate

[irvinebroque]

Summary

• Add tokio-workers crate providing tokio-compatible APIs that work on Cloudflare Workers
• Users can swap use tokio::* with use tokio_workers::* to run tokio-based code on Workers
• Add examples/tokio-workers-demo demonstrating all supported APIs

Supported APIs

Tokio API	tokio-workers Implementation
spawn, spawn_local	wasm_bindgen_futures::spawn_local with custom JoinHandle
time::sleep	JavaScript setTimeout
time::timeout	Race sleep against future
sync::*	Re-exported from tokio (already runtime-agnostic)
join!, try_join!, select!	Re-exported from futures-util
spawn_blocking	Feature-gated: spawn-blocking-panic or spawn-blocking-sync

Example Usage

// Before (won't work on Workers):
use tokio::{spawn, time::sleep, sync::mpsc};

// After (works on Workers!):
use tokio_workers::{spawn, time::sleep, sync::mpsc};

Testing

All 18 unit tests pass, plus the demo example has 7 endpoints that all work correctly:

• /spawn - Concurrent task spawning (~11ms for 3x10ms tasks)
• /sleep - Timer works as expected
• /timeout - Fast ops succeed, slow ops timeout
• /channels - mpsc channels work with multiple producers
• /mutex - Async mutex correctly synchronizes
• /concurrent - join! runs operations in parallel (~30ms not 90ms)
• /spawn-blocking - Sync computation works

[irvinebroque]

@danlapid @kflansburg @guybedford is something of this flavor a terrible idea? Some flavor of drop-in shim for Tokio things, to make it easier for someone to take an app built for Tokio, drop in a replacement library for equivalent behavior, and then get up and running? Ex:

// Before (won't work on Workers):
use tokio::{spawn, time::sleep, sync::mpsc};

// After (works on Workers!):
use tokio_workers::{spawn, time::sleep, sync::mpsc};

Point of PR is to show idea not to land anything. Came out of conversation where smelled like someone had code that probably would work with workers-rs but came across as too tricky.

Just seems like at minimum there is some guidance what concretely you can/can't use from Tokio, how you might go about adapting, etc.

We talk a bit about that here: https://developers.cloudflare.com/workers/languages/rust/crates/

But there is more to say probably? Or more guidance? Or an agent that provides guidance to give people?

[danlapid]

What's the benefit of diverging from the ecosystem here?
What is workers specific about this crate?

tokio already has built in wasm support - https://docs.rs/tokio/latest/tokio/#wasm-support

time works on wasm32-wasi but that will be solved by our work to enable a wasm-bindgen target backed by emscripten.

FWIW Ingvar had an experiment to enable wasm32-wasi with a js ffi middleware to implement wasi, IIRC we didn't go with that because wasi actually had worst crate compatibility than wasm32-unknown

In the meantime, there's https://github.com/cunarist/tokio-with-wasm which I guess is similar to what you did here?

[kflansburg]

I do think that sleep and having to use wasm_bindgen_futures::spawn_local is kind annoying for users (see). I think there are a number of things from Tokio though that do just work out of the box as @danlapid mentioned, and I'm not sure re-exporting them is valuable.

[guybedford]

This was really inspiring, thank you for posting it.

My conclusion like @danlapid is that this belongs in upstream Tokio, and the question is simply how to get there.

It probably makes the most sense to support this under an emscripten target, and aligning Wasm Bindgen with the emscripten target, as it defines an actual execution machine.

As soon as we have the Wasm Bindgen emscripten target support (hopefully coming in the next few months), I will post a Tokio PR to support these features upstream on the combined Wasm Bindgen test suite already available for Tokio.

If we can achieve such a direction we align platform support between Python and Rust workers as well, it will just take a few more months to come together first I think.