Consider using websocket rosbridge to support web assembly targets

[mxgrey]

One of the greatest strengths of the Rust ecosystem is seamless cross platform support, including the ability to compile to web assembly.

For some crates supporting web assembly is not as feasible because of system dependencies that are not implemented in Rust. With its dependency on rcl, rclrs is an example of this. In theory it may be possible to compile rcl to web assembly and then also compile a browser-compatible rmw middleware to web assembly, but I suspect there will be many hurdles to overcome to make that work out of the box for ordinary Rust developers.

What I would propose instead is to use a feature flag (e.g. "rosbridge" or "websockets") which will switch out the rcl-based implementation for a websockets implementation, with no change to the API, so end user code is exactly the same for both web targets and normal targets.

With that, a user can write a ROS node or ROS-based application, and it would run exactly the same in a web browser as it does when compiled to a desktop by just switching the feature flag based on the target platform.

[esteve]

That's a cool idea, but out of scope for rclrs. There are however, already several projects that might be useful for this usecase.

@IsabelParedes gave a talk about her projects ros2wasm and rmw_wasm at ROSCon 2023
https://github.com/ros2wasm

@allsey87 has ROS on web, which uses ROS compiled for WASM
https://discourse.ros.org/t/ros-on-web/26245
https://rosonweb.io/

Zenoh supports websockets as a plugin, I don't know how difficult it'd be to target WASM when compiling Zenoh, but it might be doable since it's all Rust
https://github.com/eclipse-zenoh/zenoh/tree/master/io/zenoh-links/zenoh-link-ws

And lastly, I've been working on an rmw implementation for libp2p, pubsub works but there's still a lot of work to do. I haven't tried it yet, but it should also support websockets via https://crates.io/crates/libp2p-websocket
https://github.com/esteve/rmw_libp2p

In any case, the C parts of the stack are quite portable (rcl, rcutils, rmw). I ported them to iOS, Android and UWP a while ago, and I remember crosscompiling them to WASM with emscripten, but I haven't tried recently. I vaguely remember that @allsey87 submitted patches not long ago to rmw, but I can't remember the details.

[mxgrey]

The point would be that the rclrs API would be able to work exactly as-is for a web application by just setting a feature flag and not needing to change anything else. The projects you're referencing wouldn't accomplish that. At best the downstream user of rclrs would have to implement a feature flag in their own application and manage interfacing with two different libraries depending on whether or not they are using the websocket bridge. This is troublesome and difficult for users to maintain.

In any case, the C parts of the stack are quite portable (rcl, rcutils, rmw)

Ultimately the rmw middleware also needs to be ported for this approach to work, but web browser environments are locked down very tightly, so any rmw middleware that isn't based on websockets won't be able to work. Even rmw_zenoh wouldn't support this out of the box since you'd still need to bridge zenoh's websocket protocol to the rosbridge websocket protocol.

All of these potential approaches would be considerably more work than adding a feature-gated rosbridge websocket implementation under the hood in rclrs.

For reference my use case is to develop a GUI application that interacts with ROS systems, and I want it to be able to compile natively for desktop environments or compile to web assembly so it can run in a web browser with no change in the code base for the different target environments.

My plan is to implement this in a fork and submit a PR for it for further discussion. If you still consider it out of scope after reviewing the PR then maybe I'll refactor it to some kind of shim in a downstream crate that duplicates the rclrs API while swapping out the implementation for rosbridge based on a feature flag, but that would feel pretty icky.

[mxgrey]

If it helps anything, I could implement the rosbridge websocket protocol in a separate crate, and then rclrs can have an optional dependency on that crate for when the "rosbridge" feature is active, and then rclrs just needs to call out to that separate crate. That way rclrs maintainers don't need to maintain the implementation of the rosbridge protocol.

[esteve]

At best the downstream user of rclrs would have to implement a feature flag in their own application and manage interfacing with two different libraries depending on whether or not they are using the websocket bridge. This is troublesome and difficult for users to maintain.

It's all the same stack, except that with emscripten, you can target wasm32-unknown-emscripten, for example. https://github.com/koute/cargo-web pretty much handles all that.

Ultimately the rmw middleware also needs to be ported for this approach to work, but web browser environments are locked down very tightly, so any rmw middleware that isn't based on websockets won't be able to work.

Yes, that's what ros2wasm did, I'm afraid I don't see where the issue is, could you elaborate? Thanks.

All of these potential approaches would be considerably more work than adding a feature-gated rosbridge websocket implementation under the hood in rclrs.

But these projects already exist, and I very much prefer contributing to them as they follow the ROS architecture much more cleanly than adding alternative code path just for this feature.

I'll refactor it to some kind of shim in a downstream crate that duplicates the rclrs API

That seems to me the best approach, rclrs only has the goal of interfacing with rcl (hence the rcl part in rclrs), adding support for this doesn't strike me as aligned with that goal. It'd be like adding another feature for supporting DDS directly via RustDDS.

[mxgrey]

Yes, that's what ros2wasm did, I'm afraid I don't see where the issue is, could you elaborate? Thanks.

Web dev is definitely not my expertise so I'd love to be wrong about this, but my understanding is that web browsers don't offer a way to communicate outside of their sandbox besides https and websockets. From skimming through the ros2wasm project, it looks like they introduce their own rmw middleware implementation which actually targets DDS, so you couldn't, for example, choose zenoh as your rmw implementation without adding yet another bridge.

No matter what, bridges are needed to run ROS inside of a web browser and have it connect to a ROS system that exists outside of the web browser. The question is how many bridges do you need to deal with. If rclrs supports rosbridge directly then everything becomes much easier for users and has better performance.

It'd be like adding another feature for supporting DDS directly via RustDDS.

I understand where you're coming from, but I don't think it's a fair comparison once you get past the surface level. The solution you're proposing with ros2wasm would bridge you into a DDS system, and then if you want to use an rmw implementation that's not compatible with their DDS implementation then you need to add another bridge.

The rosbridge approach that I'm proposing would only require you to run a rosbridge_server, and then beyond that your system can use any rmw implementation with no additional bridges. Considering how well supported the rosbridge protocol is within the community and how inter-compatible it is with all other ROS tooling, I would essentially view it as "rcl for the web".

[esteve]

If rclrs supports rosbridge directly then everything becomes much easier for users and has better performance.

It having better perfomance is debatable, given that the rcl and rmw layers are extremely thin, they are essentially a specificacion. I can see a rosbridge implementation written directly as an RMW implementation, it'd be really cool. And it can be argued that it's easier for users to follow the conventions set in the rest of the ROS ecosystem where they can switch between RMW implementations when they need it.

I understand where you're coming from, but I don't think it's a fair comparison once you get past the surface level.

I think it's pretty much the same, bypassing rcl so that the rclrs API uses a different transport entirely. Could you elaborate further why you think adding support for rosbridge in rclrs is different from adding support to any other protocol or transport directly in rclrs?

The solution you're proposing with ros2wasm would bridge you into a DDS system, and then if you want to use an rmw implementation that's not compatible with their DDS implementation then you need to add another bridge.

I don't see where DDS is being used there, the README.md seems to reference the "DDS spec", but at first glance it's not using DDS at all, I think they might have gotten the terminology wrong.

The ros2wasm has a demo that you can try where they show pubsub and clients and services, it all runs in your browser, there's no DDS https://ros2wasm.dev/

Considering how well supported the rosbridge protocol is within the community and how inter-compatible it is with all other ROS tooling, I would essentially view it as "rcl for the web".

I don't doubt it's well supported, but rclrs is not the place where transports should be plugged, it goes very much against the whole ROS architecture. The rmw layer is there for a reason, to enable client libraries and the rest of the ecosystem to have support for additional transports.

[mxgrey]

It having better perfomance is debatable, given that the rcl and rmw layers are extremely thin

I agree on this part, and I didn't mean to insinuate that rcl or rmw are the source of the inefficiency. The inefficiency I was referring to is the need for multiple sequential bridges, e.g. websockets <-> DDS <-> actual rmw. Instead with rosbridge you can just have websockets <-> actual rmw.

I don't see where DDS is being used there, the README.md seems to reference the "DDS spec", but at first glance it's not using DDS at all, I think they might have gotten the terminology wrong.

True, the exact quote in the doc is the role of this package is to implement the middleware in accordance to the DDS specification (work in progress) so I guess the intent was to use DDS to reach out past the web browser but maybe they never got around to that. In that case, I would guess their rmw only works within a single browser and can't call out past the sandbox. This doesn't give me a viable path to what I'm looking for.

Could you elaborate further why you think adding support for rosbridge in rclrs is different from adding support to any other protocol or transport directly in rclrs?

In my mind the difference is that rosbridge specifically exists to facilitate connecting to a ROS system, so it's always exactly one websocket hop away from the actual RMW. Unless I'm mistaken, this is the best that can possibly be achieved when working within a web browser.

If we were to perfectly align with the ideology that you're describing, I suppose the "right" thing to do would be to implement a rmw_rosbridge where we make an RMW plugin that implements the rosbridge websocket protocol. At face value that might be a reasonable approach, but the only open source C++ implementation of websockets I know of is websocketpp which has been unmaintained for at least 5 years, and which I personally know has many bugs that show up in production. We're actively moving Open-RMF away from using it.

On the other hand there are several good options for actively maintained websocket libraries in Rust (and the Rust ecosystem has much more emphasis on web development than the C++ ecosystem). Now I could implement a rmw_rosbridge in Rust, then do rmw_rosbridge <-> rmw <-> rcl <-> rclrs but then I'd be forced to bridge two Rust code bases through a layer of C with no tangible benefit.

In general creating an RMW implementation is a massive effort that spans a year or more with multiple developers involved, especially to reach feature completeness. It requires careful attention to pointer management, and doing it in Rust means an enormous amount of unsafe code, as we already see in rclrs. This would be a non-starter for me in the sense that I will never find time to implement it, whereas a pure Rust-based implementation of the rosbridge protocol would take me a few days at most, and then shimming it inside of rclrs will take even less.

The rmw layer is there for a reason, to enable client libraries and the rest of the ecosystem to have support for additional transports.

Philosophically I agree with this, but due to constraints in the web world, there's no viable path to supporting the rmw architecture in general within a web browser. It can only work for web-specific RMW implementations that bridge out through websocket (...or maybe email).

If we were to be ideologically pure about the RMW layer being the only place where a bridging protocol can be placed, then I expect we'll never have proper web browser compatibility for rclrs, and then no one will use rclrs on the web because no one will have enough time available to implement an rmw_rosbridge.

[esteve]

If we were to perfectly align with the ideology that you're describing, I suppose the "right" thing to do would be to implement a rmw_rosbridge where we make an RMW plugin that implements the rosbridge websocket protocol. At face value that might be a reasonable approach, but the only open source C++ implementation of websockets I know of is websocketpp which has been unmaintained for at least 5 years, and which I personally know has many bugs that show up in production. We're actively moving Open-RMF away from using it.

I never mentioned C++. If you have a look at my rmw_libp2p project, it's mostly Rust, and pubsub already works there, it uses one of the libp2p transports and CDR as a serialization library. libp2p is written in Rust, and has a Rust-based websocket transport (https://docs.rs/libp2p/latest/libp2p/websocket/index.html), which it could be used for this case.

And it's not an ideology, it's just software engineering. The design of ROS is meant exactly for the usecase you describe, and from what I understand, it wouldn't matter if the rosbridge protocol is implemented a few layer under rclrs.

I'd be forced to bridge two Rust code bases through a layer of C with no tangible benefit.

And I don't want to be forced to add a transport to rclrs with no tangible benefit either.

The requirements of the project you might be working on don't necessarily have to be the requirements of rclrs.

Philosophically I agree with this, but due to constraints in the web world, there's no viable path to supporting the rmw architecture in general within a web browser. It can only work for web-specific RMW implementations that bridge out through websocket (...or maybe email).

It's the same reason why we have rmw implementations that only work for certain operating systems or architectures. Just because there's a target that won't apply to the rest of the targets, it doesn't mean the RMW layer has to be thrown away. And I don't agree that there's no viable path to supporting the rmw architecture on a web browser when you have demos like https://ros2wasm.dev/pages/demo05/index.html already working now, pubsub and client/service all in the browser.

If we were to be ideologically pure about the RMW layer being the only place where a bridging protocol can be placed, then I expect we'll never have proper web browser compatibility for rclrs, and then no one will use rclrs on the web because no one will have enough time available to implement an rmw_rosbridge.

I think in the pursuit of making your point, I guess you'll agree that this scenario you've created is quite extreme and bordering a fallacy. How come we'll never have web browser compatiblity when you could already just use rmw_wasm now and run rclrs on top? 😉

[allsey87]

So compiling ROS components to wasm32-unknown-emscripten is not a problem (this is what I do in ROS On Web). The problems start once you reach your middleware layer. At this point, you have roughly two approaches.

If you want a web first solution you can write a custom middleware that just works in the browser (again this is what I do in ROS On Web), alternatively if you want communication between multiple browsers/devices you could use a middleware based on WebRTC (note that this would require a signalling server at a known/accessible URL). The problem here is that you would need to use the WebRTC middleware for all your devices/browsers since it is not DDS. It may also be possible to use Zenoh here, but I am not so sure about what features are available/whether it supports the wasm32-unknown-emscripten target (it may just be wasm32-unknown-unknown which, for the moment, has ABI incompatibilities with wasm32-unknown-emscripten). Also quoting from https://github.com/eclipse-zenoh/zenoh-ts:

The long-term plan is to use zenoh Zenoh written in Rust to target WASM. In its current state, it is not possible to compile Zenoh (Rust) to target WASM, and it will need to undergo a fair amount of refactoring before that can happen.

Alternatively, if you want to communicate with an existing DDS system then you do indeed need a bridge. This could be done at the RCL layer or the RMW layer with the former being a lot of work and the latter being designed for this purpose.

I'd be forced to bridge two Rust code bases through a layer of C with no tangible benefit.

This is unfortunate, but is the result of ROS being a C/C++ project. I tend to agree with @esteve, and think the only clean way to avoid this would be to start rewriting some of these core layers in Rust and have those made available behind feature flags. ROS 3 anyone?

[mxgrey]

How come we'll never have web browser compatiblity when you could already just use rmw_wasm now and run rclrs on top? 😉

I'm talking about web browser compatibility that can reach past a single web browser. rmw_wasm does not do that, so it's not useful for anyone that wants to connect a web application to a ROS system that's running outside of their web browser.

Sure it's possible to make an rmw_rosbridge, but at that point we're talking about months of development effort instead of days. I don't think I'm likely to get support for that.

We would also be creating considerably more trouble for rclrs users on the web who will have to wrangle multiple dependencies into web assembly (which is something the build farm won't help with, so they have to set it up manually), whereas a "rosbridge" feature could just turn off the rcl dependency entirely and allow rclrs to compile cleanly into a web application through ordinary use of cargo.

But I can see that this idea isn't getting traction here so I'll go ahead and weigh my other options.

[allsey87]

Just to be clear, why I don't think rclrs should provide an alternative rcl, I think it could make sense to be able to customise which rcl is used whether that be the C librcl.so/.a or some Rust replacement/specialised bridge like you are describing.

[esteve]

@mxgrey I still think this is a cool idea, just that it doesn't belong in rclrs. In any case, we could move common structs and functions into a separate crate called rclrs_api that you can reuse for your rclrs-like layer and then you'd be free to implement support for rosbridge however you want. Let me know if that'd be useful.

[mxgrey]

we could move common structs and functions into a separate crate called rclrs_api that you can reuse for your rclrs-like layer and then you'd be free to implement support for rosbridge however you want.

I'm open to something along these lines, and it's one of the paths that I've been considering, but ultimately I don't see a way to make it work without having a full-blown duplication the rclrs API in the downstream crate.

Many of the peripheral features of rclrs, such as parameter services and (in the future) actions and lifecycle nodes, are built on top of the middleware primitives of pub/sub and service/client. If I can't swap out the pub/sub and service/client implementation out from underneath rclrs itself, then I can't tie those rclrs features into a different protocol without re-implementing them.

One insane idea I have would be to:

1. Define traits for the implementation of subscriber, publisher, service, and client
2. Define a trait for spawning instances of the above-mentioned implementations (e.g. PrimitiveSpawner), and store a Box<dyn PrimitiveSpawner> in Context
3. Make every struct like Subscription, Publisher, Client, and Service just a struct { impl: Box<SubscriptionTrait> }, etc
4. Implement all these traits for the rcl bindings
5. Allow downstream packages to create a Context with a custom Box<dyn PrimitiveSpawner>.

I suppose may be somewhat like what @allsey87 is suggesting.

This would still be less convenient for users than a feature flag on rclrs, but I suppose if rclrs offers the above then I could just make a downstream crate that re-exports the whole rclrs API and then provides a feature flag to switch between the rcl and the websocket implementation.

[mxgrey]

I've immediately realized a problem with my last proposal: a Box<dyn Trait> interface wouldn't be able to support generic methods (e.g. fn publish<T>) which would make it extremely difficult to implement many of the primitives because we'd have to funnel every message through ... a trait object maybe? Which adds unnecessary heap allocations to each operation.