Proposal: Add agnocast::Node support to common/autoware_agnocast_wrapper

[Koichi98]

Background

Note: This discussion uses "DDS" to refer to the concrete middleware, but the issues described (discovery overhead, middleware thread CPU consumption) apply to the RMW layer in general.

Currently, autoware_agnocast_wrapper provides macro-level wrappers for Pub/Sub operations (AUTOWARE_CREATE_PUBLISHER, AUTOWARE_CREATE_SUBSCRIPTION, etc.), allowing build-time switching between Agnocast and ROS 2 via the ENABLE_AGNOCAST environment variable. The node itself is always rclcpp::Node.

This proposal extends the wrapper to add support for agnocast::Node.

What is agnocast::Node?

agnocast::Node is a Node class that is implemented without inheriting from rclcpp::Node. It provides an interface equivalent to rclcpp::Node (get_logger(), declare_parameter(), get_clock(), create_publisher(), create_subscription(), etc.), but internally it does not create a DDS participant at all.

In other words, it can be used as a node that does not participate in DDS and communicates exclusively through Agnocast.

This provides the following benefits:

• Complete withdrawal from Discovery Protocol: Since no rcl_node_t (DDS participant) is created, the node does not participate in DDS Discovery Protocol traffic whatsoever. In a system like Autoware with a large number of nodes, this can significantly reduce Discovery overhead.
• No CPU usage increase from DDS threads: The internal threads spawned by the DDS middleware (receive thread, event thread, etc.) do not exist, resulting in a lower per-node CPU usage baseline.

Proposed Changes

We propose extending common/autoware_agnocast_wrapper by adding an autoware::agnocast_wrapper::Node class. This class holds both rclcpp::Node and agnocast::Node internally and switches between them at runtime based on the ENABLE_AGNOCAST environment variable.

The specific changes are as follows:

1. Addition of autoware::agnocast_wrapper::Node class (node.hpp / node.cpp)
   - Holds both rclcpp::Node and agnocast::Node internally and delegates to the appropriate one based on the result of use_agnocast()
   - Wraps common APIs such as parameters, logger, clock, and callback groups
   - Pub/Sub creation is also provided as methods of the Node class
2. Runtime switching support in autoware_agnocast_wrapper.hpp
   - Publisher, Subscription, and PollingSubscriber are refactored into abstract base classes with separate Agnocast and ROS 2 subclasses
3. Addition of node_main_switchable.cpp.in template
   - In agnocast mode: uses agnocast::init() + AgnocastOnlySingleThreadedExecutor
   - In rclcpp mode: uses rclcpp::init() + rclcpp::executors::SingleThreadedExecutor as before
   - Note: The current PoC template only supports SingleThreadedExecutor, but support for MultiThreadedExecutor is planned.

A proof-of-concept implementation of the modified autoware_agnocast_wrapper is available here: Koichi98/autoware.universe#3

Discussion Points

Is it appropriate to absorb this extension in common/autoware_agnocast_wrapper (universe)?
- When using this in Autoware Core, is it acceptable to depend on a common package in universe?
- Or should the wrapper be placed on the core side?

Also, please feel free to comment if you have any other concerns.

Reference

• Agnocast Repository: https://github.com/tier4/agnocast
• Previous Discussion on Agnocast Integration: https://github.com/orgs/autowarefoundation/discussions/5835

[youtalk]

Go beyond!

[xmfcx]

Thanks for the detailed proposal and the PoC, @Koichi98 ! This is great work and a promising direction for reducing DDS overhead.

Regarding your discussion points:

When using this in Autoware Core, is it acceptable to depend on a common package in universe?

No. autoware_core is not allowed to depend on a Universe package.

If autoware_core is to depend on it, the package should move under core packages, similar to these dependencies.

However, as you can see, all Autoware Core packages (apart from the ones directly under the ROS build farm) are under the autowarefoundation organization. I'm concerned about linking an out-of-organization package as a repository dependency.

So I'd suggest either:

• Move the agnocast package under the autowarefoundation organization, and we can place it under the core/ folder alongside similar dependencies.
• OR release a stable version (in the ROS build farm) and add it as a rosdep dependency, if you'd prefer to keep it under the tier4 organization.

Or should the wrapper be placed on the core side?

Yes, the wrapper can be placed under autoware_core/common once the dependency question above is addressed.

[mitsudome-r]

I agree with Fatih on this. We are currently releasing Autoware from ROS buildfarm so we would have to make agnocast also able to install from apt package and add to rosdistro.

[sykwer]

• I’m in favor of moving the agnocast repository under the autowarefoundation organization.
• We’ve already registered Agnocast with the ROS build farm and are now waiting for it to be released.

[Koichi98]

Thank you @mitsudome-r and @xmfcx for your feedback and clarification on the dependency requirements.

I fully understand and agree with the concerns raised. We will ensure that one of the following conditions is satisfied before integrating agnocast into autoware_core as you mentioned:

1. Move agnocast under the autowarefoundation organization - As @sykwer mentioned, we are open to this approach
2. Release via ROS build farm as a rosdep dependency - We have already submitted agnocast to the ROS build farm and will be soon released

We will not proceed with core package integration until either of these requirements is fulfilled.

Regarding the wrapper approach itself (autoware_agnocast_wrapper to provide switchable agnocast::Node), I'd like to confirm that this direction is acceptable. Once the dependency issue is resolved, we plan to place the wrapper under autoware_core/common as suggested by @xmfcx.

Please let us know if there are any other concerns or suggestions!

[xmfcx]

autoware::agnocast_wrapper::Node sounds alright.

The agnocast::Node and rclcpp::Node interface should be kept equivalent in the future as well.

In the compile time, depending on the ENABLE_AGNOCAST environment variable, both options should be possible to compile.

For this, we might need to double the build & test infrastructure. To build and test with and without agnocast.

And finally, in the following situation, the autoware should still compile with rclcpp:

• agnocast is not installed/present.
• ENABLE_AGNOCAST=0
• agnocast_wrapper will fallback to rclcpp and still compile as expected.

If all these are supported, we can proceed with the agnocast support 👍

[mitsudome-r]

Move agnocast under the autowarefoundation organization - As @sykwer mentioned, we are open to this approach

I've talked with @sykwer and @Koichi98 on this. They would like to test if CI workflow works properly before transferring the repo so I've temporarily created a copy of agnocast under awf org and added them as admins. https://github.com/autowarefoundation/agnocast_ci_test

We can remove the repository once the checks are done and move on with the transfer.

[mojomex]

Hey @Koichi98, awesome proposal!
How hard do you reckon it is to support this in the Foxglove Bridge so that people can still visualize all topics?

[Koichi98]

@mojomex
Thanks! Agnocast provides Agnocast/ROS2 bridge feature that works with agnocast::Node as well. It dynamically detects entities to create bridge publishers/subscribers, enabling both Agnocast pub → ROS 2 sub and ROS 2 pub → Agnocast sub.

So the data flow would be: Agnocast pub → Agnocast Bridge → Foxglove Bridge → Foxglove Studio

If you really want to avoid the double-bridge overhead, you could fork Foxglove Bridge and integrate Agnocast directly. But visualization tools typically don't have strict latency requirements. Also, the Agnocast Bridge uses CallbackIsolatedExecutor, so you can raise the thread priority for specific topic callbacks if needed.