Skip to content

Streamline and extend ros2 interface#9420

Closed
berndgassmann wants to merge 2 commits intocarla-simulator:ue4-devfrom
Motor-Ai:mai/ue4-extend-ros2
Closed

Streamline and extend ros2 interface#9420
berndgassmann wants to merge 2 commits intocarla-simulator:ue4-devfrom
Motor-Ai:mai/ue4-extend-ros2

Conversation

@berndgassmann
Copy link
Contributor

@berndgassmann berndgassmann commented Nov 7, 2025

Description

See the extensive commit message.

But let me add a comment to compared this commit to the PR from last year, this time there were only minor changes introduced into the build system settings. Still most of the build-system cleanup in PR#7919 is valid to make CARLA compile e.g. on Ubuntu 24.04, but those are not required anymore for this build.

Fixes #

Where has this been tested?

  • Platform(s): Ubuntu 22.04
  • Python version(s): 3.10
  • Unreal Engine version(s): 4.26

Possible Drawbacks

Some of the ROS interfaces have been renamed (removed Ego from the name, as each vehicle can be controlled via ROS if required).
The Fix of the rotation direction of roll and pitch might have some impact in case others had fixed in their code outside before; but roll and pitch rotations are often not super relevant.
The improved synchronization handling might have to be considered in downstream applications performing synchronization. But with simple adaptions those should all work out of the box.


This change is Reviewable

@berndgassmann
Copy link
Contributor Author

The respective changes to the carla ros messages are submitted here:
carla-simulator/ros-carla-msgs#24

@berndgassmann
Copy link
Contributor Author

sorry, but this branch already contained the other PR, so later maybe a squash merge might make sense after #9419 was merged to get a clean commit history.

@Blyron
Copy link
Contributor

Blyron commented Nov 8, 2025

This PR fails to compile c++ example in ubuntu and editor in windows with next errors;

2025-11-08T00:48:29.6885937Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Sensor/CustomV2XSensor.h(60): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:30.2888830Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Sensor/CustomV2XSensor.h(60): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:30.9534091Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Sensor/CustomV2XSensor.h(60): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:34.7758124Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Sensor/CustomV2XSensor.h(60): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:36.0647915Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Sensor/CustomV2XSensor.h(60): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:37.5866466Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Sensor/CustomV2XSensor.h(60): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:40.3688341Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Sensor/CustomV2XSensor.h(60): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:42.5347962Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Sensor/CustomV2XSensor.h(60): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:42.5415861Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Server/ServerSynchronization.h(221): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:42.5576259Z C:/a/_w/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Server/CarlaServer.cpp(3619): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:42.5577896Z C:/a/_w/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Server/CarlaServer.cpp(3616): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:42.5579514Z C:/a/_w/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Server/CarlaServer.cpp(3622): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:42.5581142Z C:/a/_w/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Server/CarlaServer.cpp(3634): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:42.5582764Z C:/a/_w/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Server/CarlaServer.cpp(3631): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:42.5584466Z C:/a/_w/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Server/CarlaServer.cpp(3642): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:42.5586092Z C:/a/_w/carla/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Server/CarlaServer.cpp(3650): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:48:45.7566894Z C:\a_w\carla\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Sensor/CustomV2XSensor.h(60): error C7555: the use of designated initializers requires at least "/std:c++20" 2025-11-08T00:49:32.5012710Z make: *** [CarlaUE4Editor] Error 6 2025-11-08T00:49:32.5059970Z ##[error]Process completed with exit code 2

@berndgassmann
Copy link
Contributor Author

Ah, ok. Will fix the build -- I believe I am still on my Unreal Engine branch with c++20 support on Linux. Thought I removed all c++20 stuff, so that c++17 should be enough. And will take care also on Windows build... Which in the end is suffering from the same Thanks for activating the CI runs.

This commit is extending the ROS2 support of CARLA. It simplifies the
interface to allow easier extensions in future. One part of it is the
enablement of shared sensor/actor stream usage, which results in lower
processing overhead if ROS2 is enabled and gets rid of UE-sensor
individual code changes for ROS2. To achieve such, also some refactoring
on the core LibCarla parts were required.

Actor:
- Moved Blueprint handling sources to carla/actor from carla/client
to be able to make use of that functionalilty within ROS2 server calls

DataTypes:
- Introduced geom::AngularVelocity, geom::Velocity,geom::Acceleration,
geom::Quaternion to make the Unreal -> Carla -> ROS2 data
conversion straigth forward (-> that
- Added some Math functions for Quaternions
- Renamed MakeSafeUnitVector->MakeUnitVector (there is no 'unsafe'
variant allowed anymore)

Streaming:
- Moved streaming/detail/tcp/Message.h -> streaming/detail/Message.h to
be deployed for non-tcp communication streams like ROS2.
- Moved general (tcp-independent parts) from tcp/ServerSession to
streaming/detail/Session.h to be used for ROS2 streams. Had to
rename a Write() function to WriteMessage because of virtual
function overloaded with template function not possible.
- Make Dispatcher a shared_pointer to be able to access from ROS2.

Serializer:
- Provide GetHeaderOffset() functions where missing.
- Add sensor_relative_transform (Location and Quaternion parts) to
SensorHeader to support relative TF. Keep it in the header also
when not compiled with ROS2 to keep ROS enabled server builds
compatible with non-ROS enabled client builds as it's not that
much of an overhead.
- Allow to replace move semantic from Deserialize on ROS2-server
builds to allow multiple sinks (i.e. TCP-Server and ROS2 to deserialize
the data without destoying it).
- Added VehicleAckermannControl to ActorDynamicState in form of a
union with VehicleControl to save space
- Extract sensor/data/Array.h into a ArrayConst.h base class to be used
for const data acess on ROS2 Deserialization of RawEpisodeState.

UE4:
- Updated according to LibCarla changes.
- BoundingBoxCalulator: added GetSkeletalMeshVerticies to allow
ROS2 access to detailed mesh as exact ground truth
- Vehicle/Walker Control were extended by timestamp of the last
control input to have a notion on how old the last queried control value
was

V2X:
- V2XSensor allows selection of virtual communication channels via
sensors 'channel_id ' attribute.
- V2XCustomSensor sends bytes instead of std::string to spport
binary data blobs
- Allow V2X-sensors without an owner if these are placed within the
infrastructure (V2I communication)

BUGFIX:
- Fixed geom::Rotation::RotateVector() rotation directions of pitch and
roll!
- Added Math::GetVectorAngle(), Math::GetVectorAngleAbs() and
supporting functions. Use within MotionPlanStage to get correct
angle delta.
- Added RightHandedVector3D.h (internal class deployed for implicit
correct rotation of CARLA left handed vectors)
- Allow to comment TransformationMatrix access out by ifdef to
prevent from erroneous misuse (default access is still enabled!), since
the TransformationMatrix does NOT provide a right-handed rotation (as
the user might expect)
- Allow building CARLA without internet connection

Minor:
- Handled some compiler warnings

ROS2:
- Integrated ROS2 server extensions to server CMakeList.txt (and
simplified some CMake code using loops)
- Make heavily use of base classes to remove duplicated code
- Ensure only on Domain Participant to get rid of bind() port
already in use error
- ros2 UEPublisher classes for sensors accesses directly the data
stream already prepared for the TCP-Clients, minimizing the overhead of
ROS2 variant when no topic is subscribed
- Enable Boost and Asio exceptions for FastDDS to ensure its working
properly
- Introduced additional publishers:
  + traffic-signs
  + traffic-lights
  + v2xsensor
  + actor_list, sensor_list
  + ...
- Introduced a basic set of ros2 service calls to control Carla:
SpawnObject, SetEpisodeSettings, LoadMap, GetBlueprints,
GetAvailableMaps, DestroyObject
- ROS2 V2X Sensor
- Vehicle publisher implements a variety of individual publishers to
mimick also old ROS-bridge convenient publisher:
  + CarlaVehicleInfo
  + CarlaVehicleControlStatus
  + Speed
  + Odometry
  + Object
  + ObjectWithCovariance
  + VehicleTelemetryData
- Vehicle subscribers:
  + VehicleControlSubscriber
  + AckermannControlSubscriber,
  + SetTransformSubscriber
- use of c++17 to support std::sample() usage

CarlaServer uses rpc::RpcServerInterface to communicate with
TCP-Clients as well with the (integrated) DDS-Client

EnableForRos:
- Dynamic switching on ROS visibility is possible for all actors now
- Default startup behavior is selectable by ROS2TopicVisibility
parameter in DefaultGame.ini: If true, then all and every topic that
is currently offered by the implementation is visible from the
beginning. If false, then only the sensors/actors created via the
Client- or ROS-Interface are visible by defaults. Others can be
activated via EnableForRos() calls (this allows for disabling interfaces
e.g. for leaderboard)

- Introduced fine grained ServerSynchronization mechanism, where each
client has the possibility to interact with the synchronization of
the
carla-server on their own. Especially the ROS interface provides
means
of a time window, the server is allowed to run. Like this, every
client
    can prevent the carla-server to run too fast depending on their
    individual speed.
There is no sync-master anymore. Every client decides for its own if
it
requires synchronization or not. CarlaStatusPublisher provides
insights into current synchronization state.
    Drawback of this change: some existing code might have to be changed
    (see removal of synchronous_master in generate_traffic.py).

    Ensure multiple service calls at once are working:
    - qos history kind eprosima::fastdds::dds::KEEP_ALL_HISTORY_QOS
- qos reliablility kind
eprosima::fastdds::dds::RELIABLE_RELIABILITY_QOS
    - qos durability kind
    eprosima::fastdds::dds::TRANSIENT_LOCAL_DURABILITY_QOS
    for reader, writer, topic
- adapt examples includes and Makefile
- rename Quaterion::Rotation() -> Quaternion::Rotator() because of name
clash with Rotation class
- remove c++20 designators
- fix geom::Math::GetVectorAngle()
@berndgassmann
Copy link
Contributor Author

Reproduced the CI Linux error and fixed it.
All Windows CI compilation errors have been addressed, but only tested on Linux so far, let's see if CI is happy.

@Blyron
Copy link
Contributor

Blyron commented Nov 10, 2025

Windows build is ok now. But Ubuntu is not passing smoke tests.

@berndgassmann
Copy link
Contributor Author

Carved out the geometry changes into #9425 to ease integration.

@berndgassmann
Copy link
Contributor Author

Carved out preparation steps on server side into #9431 . Are independent from the previous PR, so also possible to merge in a first step.

@berndgassmann berndgassmann marked this pull request as draft November 14, 2025 10:06
@MarcelPiNacy-CVC MarcelPiNacy-CVC self-assigned this Nov 20, 2025
@MarcelPiNacy-CVC
Copy link
Contributor

Hello, I'll be having a look at this set of PRs in the following days.
I'll contact you if anything comes up.

@berndgassmann
Copy link
Contributor Author

Hi Marcel,
Great, thanks.
This one here I'll be removing most probably today afternoon.
The current three other PRs named "Step1" could be merged independently from each other without any pre-requirement. If you want me to merge the other ue4-dev head changes to them...
I have prepared two "Step2" PRs which each builds up-on one of the "Step1" PRs also delivering some preparation steps on top. I can push them initially e.g. as draft this afternoon.
Then I most probably also will be able to finish the last "Step3" one which builds then on all of the others and executes the reorg in the ros2 folder. I've had some doubts on some memory observations when running the smoke_tests of that step which I've debugged and resolved this week. If no consumer was connected to the "objects" topic on ROS2 side, the object list within the message was constantly growing from frame to frame eating up memory until the RAM was exhausted. But since I had the fear that I did anything major wrong within the memory management of the Buffer when using the streaming Buffers also from ROS2 server side including deserialization therein requiring some tweaking of the fastcdr-code to accept vectors with non-standard allocator that I used to get rid of any copies in case there is no data adjustment required, I've taken my time to again deep dive into the code with debugger and log outputs to ensure, it behaves as I've designed it. An initial trial with using valgrind to debug failed, because even with UE4 -ansimalloc unreal engine isn't fully starting up...
But now memory is ensured to work properly in the upcoming "Step3" commit. Not this branch here yet...
Ah, if you want to do me a favor, you could create a tag "v2.2.1_c6" on the head of this branch (one commit ahead of v2.2.1_c5 with my change):
https://github.com/carla-simulator/rpclib/tree/carla-callbacks
Because that's needed for the "Step2" on sychonization -- at least if I don't want to refer to carla-callbacks within the Setup.sh instead of a proper tag...
Thanks, and if you have any doubts or require more context on some the changes, just put a note into the line of code...
Bernd.

@berndgassmann
Copy link
Contributor Author

Now this PR is replaced by 5 individual ones providing a step-by-step approach with some preparations and finally the actual ROS2 changes.
See #9451 for some further details and dependencies.

@berndgassmann berndgassmann deleted the mai/ue4-extend-ros2 branch January 23, 2026 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants