Recommended controller for high-frequency position commands from RL policy

[Sylelil]

Hi,

I’m using the ur_robot_driver with a real UR10e robot and I have a reinforcement learning (RL) policy that outputs joint position commands at 60 Hz or higher.

I would like to know:

• Which controller is recommended for sending direct joint position commands at this high frequency while ensuring safe and smooth execution?
• Is the forward_position_controller suitable for this use case on real hardware, or should I use one of the trajectory controllers (e.g., scaled_joint_trajectory_controller)?
• Are there best practices for integrating high-frequency position commands from an RL policy with the UR hardware interface?

Thanks for your guidance!

[ashishrai12]

For high-frequency Reinforcement Learning (RL) policies (60Hz+) on a real UR10e, the recommended approach depends on how much jitter your policy can handle and whether you need the robot to stop safely if the network lags.

1. Recommended Controller: joint_trajectory_controller (JTC) Contrary to what the name suggests, you can use the scaled_joint_trajectory_controller for "streaming" commands by sending short, single-point trajectories.

Why: On real hardware, the "scaled" version of this controller is critical because it stays in sync with the robot's internal hardware clock. If the robot's safety system slows down the motion (due to a protective stop or singularity), the controller scales the trajectory execution accordingly.

How: Send a JointTrajectory message with a single point and a time_from_start value that matches your policy step (e.g., 0.016s for 60Hz).

2. Is forward_position_controller suitable? Technically yes, but it is not recommended for real hardware RL for two reasons:

No Interpolation: It sends the raw position directly to the hardware interface. If your RL policy has any jitter or the network drops a packet, the robot will experience "jerky" motion or high-frequency vibrations that can trip a protective stop.

Safety: It lacks the sophisticated state-tracking and scaling found in the trajectory controllers.

3. Best Practices for RL Integration:

Pre-compute Smoothness: If your policy output is "noisy," pass the commands through a low-pass filter or a second-order Butterworth filter before sending them to the ROS 2 controller.

Asynchronous Buffer: Use a separate node to buffer your RL outputs. If the RL policy takes 20ms to compute one step but 10ms for the next, the buffer ensures the UR driver receives a steady heartbeat of commands.

Use the Real-Time Kernel: For 60Hz+ on a real robot, ensure your ROS 2 host is running a PREEMPT_RT patched Linux kernel to avoid "deadline missed" errors in the hardware interface.