Action Server connection goes kaput after ~15 minutes

[cpeavy2]

How are you connecting to your Create 3?

Wi-Fi (ROS 2)

Computer(s) Model(s) and Operating System(s)

Dell (i7-4500U CPU), Ubuntu 22.04.3 LTS

Which version of ROS 2 is installed on your computer?

Humble

Which firmware version is installed on your robot?

H.2.1

Which RMW is your robot running?

FastRTPS

Does your robot have an assigned namespace? If so, please share.

No

Is the robot connected to a network? If so, what is the network type?

2.4GHz Wi-Fi

Are there multiple Create 3 robots connected to your network?

No

Is multicast enabled?

Yes

What is the Adapter Board's USB/BLE Toggle currently switched to?

Bluetooth (default)

Describe your question.

The network connection (specifically the action server connection) to my Create 3 goes kaput after ~15 minutes. I've experienced this with two computers (Ubuntu 22.04). I'll run the /dock, /undock action commands alternately, one after another.

ros2 action send_goal /undock irobot_create_msgs/action/Undock "{}"
ros2 action send_goal /dock irobot_create_msgs/action/Dock "{}"

It works fine for about 15 minutes, then the robot stops moving "waiting for action server."

I'll pick up another computer, and the same thing... I'll run dock/undock repeatedly (while the other computer is "waiting on action server"). It works fine for about 15 minutes, then stops hearing the action server (waiting for action server). I can still see the action server with "ros2 action list."

Teleop_keyboard still works but I get that... the workstation is publishing and the Create 3 is subscribing to /cmd_vel but the action server seems to have trouble sending the goal.

[slowrunner]

Anything suspicious in the log?

My galactic Create3 completed three cycles of undock, run 7h41m to 14%, dock, charge 1h38m to 100%, repeat, with no issue, But there was a 45 second “lease lost” in the log, that luckily did not coincide with any active action server activity.

[cpeavy2]

I'll leave it overnight "Waiting for an action server to become available..." and it doesn't become available. I reboot the computer and it's fine for 10 or 15 minutes. Or I can use another computer while the one is hung up... eventually it goes kaput too, though. What log should I be looking at?

[cpeavy2]

It seems the ability to send actions comes back after several hours but if you run Undock, Drive Distance, and Dock again (over and over) the robot stops moving again after ~10 minutes.... "Waiting for an action server to become available..." I've attached the "warnings" log.
warnings.log

[alsora]

This seems to be an issue caused by your network connection.
See this post from another user #466.
What fixed the issue for them was to switch the DDS configuration to use unicast rather than multicast.

You can refer to this page on how to configure the DDS on your laptop and then use the configuration provided under "disable multicast": https://iroboteducation.github.io/create3_docs/setup/xml-config/

[cpeavy2]

What log should I be looking at?

[brianabouchard]

Hmm, that's strange. Before you change any settings in your .bashrc file, make sure to stop the ROS 2 daemon by runnning ros2 daemon stop then after you've made changes, be sure to source ~/.bashrc.

For the logs, if you go to the webserver by opening a browser and going to your robot's IP address, there is a tab for logs and it will let you export them from there. If you can grab them when it isn't working and post them here, that would be helpful.

[cpeavy2]

After docking and running forward using actions, I run: ros2 daemon stop, and the system replies, "The daemon is not running."

At which point it stopped responding again (Waiting for an action server to become available...). :-]]]

Computer #2 is still able to send action commands... I run: ros2 daemon stop, the system replies same as above but robot still responds to action commands.

Will look at logs.

[cpeavy2]

Here's the log:
And the Warnings log:
I'm not really seeing anything special there.
Also, it's strange. I'll copy the log. Run the robot some. Go back to the log and it doesn't seem to have changed. No updated info. I'd expect something to be logged every time you ran it.
messages.txt
warnings.log

[cpeavy2]

A friend who's quite knowledgeable about ROS suggested I try Cyclone. It's the same symptom. It runs for ~10 minutes and then goes kaput. We're back on Fast DDS.

[cpeavy2]

Well, here's an update. I got another TMobile 5G gateway (for another reason) and now the docking and undocking actions work like a charm, continuously. Thanks!