Voice controlling Create 3

[skainen]

How are you connecting to your Create 3?

Wi-Fi (ROS 2)

Computer(s) Model(s) and Operating System(s)

Raspberry Pi 3 / Raspberry Pi OS, Raspberry Pi 4 / Ubuntu 20.04 Server OS

Which version of ROS 2 is installed on your computer?

Galactic

Which firmware version is installed on your robot?

Latest

Which RMW is your robot running?

I don't know

Does your robot have an assigned namespace? If so, please share.

No response

Is the robot connected to a network? If so, what is the network type?

Phones WiFi hotspot

Are there multiple Create 3 robots connected to your network?

No

Is multicast enabled?

None

What is the Adapter Board's USB/BLE Toggle currently switched to?

None

Describe your question.

Greetings.

So I am trying to control a Create 3 robot with voice.
I have a Raspberry Pi 4 with Ubuntu 20.04 server OS on it, which has ROS 2 Galactic. I already have a code that is used to control the robot, move, dance etc.
I also have another Raspberry Pi, which is Raspberry Pi 4 that has an older Raspberry Pi OS on it. I am using Respeaker 4-mic array to detect speech. I have everything used for it installed and I have a code that records for 5 seconds, detects hotwords within the 5 seconds. It then executes the command given.
For example:
if "lights on" in transcript:
power.on()
pixel_ring.set_brightness(10)
pixel_ring.set_color(rgb=0xFF0000)

So it creates a 5 second file over and over again, and before starting over it looks for hotwords inside the file. It uses google-cloud-speech.

The problem
Reason why I have 2 Raspberry Pi's is that Ros 2 is only supported on Ubuntu 20.04 Server OS.
It doesn't support Respeaker 4-mic array, so I have to use the Raspberry Pi OS for the mic.
I am already able to control the robot remotely on my Raspberry Pi 3 which has the microphone, by using Putty.

So my question is: is there a way to have the code running on my mic Raspberry, which detects the hotwords and converts it to text, then send the text to the robot Raspberry that has a code running with the commands to execute.
So again, when I give a hotword, the mic Raspberry Pi converts it to text, sends it to robot Rasbperry Pi, and robot Raspberry Pi executes the command.

Thank you in advance.
-SK

[kwujciak]

I'm no expert on this, but my initial idea is that you can connect them via ethernet cable. This would be the most secure and consistent way to have two Pis communicate. Otherwise, you could try publishing and subscribing. For example, one of your Pis converts the hot words to text and publishes that text to a topic. The other Pi would be running a script that subscribes to that topic, and performs some task based on the text it saw when subscribing to the topic. This website seems to have some good info.

[justinIRBT]

Depending on how much patience you have, another option is to try manually compiling ROS2 on the Raspberry Pi 3. At one point, I compiled Foxy on an RPi2 running Raspbian, I made some notes when I did it that I included below. Note, if you go down this route, I am not sure how much community support you will find on forums if you get into challenging edge cases, I would suggest time boxing it to see how far you get:

ROS2 Foxy install on RPi2 notes (I would expect differences for Galactic)

Follow the ROS2 linux setup notes (https://index.ros.org/doc/ros2/Installation/Foxy/Linux-Development-Setup/) with the following adjustments:



For [Install development tools and ROS tools](https://index.ros.org/doc/ros2/Installation/Foxy/Linux-Development-Setup/#id5) section

fails to find python3-colcon-common-extensions and vcstool, remove lines from apt-install
pip3 install -U colcon-common-extensions
pip3 install -U vcstool
pip3 install -U rosdep
create ~/.bash_aliases file with export PATH="$[PATH:/home/pi/.local/bin](http://path/home/pi/.local/bin)"
For [Install dependencies using rosdep](https://index.ros.org/doc/ros2/Installation/Foxy/Linux-Development-Setup/#id7) section

give full path to rosdep for use with sudo → sudo ~/.local/bin/rosdep init
Add -r to rosdep install →  “rosdep install -r —from-paths…”
pip3 install -U ifcfg

pip3 install -U lark-parser

Before build, ignore some directories we don't need on RPi:

pi@raspberrypi:~/ros2_foxy $ touch \

> src/ros2/rviz/COLCON_IGNORE \

> src/ros-visualization/COLCON_IGNORE \

> src/ros2/ros1_bridge/COLCON_IGNORE

There is an issue linking again the atomic library on RPi as mentioned here: (https://github.com/ros2/rcutils/issues/171)

I didn't find an elegant solution in my allotted search time for an answer (there must be a better one than what I ended up using), so I ended up adding 

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -latomic")

or if that doesn't work

target_link_libraries(${PROJECT_NAME} -latomic)

to the following CMakeLists.txt files (mostly the linking errors pointed to [librcl.so](http://librcl.so/), so its possible that there is a better fix to that package that would not propagate the errors to other packages):

src/eProsima/Fast-DDS/CMakeLists.txt
src/ros2/rcl/rcl/CMakeLists.txt
src/ros2/rcl/rcl_action/CMakeLists.txt
src/ros-tooling/libstatistics_collector/CMakeLists.txt
src/ros2/rcl/rcl_lifecycle/CMakeLists.txt
src/ros2/rclpy/rclpy/CMakeLists.txt
src/ros2/rclcpp/rclcpp/CMakeLists.txt
src/ros2/system_tests/test_communication/CMakeLists.txt
I built on a Raspberry Pi 2 (not recommended) and occasionally had the system lock up during the colcon build.  To throttle the resources for individual packages, you can use the command:

MAKEFLAGS="-j1 -l1" colcon build --symlink-install --executor sequential --packages-select rclcpp

I did my primary building with

MAKEFLAGS="-j3" colcon build --symlink-install --executor sequential

[skainen]

Thanks for both answers. I'm now using ZeroMQ to publish and subscribe. I'll also try out manually compiling ROS2 in future, so I wouldn't need two Pi's for it to work.