Skip to content

Commit c664611

Browse files
francocipollonefrancocipollone
committed
Improvements to documentation.
Signed-off-by: Franco Cipollone <franco.c@ekumenlabs.com>
1 parent 4ffdbab commit c664611

8 files changed

Lines changed: 103 additions & 4 deletions

File tree

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,8 @@ For simulation, refer to [workspace setup](#workspace-setup) instructions.
4747
- [`dora_andino_hal`](dora_node_hub/dora_andino_hal): Integration of the andino hal with Dora.
4848
- [`dora_andino_mujoco_sim`](dora_node_hub/dora_andino_mujoco_sim): Integration of the andino mujoco simulation with Dora.
4949
- [`dora_diff_drive_controller`](dora_node_hub/dora_diff_drive_controller): Differential drive controller to be used in Dora frameworks.
50+
- [`dora_gemini_diff_drive_navigation`](dora_node_hub/dora_gemini_diff_drive_navigation): Uses Google's LLM Gemini to navigate a differential drive robot.
51+
- [`dora_string_publisher_ui`](dora_node_hub/dora_string_publisher_ui): Creates a GUI for the user to publisher an arbitrary string on demand.
5052
- [`dora_teleop_keyboard`](dora_node_hub/dora_teleop_keyboard): Dora node for teleoperating mobile robots using the keyboard.
5153

5254

‎andino_dora/README.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,3 +69,50 @@ Run the Rerun server locally (not on the andino):
6969
```
7070
rerun --serve --web-viewer --bind <your_ip>
7171
```
72+
73+
### gemini_navigation.yml
74+
75+
```mermaid
76+
flowchart TB
77+
camera["**camera**"]
78+
dora_andino_hal["**dora_andino_hal**"]
79+
dora_diff_drive_controller["**dora_diff_drive_controller**"]
80+
dora_gemini_diff_drive_navigation["**dora_gemini_diff_drive_navigation**"]
81+
dora_string_publisher_ui[\"**dora_string_publisher_ui**"/]
82+
rerun-viz[/"**rerun-viz**"\]
83+
subgraph ___dora___ [dora]
84+
subgraph ___timer_timer___ [timer]
85+
dora/timer/millis/100[\millis/100/]
86+
end
87+
end
88+
dora/timer/millis/100 -- tick --> camera
89+
dora_diff_drive_controller -- joints_speed_cmd --> dora_andino_hal
90+
dora/timer/millis/100 -- tick --> dora_andino_hal
91+
dora_gemini_diff_drive_navigation -- cmd_vel --> dora_diff_drive_controller
92+
dora/timer/millis/100 -- cmd_vel_tick --> dora_gemini_diff_drive_navigation
93+
dora_string_publisher_ui -- submitted_string as command --> dora_gemini_diff_drive_navigation
94+
camera -- image --> dora_gemini_diff_drive_navigation
95+
dora/timer/millis/100 -- tick --> dora_gemini_diff_drive_navigation
96+
camera -- image --> rerun-viz
97+
```
98+
99+
Runs a dataflow that uses [Gemini](https://gemini.google.com/app) API for navigating the Andino based on user inputs.
100+
101+
1. Obtain a [Gemini API Key](https://aistudio.google.com/apikey) and add it as environment variable to the correspondent node in the [gemini_navigation.yml](graphs/gemini_navigation.yml) dataflow.
102+
103+
2. Build the dataflow:
104+
```
105+
dora build graphs/gemini_navigation.yml
106+
```
107+
108+
3. Run the dataflow locally:
109+
```
110+
dora run graphs/gemini_navigation.yml
111+
```
112+
113+
4. Run the Rerun server locally (not on the *andino*):
114+
```
115+
rerun --serve --web-viewer --bind <your_ip>
116+
```
117+
118+
5. Use the GUI to input a command to be followed by the Andino.

andino_dora/graphs/gemini_control.yml renamed to andino_dora/graphs/gemini_navigation.yml

File renamed without changes.

‎andino_dora_sim/README.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,47 @@ Run the dataflow locally:
5050
```
5151
dora run graphs/mujoco_sim.yml
5252
```
53+
54+
### mujoco_sim_gemini_navigation.yml
55+
56+
```mermaid
57+
flowchart TB
58+
dora_andino_mujoco_sim["**dora_andino_mujoco_sim**"]
59+
dora_diff_drive_controller["**dora_diff_drive_controller**"]
60+
dora_gemini_diff_drive_navigation["**dora_gemini_diff_drive_navigation**"]
61+
dora_string_publisher_ui[\"**dora_string_publisher_ui**"/]
62+
rerun-viz[/"**rerun-viz**"\]
63+
subgraph ___dora___ [dora]
64+
subgraph ___timer_timer___ [timer]
65+
dora/timer/millis/1[\millis/1/]
66+
dora/timer/millis/17[\millis/17/]
67+
dora/timer/millis/100[\millis/100/]
68+
end
69+
end
70+
dora/timer/millis/100 -- camera_tick --> dora_andino_mujoco_sim
71+
dora_diff_drive_controller -- joints_speed_cmd --> dora_andino_mujoco_sim
72+
dora/timer/millis/1 -- model_tick --> dora_andino_mujoco_sim
73+
dora/timer/millis/17 -- viewer_tick --> dora_andino_mujoco_sim
74+
dora_gemini_diff_drive_navigation -- cmd_vel --> dora_diff_drive_controller
75+
dora/timer/millis/100 -- cmd_vel_tick --> dora_gemini_diff_drive_navigation
76+
dora_string_publisher_ui -- submitted_string as command --> dora_gemini_diff_drive_navigation
77+
dora_andino_mujoco_sim -- camera_image as image --> dora_gemini_diff_drive_navigation
78+
dora/timer/millis/100 -- tick --> dora_gemini_diff_drive_navigation
79+
dora_andino_mujoco_sim -- camera_image as image --> rerun-viz
80+
```
81+
82+
Runs a dataflow that uses [Gemini](https://gemini.google.com/app) API for navigating the Andino based on user inputs.
83+
84+
1. Obtain a [Gemini API Key](https://aistudio.google.com/apikey) and add it as environment variable to the correspondent node in the [gemini_navigation.yml](graphs/gemini_navigation.yml) dataflow.
85+
86+
2. Build the dataflow:
87+
```
88+
dora build graphs/gemini_navigation.yml
89+
```
90+
91+
3. Run the dataflow locally:
92+
```
93+
dora run graphs/gemini_navigation.yml
94+
```
95+
96+
4. Use the GUI to input a command to be followed by the Andino.

andino_dora_sim/graphs/mujoco_sim_gemini_control.yml renamed to andino_dora_sim/graphs/mujoco_sim_gemini_navigation.yml

File renamed without changes.

‎dora_node_hub/dora_gemini_diff_drive_navigation/dora_gemini_diff_drive_navigation/controller.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,9 @@ def create_twist_array(linear_vel: float, angular_vel: float) -> np.ndarray:
3535
class DiffDriveGeminiControl:
3636
"""A controller for a differential drive robot using Gemini API."""
3737

38+
# TODO(francocipollone): Allow to set diff drive robot parameters (wheel radius, wheel separation) via constructor.
39+
# Same thing for the camera parameters. For now, these parameters are hardcoded.
40+
# TODO(francocipollone): Provide a way to extend the system instruction via constructor.
3841
SYSTEM_INSTRUCTION = R"""
3942
You are a differential drive robot controller (WHEEL_RADIUS: 0.0315 [m] and WHEEL_SEPARATION: 0.137 [m])
4043
with a camera pointed forward.
@@ -58,7 +61,7 @@ class DiffDriveGeminiControl:
5861
- Resolution: 640 x 480 pixels
5962
"""
6063

61-
def __init__(self, model: str = "gemini-2.0-flash-lite"):
64+
def __init__(self, model: str = "gemini-2.5-flash"):
6265
"""Initialize the DiffDriveGeminiControl with the specified model."""
6366
API_KEY = os.getenv(
6467
"GEMINI_API_KEY", ""

‎dora_node_hub/dora_gemini_diff_drive_navigation/dora_gemini_diff_drive_navigation/main.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ def main() -> None:
3232
if not command:
3333
print("COMMAND environment variable is not set. Using command via input event.")
3434
# Optional node configuration
35-
model = os.getenv("MODEL", "gemini-2.0-flash-lite")
35+
model = os.getenv("MODEL", "gemini-2.5-flash")
3636

3737
# Initialize the controller with the specified model.
3838
controller = DiffDriveGeminiControl(model=model)
@@ -102,7 +102,7 @@ def main() -> None:
102102
try:
103103
velocities = generated_velocities_future.result()
104104
except Exception as e:
105-
print(f"❌ ERROR from velocities generation: {e}")
105+
print(f"Error from velocities generation: {e}")
106106
generated_velocities_future = None
107107
print(f"Velocities generation future done: {velocities}")
108108
if command.lower() == "stop" and (

‎pyproject.toml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,5 +6,8 @@ dependencies = [
66
]
77

88
[tool.uv.workspace]
9-
members = ["dora_node_hub/dora_andino_mujoco_sim"]
9+
members = [
10+
"dora_node_hub/dora_andino_mujoco_sim",
11+
"dora_node_hub/dora_gemini_diff_drive_navigation",
12+
]
1013
exclude = ['andino', 'andino_dora', 'andino_dora_sim',]

0 commit comments

Comments
 (0)