|
| 1 | +# ros2_mcp Collaboration Discussion: Prepared Content |
| 2 | + |
| 3 | +_Context: Issue [miaodx/roboharness#150](https://github.com/MiaoDX/roboharness/issues/150) — open a collaboration discussion on the ros2_mcp repository._ |
| 4 | + |
| 5 | +**Post at:** https://github.com/LCAS/ros2_mcp — open a **Discussion** (or Issue if Discussions are disabled) |
| 6 | + |
| 7 | +**Title:** `Integration idea: roboharness for checkpoint-based visual testing alongside ros2_mcp real-time interaction` |
| 8 | + |
| 9 | +--- |
| 10 | + |
| 11 | +## Full Discussion Content |
| 12 | + |
| 13 | +Hi ros2_mcp team, |
| 14 | + |
| 15 | +I maintain [roboharness](https://github.com/MiaoDX/roboharness) — a visual testing harness for AI coding agents in robot simulation. I wanted to open a discussion about a potential complementary integration, since the two tools appear to be solving different but related problems in the "AI agent + robotics" space. |
| 16 | + |
| 17 | +### What ros2_mcp does (as I understand it) |
| 18 | + |
| 19 | +ros2_mcp exposes ROS2 topic interaction — subscribe, publish, read joint states, call services — over the MCP protocol. This lets AI coding agents **actively control** a running ROS2 system in real-time. |
| 20 | + |
| 21 | +### What roboharness MCP does |
| 22 | + |
| 23 | +roboharness exposes **checkpoint-based visual testing** over MCP. Rather than real-time interaction, it pauses the simulation at named moments to capture multi-view screenshots and structured state, then lets the agent evaluate whether the robot behaved correctly: |
| 24 | + |
| 25 | +``` |
| 26 | +capture_checkpoint → multi-view PNGs + state.json at a named moment |
| 27 | +evaluate_constraints → pass/fail verdict with per-metric details |
| 28 | +compare_baselines → regression detection against historical runs |
| 29 | +evaluate_batch_trials → aggregate CI-ready success rate across N trials |
| 30 | +``` |
| 31 | + |
| 32 | +The output is designed for AI agents to read: PNG screenshots the agent can see, JSON state it can parse. |
| 33 | + |
| 34 | +### Why these seem complementary, not competing |
| 35 | + |
| 36 | +ros2_mcp is about **interaction**: the agent drives the robot, reads sensor data, reacts to events in real-time. |
| 37 | + |
| 38 | +roboharness is about **evaluation**: the agent pauses, looks at what happened, checks constraints, and decides whether the outcome was correct. |
| 39 | + |
| 40 | +A combined workflow might look like this: |
| 41 | + |
| 42 | +``` |
| 43 | +1. [ros2_mcp] Subscribe to /joint_states and /cmd_vel |
| 44 | +2. [ros2_mcp] Publish nav goal → robot starts moving |
| 45 | +3. [roboharness] capture_checkpoint("pre_approach") → save front + top-down PNGs + pose |
| 46 | +4. [ros2_mcp] Monitor /odom until robot reaches waypoint |
| 47 | +5. [roboharness] capture_checkpoint("at_waypoint") → verify pose constraints |
| 48 | +6. [roboharness] evaluate_constraints → did joint limits hold? Was the final pose within 5cm of goal? |
| 49 | +7. [roboharness] compare_baselines → is this run better or worse than the last 5? |
| 50 | +``` |
| 51 | + |
| 52 | +The agent uses ros2_mcp to operate the robot, and roboharness to judge whether the operation succeeded. |
| 53 | + |
| 54 | +### Integration question |
| 55 | + |
| 56 | +A few things I'd find useful to understand from your side: |
| 57 | + |
| 58 | +1. **Simulation vs. real robot:** Does ros2_mcp work exclusively with real ROS2 systems, or does it also work with Gazebo / Isaac Sim publishing topics? Roboharness currently targets simulation (MuJoCo, Isaac Lab, Gymnasium-based envs); the overlap is probably Gazebo + Isaac Sim. |
| 59 | + |
| 60 | +2. **State capture overlap:** ros2_mcp presumably exposes `/joint_states` and TF data. Roboharness also captures joint state at checkpoints. Is there a clean way to let roboharness read from ros2_mcp's ROS2 connection rather than maintaining a separate one? Or does it make more sense to keep them independent? |
| 61 | + |
| 62 | +3. **Checkpoint trigger from ROS events:** Ideally, `capture_checkpoint` would be triggered by a ROS event (e.g., "capture when the gripper closes" = subscribe to a topic and fire on state change). Is that a pattern you'd consider exposing in ros2_mcp? |
| 63 | + |
| 64 | +I'm not proposing a merge — both tools have different scopes. But if there's appetite for a combined example (e.g., "use ros2_mcp to drive a Gazebo arm + roboharness to evaluate grasp quality"), I'd be happy to contribute one. |
| 65 | + |
| 66 | +GitHub: https://github.com/MiaoDX/roboharness |
| 67 | +roboharness MCP server docs: https://github.com/MiaoDX/roboharness/tree/main/src/roboharness/mcp |
| 68 | +MIT License, Python 3.10+ |
| 69 | + |
| 70 | +--- |
| 71 | + |
| 72 | +## Posting Notes |
| 73 | + |
| 74 | +- **Where:** https://github.com/LCAS/ros2_mcp — use the Discussions tab if available; otherwise open an Issue |
| 75 | +- **Tone check:** The three questions at the bottom should read as genuine technical questions, not rhetorical ones. Edit if they don't. |
| 76 | +- **Goal:** Establish a connection and get feedback on whether a combined workflow example is useful — not request any code changes on their side |
| 77 | +- **Follow-up:** After posting, add the discussion/issue URL as a comment on [issue #150](https://github.com/MiaoDX/roboharness/issues/150) so the response thread is tracked |
| 78 | + |
| 79 | +--- |
| 80 | + |
| 81 | +## Related issues |
| 82 | + |
| 83 | +- **#149** — ROS Discourse post (broader ROS community outreach, different angle) |
| 84 | +- **#152** — Gazebo/ROS2 showcase (the implementation that would make this integration concrete) |
| 85 | +- **#146** — awesome-ros2 listing (deferred — requires external repo PR) |
0 commit comments