ARKit phone teleop for the arm, and a sim that the app can find - #740
Draft
DavidDobas wants to merge 6 commits into
Draft
ARKit phone teleop for the arm, and a sim that the app can find#740DavidDobas wants to merge 6 commits into
DavidDobas wants to merge 6 commits into
Conversation
/robot/info now carries simulation=true (the CLI's /.dockerenv rule), so the app can skip WebRTC and tag the robot. The host world server runs a LAN discovery beacon (udp/19090: broadcast + unicast replies to probes, on the sender's port and the port the probe declares); the launcher passes it the published ports, the robot-name file and the checkout version, and restarts a world server that is not announcing.
…t arm state Gripper and joint-space commands read the measured arm; issued right after a park they raced the feeds coming back and were refused with "No arm state available" — the first tap after a minute of idling always failed.
/armsdk/stream_pose gains "grip" (0 closed … 1 open); the follower streams it as j6 alongside the IK joints, so the operator's thumb drives the claw through the same velocity-clamped slew as the arm and the last value becomes the standing grip when the stream ends. Absent, the standing grip is kept.
The LMA solver has no notion of limits; from a folded posture it returned joints past their range that the driver then clamped silently, so a small requested step moved the arm somewhere else entirely (a +3 cm lift from rest went 4 cm down in the sim). Out-of-range solutions are now dropped per seed, and a target with none left is reported as unreachable — which the pose follower shows the operator — instead of being mis-executed.
…t seed tried on_delta stored q_out — the last seed's raw result, None when that seed's solve failed — as the seed for the next solve. With the current posture succeeding and the zero seed failing (the edge of reach, where teleop lives) the node kept None and crashed on the next /mars/arm/state message. Seed with best_solution; and respawn the IK node so a crash cannot leave the arm without IK.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two things the controller app needs from the OS, on the real robot and in the sim:
Phone teleop (ARKit) —
arm_sdk_servergains/armsdk/stream_pose(std_msgs/StringJSON{session, x, y, z, qx, qy, qz, qw}): a 6-DoF end-effector delta in base_link axes relative to the arm's pose when the operator pressed the button. A latest-wins follower anchors on the live/fk_posethe first time a session is seen, composes anchor ⊕ delta, solves IK through the existing/ik_deltanode and feedsManipulation.stream_joints; it reportsidle|following|unreachable|refusedon/armsdk/stream_pose/status.Manipulation.stream_pose()is the one-tick IK→stream step. The IK node logs each solve at debug (30 Hz would flood) and keeps the current-posture solution when it already fits, so a streamed target doesn't hop IK branches.Sim discovery —
/robot/infocarriessimulation: truein the container. The host world server runs a LAN beacon onudp/19090(broadcast, plus unicast replies to app probes); the launcher passes it the published ports,data/robot_info.jsonand the git version, and restarts a world server that isn't announcing.INNATE_SIM_BEACON=0opts out.Companion app PR: innate-inc/innate-controller-app (ARKit control screen, sim-aware camera/WebRTC, discovery).
Testing
+5/0/+3 cmfrom a working pose landed at+5.0/0.0/+2.8; a second press with0/−3/0 cm + 0.3 rad pitchlanded at0.0/−3.0/−0.2relative to the new pose; status transitionsfollowing → idleper session;move_tovia the action unaffected./robot/infoover rosbridge showssimulation: truein the sim.ruff+ format clean; brain_client pytest 282 passed; launcher tests 67 passed; no new basedpyright errors in touched files.