Skip to content

Start with conversational missions and add held-object throwing - #779

Draft
axelpey wants to merge 8 commits into
codex/webapp-onboarding-tourfrom
codex/complete-onboarding
Draft

Start with conversational missions and add held-object throwing#779
axelpey wants to merge 8 commits into
codex/webapp-onboarding-tourfrom
codex/complete-onboarding

Conversation

@axelpey

@axelpey axelpey commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

First arrival offers three prepared missions: put away a LEGO brick, find the Backrooms exit, or cross the street. IntroAgent guides ordinary conversation using map-matched spatial memories. Only the physical mission result or explicit Skip unlocks the interface; reopening resumes the same attempt. Completed/skipped first runs persist on the stable broker origin. Help is a separate, responsive interface tour, and completing/skipping a first mission suppresses the redundant Challenges introduction.

Adds ThrowObject for a small object already held, with synchronized gripper release and grasp checks before the committed swing. Preserves wrist roll, guides the agent to approach within short-toss range, and tells it to finish explicitly requested action sequences. Pickup retention is checked after carry. Misses remain recoverable through chat.

Stacked on draft #721 (based on #757). Includes OS environment bridge #776; paired broker controls/completion persistence are in innate-cloud#110. The local combined preview also includes CTA change innate-cloud#107. Durable LEGO pickup is a separate dependency: local integration 2af6ae82e borrowed the validated Astra candidate based on 0a19eff and OpenAI transport from #772; those files are NOT part of this PR. Successful pickup→approach→throw evidence is preserved at local 7ed351fc3. The pickup speed task is still revising its candidate, so one final compatibility run is pending before the combined work is ready.

Validation: all nine webapp test files pass; focused challenge/memory/intersection physics tests pass; three ROS-backed throw tests pass; viewer environment tests, build and typecheck pass. Full webapp typecheck still has existing errors outside these changes. Live desktop/mobile chooser, conversational retry, reopen during pickup, Skip, keyboard handling and passive Help verified. All three missions passed real physical judges. The cleanup retry completed pickup→30 cm approach→throw and settled the brick inside the box. A fresh first-run street pass revealed the UI automatically; completion survived a fresh iframe origin, without a second introduction. Tests also caught a mistaken agent success claim: it did not unlock the interface, and normal chat correction continued the mission. Agent distance/destination judgments can still need user correction. No physical robot testing or deployment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant