Add opt-in Agent Mode action publishing (GA-AUD)#93
Merged
Conversation
Wires the Home Assistant side of Agent Mode so the integration can publish
controller actions to the executor shipped in GA-109. When enabled, each
analyzed frame additionally asks the LLM for one action
(PromptBuilder.build_action), validates it against the schema + a button
whitelist (parse_action), and publishes it to
gaming_assistant/{client_id}/action.
Strictly opt-in and safe by design: a runtime-only master switch that
always resets to OFF on restart (CONF_AGENT_MODE, default False), exposed
via a new set_agent_mode service and an Agent Mode switch entity. Action
generation lives behind asyncio.wait_for and a broad try/except so it can
never disrupt the normal tip pipeline. The executor still enforces its own
whitelist, dry-run and emergency stop.
- const: CONF_AGENT_MODE, DEFAULT_AGENT_MODE, MQTT_ACTION_TOPIC, AGENT_VALID_BUTTONS
- coordinator: agent_mode flag + setter, async_publish_action, isolated
_maybe_publish_agent_action hook after a successful tip
- image_processor: generate_action() (build_action -> LLM -> parse_action),
returns None on invalid/no_op output (never raises)
- service set_agent_mode + Agent Mode switch + EN/DE/ES/FR/IT strings
- tests/test_agent_mode.py: generate_action behaviour + wiring contracts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements GA-AUD — the Home Assistant side of Agent Mode. The integration can now publish controller actions to the executor shipped in #92 (GA-109), so Agent Mode works end-to-end.
When enabled, each analyzed frame additionally:
PromptBuilder.build_action),parse_action),gaming_assistant/{client_id}/actionfor the executor.The normal tip pipeline is unchanged — this is an additional inference per frame (the chosen "tip + action" behavior).
Safety model
CONF_AGENT_MODE(defaultFalse), exposed via a newgaming_assistant.set_agent_modeservice and an Agent Mode switch entity.entry.dataonly, so I deliberately did not persist agent mode. The AI never controls inputs unless deliberately re-enabled.asyncio.wait_for+ a broadtry/except, so a bad model reply / timeout can never disrupt the tip pipeline.parse_action;no_op/invalid output is dropped (never published).--dry-run, and emergency stop. Recommended first run: executor in--dry-run.Changes
CONF_AGENT_MODE,DEFAULT_AGENT_MODE,MQTT_ACTION_TOPIC,AGENT_VALID_BUTTONS.agent_modeflag +set_agent_mode()setter,async_publish_action(), and the isolated_maybe_publish_agent_action()hook after a successful tip.generate_action()—build_action→ LLM →parse_action; returnsNoneon invalid/no_opoutput, never raises.set_agent_mode(+services.yaml), Agent Mode switch entity, andagent_modestrings in EN/DE/ES/FR/IT.tests/test_agent_mode.py— behavioural tests forgenerate_action+ static wiring contracts (the coordinator can't be imported without a real HA env, mirroringtest_coordinator.py).Test plan
python -m unittest discover -s tests→ 313 tests, all green (+15 new).pytest --cov=custom_components/gaming_assistant --cov-fail-under=45→ 48.30%.py_compile+ JSON validation for all touched files.Follow-up
Per-action confirmation UI (the remaining half of GA-AUD) is intentionally out of scope.
https://claude.ai/code/session_01VaYnzPqHF5kxB3stM73g5i
Generated by Claude Code