fix(build-agent): validate harness/provider pairing, ask connection mode upfront

[bekossy]

Context

When you build a simple, no-tool Agenta agent through this skill, the Agenta UI's Model & Harness sometimes never resolves (shows "Select a provider", nothing selected), even though creation reports success. Reproduced 4 times during QA testing against a self-hosted instance, across two coding agents.

Root cause: the coding agent occasionally wrote a config that paired harness.kind: "claude" (the Claude Code harness, which only runs Anthropic models) with llm.provider: "openai" and a raw model id. The create and commit endpoints accept this mismatch with no error, but the agent never resolves to a working Model & Harness and never runs correctly.

Changes

lib.sh gets two new checks: validate_agent_config (full check, used by create-agent.sh) and validate_agent_config_delta (lighter check, tolerant of a partial update-agent.sh delta). Both refuse a config where harness.kind is "claude" and llm.provider is anything other than "anthropic".

Before, this config was accepted silently:

{"harness": {"kind": "claude"}, "llm": {"provider": "openai", "model": "gpt-4o-mini"}}

After, create-agent.sh and update-agent.sh reject it before making any API call, with a message explaining the coupling and how to fix it.

Also, per review feedback on the credentials setup flow: the skill previously always wrote llm.connection.mode as "self_managed" without asking. It now explicitly asks the user which mode they want at setup, self_managed (bring your own provider key) or agenta (use the project's own managed key), and only asks for a provider key in the self_managed case. This is orthogonal to the harness/provider coupling above; both connection modes still require llm.provider: "anthropic" when harness.kind is "claude".

SKILL.md and config-schema.md now explain the coupling explicitly, since it is not one of the "fewest calls" simplifications a caller should improvise on.

Tests

No existing test harness for the shell scripts in this repo, so verified end to end against a live self-hosted instance:

• A mismatched config (harness=claude, provider=openai) is rejected client-side by create-agent.sh. No API call is made, exit 1 with the new error message.
• A config with harness=claude, provider=anthropic, connection.mode=agenta passes validation (confirms the new connection mode does not get flagged as a mismatch).
• A known-good boilerplate config still creates and invokes fine, no regression.
• update-agent.sh with a partial delta (only instructions) still succeeds, confirming the lighter delta check does not over-reject legitimate partial updates.
• All edited scripts pass bash -n syntax checks.

One documented limitation, called out in lib.sh's comment: validate_agent_config_delta only catches a harness/provider mismatch when both fields are present together in the delta itself. A delta that changes only llm.provider (relying on merge to keep a previous harness.kind: "claude") will not be caught. update-agent.sh's own guidance is to always pass the complete config, which avoids this gap.

What to QA

• Ask the coding agent to build a simple, no-tool agent. During credentials setup it should now ask which llm.connection.mode you want, not just silently assume self_managed.
• Try to get it to write a mismatched harness/provider config (ask for a non-Claude model while keeping the default harness). create-agent.sh should refuse with the CONFIG ERROR message rather than creating a broken agent.
• Regression: build a normal agent with the default boilerplate. It should still create and pass RESOLVED as before.

Independent of the trigger-revision-pinning PR (different scripts; only a shared SKILL.md/lib.sh edit in a different location of each file). Should merge cleanly in either order, or with a trivial rebase.

Closes #3.

[mmabrouk]

this is wrong. @bekossy you need to explain the meaning of self-managed to the model. it always makes this mistake. self-managed means the runner is using subscription claude code / codex. which means no need to configure the api key. In cloud you can't use self-managed at all. only agenta managed (meaning the user provide their api keys in agenta and they are used)

[bekossy]

You're right, I had it backwards. Fixed in config-schema.md and SKILL.md: self_managed means the runner itself is logged into its own Claude Code / Codex subscription, so Agenta needs no provider API key at all. agenta means the user's provider key lives in Agenta's vault and Agenta uses it on every call. Also added the point that self_managed isn't available on cloud at all since cloud runners have no subscription of their own, so cloud always means agenta. Added a setup transcript to the PR description showing the corrected flow, and resolved the merge conflicts from #12/#14.

[mmabrouk]

Thanks @bekossy please like the first pr show examples of transcripts before and after (from setup with the agent asking to set up your connection up to this point)

[bekossy]

Addressed both:

• Fixed the self_managed/agenta semantics (was backwards, see reply on the inline comment) in SKILL.md and config-schema.md.
• Added a "Setup transcript" section to the PR description showing the before/after conversational flow, from the agent asking about self-hosted vs cloud through the connection-mode question.
• Resolved the merge conflicts from fix(build-agent): pin trigger revisions, default to latest when omitted #12/fix(build-agent): print event descriptions from discover-triggers.sh #14 (both already merged to main).

Ready for another look.