fix(build-agent): validate harness/provider pairing, ask connection mode upfront

skills/build-agent/SKILL.md

@@ -34,6 +34,13 @@ self-hosts (it defaults to Agenta cloud, `https://cloud.agenta.ai`).
 - Ask the user for their API key. Point them to it: the **API keys** page in their Agenta
   project settings (on cloud, under `https://cloud.agenta.ai`). Ask for the host URL only if
   they self-host, in which case it is their own Agenta domain.
+- **Ask which LLM connection mode they want** (`llm.connection.mode` in the config below):
+  `self_managed` (they bring their own provider API key — the default this skill writes) or
+  `agenta` (use their Agenta project's own managed/subscription key, no separate provider key
+  needed). Only ask this for cloud users — a self-hosted instance almost always means
+  `self_managed`, since it doesn't carry Agenta's own managed keys. Only in the `self_managed`
+  case do you also need a provider key: confirm the provider (`anthropic`, for the default
+  `claude` harness — see the coupling note below) and point them to that provider's key page.
 - Then offer to set it up for them, either way they prefer:
   - **Write it for them:** create a `.env` file in the working directory with
     `AGENTA_API_KEY=...` (and `AGENTA_HOST=...` if self-hosting). `lib.sh` picks up a local
@@ -66,10 +73,19 @@ The rest is fixed boilerplate — copy it exactly:
 }
 ```
 
-Keep `llm` / `harness` / `runner` / `sandbox` exactly as above unless told otherwise. `model`
-is an alias (`sonnet`, `opus`, `haiku`, `default`), never a raw model id. For the full config
-field by field — tool entries, skill entries, and the fields that 500 if misplaced — read
-`references/config-schema.md`.
+Keep `llm` / `harness` / `runner` / `sandbox` exactly as above unless told otherwise, except
+for `llm.connection.mode` (`self_managed` vs `agenta` — see the credentials step above) and
+`llm.model` (an alias: `sonnet`, `opus`, `haiku`, `default`, never a raw model id). **This is
+not one of the "fewest calls" simplifications** — do not improvise or substitute a different
+provider here even for the simplest one-line agent. `harness.kind: "claude"` only runs
+Anthropic models: `llm.provider` must stay `"anthropic"`. Pairing `harness.kind: "claude"`
+with any other provider (e.g. an `"openai"`/raw-model-id config you might see elsewhere, such
+as in the platform's own bare API schema default — that default is unrelated to this skill and
+not compatible with this harness) is accepted by the create/commit endpoints with no error, but
+the agent then gets no Model & Harness resolved in the Agenta UI and never runs correctly.
+`create-agent.sh` / `update-agent.sh` now reject this mismatch before calling the API — if you
+hit that error, fix the config rather than retrying. For the full config field by field — tool
+entries, skill entries, and the fields that 500 if misplaced — read `references/config-schema.md`.
 
 ## The loop (run it in order, skip what the ask doesn't need)
 
@@ -87,8 +103,8 @@ field by field — tool entries, skill entries, and the fields that 500 if mispl
    names the exact tools in order and ends on the terminal action — otherwise the run wanders
    or stops short. Depth in `references/writing-instructions.md`.
 5. **Create and test in one shot:** `bash scripts/build.sh <config> <slug> "<test message>"`.
-   Read the OUTPUT and RESOLVED lines. RESOLVED must show
-   `harness=claude model=sonnet connection=self_managed`.
+   Read the OUTPUT and RESOLVED lines. RESOLVED must show `harness=claude model=sonnet` and
+   the `connection` mode you configured (`self_managed` or `agenta`).
 6. **If it needs a trigger**, create it against the variant id: a schedule with
    `bash scripts/create-schedule.sh`, an event subscription with
    `bash scripts/create-subscription.sh` (only after `discover-triggers.sh` and a `ready`
@@ -155,6 +171,15 @@ Keep to the loop above for a simple agent. Read one of these only when the task
 
 ## Hard rules (these prevent the usual failures)
 
+- **`harness`/`runner`/`sandbox` are fixed, not a "fewest calls" shortcut; `llm.provider` is
+  fixed to the harness.** Copy them byte-for-byte from the boilerplate, even for the simplest
+  one-line agent — do not simplify or substitute a provider here. `harness.kind: "claude"` only
+  runs Anthropic models; if `llm.provider` is anything but `"anthropic"`, the agent gets no
+  Model & Harness resolved in the Agenta UI and silently never runs, even though creation
+  itself succeeds with no error. `create-agent.sh` / `update-agent.sh` reject this mismatch
+  before calling the API — treat that rejection as a config bug to fix, never as a reason to
+  retry with different values. `llm.connection.mode` is the one field in this block you DO
+  choose (see the credentials step): `self_managed` or `agenta`.
 - **Fewest calls.** A no-tools agent is exactly two actions: write the config, run `build.sh`.
   Don't re-list, don't re-verify facts already given here, don't re-read the schema.
 - **Test before you declare done.** A claim with no `build.sh` / `test-agent.sh` behind it does

skills/build-agent/references/config-schema.md

@@ -48,12 +48,28 @@ Keep `llm`, `mcps`, `harness`, `runner`, and `sandbox` exactly as in the templat
 user asks otherwise.
 
 - `llm.model` is an **alias** — `sonnet`, `opus`, `haiku`, or `default` — never a raw model id.
-- `llm.provider` is `anthropic` and `llm.connection.mode` is `self_managed` in the default
-  setup.
+- `llm.provider` is `anthropic` in the default setup. `llm.connection.mode` is the one field in
+  this block you actually choose: `self_managed` (the user's own provider API key — the
+  default written above) or `agenta` (the project's own managed/subscription key, no separate
+  provider key needed). Ask the user which they want during credentials setup (`SKILL.md`);
+  don't assume `self_managed` for a cloud user without asking.
 - `harness.kind` is `claude`. `runner.kind` is `sidecar`. `sandbox.kind` is `local`.
 
-RESOLVED (printed by `test-agent.sh`) must echo these back:
-`harness=claude model=sonnet connection=self_managed`. If it does not, the run fell back to a
+**`harness.kind` and `llm.provider` are coupled, not independent fields.** `harness.kind:
+"claude"` only runs Anthropic models — `llm.provider` must stay `"anthropic"` whenever
+`harness.kind` is `"claude"`. Don't change `llm.provider`/`llm.model` while leaving
+`harness.kind: "claude"` in place, even if you've seen a different default somewhere else
+(for example, the platform's own bare API schema for this endpoint shows a different
+`provider`/`harness.kind` pairing entirely — that's the platform's raw baseline for building
+agents *without* this skill, and it is not compatible with the `claude` harness this skill
+uses). This mismatch is accepted by the create/commit endpoints with no server-side error,
+but the agent then gets no Model & Harness resolved in the Agenta UI and never runs
+correctly — `create-agent.sh` / `update-agent.sh` now check for it and refuse to proceed if
+found, rather than let you learn this from a silently broken agent. `llm.connection.mode` is
+not part of this coupling — either mode works with the `claude` harness.
+
+RESOLVED (printed by `test-agent.sh`) must echo these back: `harness=claude model=sonnet` and
+the `connection` mode you configured. If the harness or model differ, the run fell back to a
 default — fix the config and re-test.
 
 ## A tool entry (gateway object)

skills/build-agent/scripts/create-agent.sh

@@ -13,6 +13,8 @@ CONFIG_FILE="${1:?usage: create-agent.sh <config.json> <slug> [name]}"
 SLUG="${2:?usage: create-agent.sh <config.json> <slug> [name]}"
 NAME="${3:-$SLUG}"
 
+validate_agent_config "$CONFIG_FILE" || exit 1
+
 AGENT_CONFIG="$(cat "$CONFIG_FILE")"
 BODY="$(jq -n --arg slug "$SLUG" --arg name "$NAME" --argjson agent "$AGENT_CONFIG" '
   {application: {slug: $slug, name: $name,

skills/build-agent/scripts/lib.sh

@@ -42,3 +42,80 @@ _AUTH=(-H "Authorization: ApiKey $AGENTA_API_KEY" -H "Content-Type: application/
 agenta_post()   { curl -s -X POST   "$AGENTA_HOST$1" "${_AUTH[@]}" -d "$2"; }
 agenta_get()    { curl -s           "$AGENTA_HOST$1" "${_AUTH[@]}"; }
 agenta_delete() { curl -s -X DELETE "$AGENTA_HOST$1" "${_AUTH[@]}"; }
+
+# Checks that harness.kind and llm.provider are a consistent pairing, when both are
+# present in the given config file. The Claude Code harness ("claude") only runs
+# Anthropic models — pairing it with any other provider is accepted by the create/commit
+# endpoints with no server-side error, but the agent then gets no Model & Harness
+# resolved in the Agenta UI and won't run correctly. This check is tolerant of missing
+# fields (safe to use on a partial update-agent.sh delta, which merges onto the existing
+# config) — it only fires when both fields are actually present and inconsistent.
+#
+# Usage: _check_harness_provider_consistency <config.json>
+_check_harness_provider_consistency() {
+  local config_file="$1"
+  local harness_kind provider model
+  harness_kind="$(jq -r '.harness.kind // empty' "$config_file" 2>/dev/null || true)"
+  provider="$(jq -r '.llm.provider // empty' "$config_file" 2>/dev/null || true)"
+  model="$(jq -r '.llm.model // empty' "$config_file" 2>/dev/null || true)"
+
+  if [ -n "$harness_kind" ] && [ -n "$provider" ] && [ "$harness_kind" = "claude" ] && [ "$provider" != "anthropic" ]; then
+    {
+      echo "CONFIG ERROR: harness.kind=\"claude\" requires llm.provider=\"anthropic\" — the"
+      echo "  Claude Code harness only runs Anthropic models. Found llm.provider=\"$provider\","
+      echo "  llm.model=\"$model\" instead."
+      echo "  This exact mismatch is accepted by the API with no error, but the agent then"
+      echo "  gets no Model & Harness resolved in the Agenta UI and won't run correctly."
+      echo "  Fix: set llm.provider back to \"anthropic\" and llm.model to one of"
+      echo "  sonnet/opus/haiku/default (see SKILL.md). This skill does not currently support"
+      echo "  building agents on a non-Claude harness — do not substitute a different provider"
+      echo "  while leaving harness.kind=\"claude\"."
+    } >&2
+    return 1
+  fi
+  return 0
+}
+
+# Full validation for a brand-new agent config (create-agent.sh). Requires the fixed
+# boilerplate fields to all be present — a simple agent has no reason to omit any of
+# them — plus the harness/provider consistency check above.
+#
+# Usage: validate_agent_config <config.json>
+# Prints nothing on success. On failure, prints a specific CONFIG ERROR to stderr and
+# returns non-zero — the caller should stop and fix the config rather than proceed.
+validate_agent_config() {
+  local config_file="$1"
+  local field val
+
+  for field in '.llm.model' '.llm.provider' '.llm.connection.mode' \
+               '.harness.kind' '.runner.kind' '.sandbox.kind'; do
+    val="$(jq -r "$field // empty" "$config_file" 2>/dev/null || true)"
+    if [ -z "$val" ]; then
+      {
+        echo "CONFIG ERROR: required field $field is missing or empty in $config_file."
+        echo "  instructions / tools / mcps / skills are the only fields you write yourself —"
+        echo "  llm / harness / runner / sandbox must be copied exactly from the boilerplate"
+        echo "  in SKILL.md, even for the simplest agent. Fix the config before creating/testing."
+      } >&2
+      return 1
+    fi
+  done
+
+  _check_harness_provider_consistency "$config_file"
+}
+
+# Lighter validation for an update-agent.sh delta, which deep-merges onto the existing
+# config and so may legitimately omit any of the boilerplate fields. Only checks for an
+# internal inconsistency in what IS being submitted.
+#
+# LIMITATION: this does not fetch and merge the current committed config, so it only
+# catches a mismatch when both harness.kind and llm.provider are present together in the
+# delta itself. A delta that changes ONLY llm.provider (relying on merge to keep a
+# previous harness.kind: "claude") will NOT be caught here. This is why update-agent.sh's
+# own guidance is to always pass the complete config, boilerplate included — do that
+# rather than relying on this check as a full safety net for a minimal delta.
+#
+# Usage: validate_agent_config_delta <config.json>
+validate_agent_config_delta() {
+  _check_harness_provider_consistency "$1"
+}

skills/build-agent/scripts/update-agent.sh

@@ -18,6 +18,8 @@ VARIANT="${1:?usage: update-agent.sh <variant_id> <config.json> [message]}"
 CONFIG_FILE="${2:?usage: update-agent.sh <variant_id> <config.json> [message]}"
 MESSAGE="${3:-update agent config}"
 
+validate_agent_config_delta "$CONFIG_FILE" || exit 1
+
 AGENT_CONFIG="$(cat "$CONFIG_FILE")"
 BODY="$(jq -n --arg vid "$VARIANT" --arg msg "$MESSAGE" --argjson agent "$AGENT_CONFIG" \
   '{workflow_revision:{workflow_variant_id:$vid, message:$msg,