Skip to content

fix: document publish-only type validation and empty traces on --api-name preview - #322

Open
james-enperso wants to merge 1 commit into
forcedotcom:mainfrom
james-enperso:fix-agent-publish-type-and-trace-gaps
Open

fix: document publish-only type validation and empty traces on --api-name preview#322
james-enperso wants to merge 1 commit into
forcedotcom:mainfrom
james-enperso:fix-agent-publish-type-and-trace-gaps

Conversation

@james-enperso

Copy link
Copy Markdown

Summary

Three gaps in the agentforce-generate reference docs, each hit while wiring an Agent Script
service agent to an apex:// invocable. All changes are additive — no existing guidance is
removed or contradicted without evidence.

Changes

references/complex-data-types.md

  • Note that sf agent validate authoring-bundle returns success: true for an action param with
    the wrong numeric type — the check is server-side and only runs at publish. Includes the verbatim
    400 message, which names the exact fix and is worth reading rather than guessing.
  • Distinguish Apex Integer from Apex Decimal.

references/agent-validation-and-debugging.md

  • Document that a preview started with --api-name writes trace files containing only {}; only
    --authoring-bundle produces readable traces.
  • Document the published-agent session layout (keyed by Bot ID, not agent name).
  • Add a discriminating-value fallback for verifying action invocation when no trace is available.

Motivation

1. A green validate is not evidence of correct types. The existing "fails at publish" note is
right, but in practice authors run sf agent validate, see success: true, and reasonably assume
the bundle is sound. Naming the validate/publish split explicitly saves a publish-fix-republish
cycle.

2. Integer and Decimal differ. The current "Key insight" reads as universal:

Bare number works in variable declarations but fails at publish in action inputs/outputs.

Observed on API v67.0 / sf CLI 2.145.6:

Apex type Declared as Publish
Integer (input and output) bare number rejected — 400 naming lightning__integerType
Decimal (output) bare number succeeds, correct values at runtime

The Decimal case held across two separate agents. Taken literally, the current wording sends
authors looking for a complex type a Decimal does not need. The decision tree's
lightning__doubleType entry has been left in place — this PR adds the observation alongside it
rather than overriding it, since inputs and flow:// targets were not exercised here.

3. The trace rule can be unsatisfiable. agent-validation-and-debugging.md is emphatic, and
rightly so:

After EVERY preview utterance, read the trace — not just the agent's text response.

But --api-name previews write {}. That is fine when --authoring-bundle is available — except a
service agent cannot always be previewed that way (on a restricted network that path was
unreachable while --api-name worked), which leaves the only usable preview path producing no
trace. Without a documented fallback the honest options are to skip verification or to overstate
the evidence. The added technique closes that gap: test against records whose values an LLM could
not guess, ideally ones that exercise a distinctive code path in the action implementation.

Concrete example from this work — an invocable falls back to Date.today().year() when a record
has no transactions. The agent reported that fallback year for exactly such a record, which is not
something the model could have produced without executing the Apex.

Notes

  • Verified on API v67.0 with sf CLI 2.145.6 against a single Enterprise Edition org. Happy to
    narrow the wording if the Decimal behaviour is version- or org-shaped rather than general.
  • Only skills/ is edited. plugins/builder/salesforce-development/skills/ holds an identical
    copy, but recent history shows it is produced by the release workflow rather than hand-edited, so
    it is left alone — glad to mirror the change if maintainers prefer.
  • The trace-evaluation addition includes an anchor link to #trace-evaluation in the same file.

…name preview

Three findings from wiring an Agent Script service agent to an apex://
invocable (API v67.0, sf CLI 2.145.6):

1. `sf agent validate authoring-bundle` returns success:true for an action
   param with the wrong numeric type. Only the server-side check during the
   publish step catches it, so a green validate is not evidence the action
   I/O types are correct.

2. Apex Integer and Apex Decimal do not behave the same. An Integer param
   as bare `number` is rejected at publish; a Decimal output as bare
   `number` publishes and runs correctly. The existing "bare number fails
   at publish in action inputs/outputs" note reads as universal, which
   sends authors hunting for a complex type a Decimal does not need.

3. Preview started with --api-name writes trace files containing only {}.
   Only the --authoring-bundle path produces readable traces. This is not
   academic: a service agent cannot always be previewed via
   --authoring-bundle, leaving --api-name as the only option and making the
   "read the trace after every utterance" rule unsatisfiable. Documented a
   discriminating-value fallback for that case.

All additive; no existing guidance removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant