Schema-aware property tooling for AI agents (CMS 17.4.0)

[mattbrailsford]

Summary

• Bumps the Umbraco CMS dependency floor to 17.4.0-rc2 (.NET + npm) and consumes the new IValueSchemaProvider / IPropertyEditorSchemaService APIs so AI agents have a programmatic source of truth for the value shape every property editor accepts on write.
• Threads the resulting JSON Schema through the agent tooling that LLMs use to plan and execute property writes — embedded per-property in get_content_type_schema, available standalone via get_property_value_schema, and inlined into the Entity Context block of the system prompt so the LLM sees the write-shape next to the rendered current value without an extra tool call.
• Adds a shallow x-allowedElementTypes enrichment to block-list / block-grid / rich-text-with-blocks schemas (the CMS only emits element-type GUIDs, no aliases) plus a guidance note pointing the LLM at get_content_type_schema for follow-up element-type schemas. Closes the LLM hallucination of block names.
• Tightens the set_value frontend tool description to be explicit about replace semantics, the read-merge-write requirement for collections, and the schema-first workflow for non-string editors.
• Hardens the AG-UI streaming layer so failures actually surface: tool exceptions become structured ToolInvocationError results instead of opaque [unknown:ErrorContent], MEAI ErrorContent (provider rate-limits, content filters) is logged and rendered inline, and unhandled AIContent subtypes log at debug level so nothing else can disappear silently.
• Adds Information-level diagnostic logging at every hop in the tool-call streaming pipeline (AIToolReorderingChatClient, AGUIStreamingService, AIFrontendToolFunction) so the next time a tool call goes missing the cause is one log filter away.

Out of scope / known limitations

The schema-discovery and write-friendly tooling makes the LLM able to author block-list / block-grid values, but not always willing — Anthropic Sonnet 4.6 in particular sometimes substitutes narration ("I'll add the block...") for a tool_use emission when the structured argument is large. The pipeline routes the call correctly when it does emit one (verified by the new diagnostics on a Sonnet vs. GPT cross-check). This is a tool-granularity issue, not a pipeline bug: set_value asks the model to author the full layout/contentData/expose envelope with cross-referenced GUIDs, which models are genuinely bad at. Follow-up issue: a family of frontend domain-operation tools (add_block, remove_block, set_block_property, etc.) that take simpler inputs and do the envelope assembly client-side. Not in this PR.

Detail by area

Dependency bump

• Directory.Packages.props: every Umbraco.Cms.* floor bumped from [17.3.0, 17.999.999) to [17.4.0-rc2, 17.999.999). RC version pinning is required because npm/NuGet semver excludes prereleases from stable lower bounds. Microsoft.Extensions.Caching.Memory and Microsoft.Extensions.Options floors raised to 10.0.6 to satisfy the new CMS transitive requirement.
• package.json files (root + 5 client workspaces): @umbraco-cms/backoffice peer bumped to ^17.4.0-rc2. diff bumped to ^9.0.0 to satisfy the new backoffice peer; @types/diff removed (diff@9 ships its own types).
• Provider packages.lock.json files updated to reflect transitive bumps.
• When 17.4.0 stable releases the floor should be relaxed back to [17.4.0, 17.999.999) / ^17.4.0.

Schema-aware tools (Umbraco.AI/src/Umbraco.AI.Core/Tools/Umbraco/)

• GetContentTypeSchemaTool accepts both alias and GUID, falls through Content → Element → Media → Member buckets, and survives the cache's not-registered throws. Each property in the response now carries its DataTypeKey (Guid) and ValueSchema (JsonObject) — schemas resolved in parallel via IPropertyEditorSchemaService.GetSchemaAsync.
• GetPropertyValueSchemaTool — new tool for focused single-data-type schema lookups. Friendly messages for DataTypeNotFound and SchemaNotSupported operation statuses.
• BlockSchemaEnricher — walks any schema the schema service produced and attaches x-allowedElementTypes ({ key, alias } per allowed element type) and x-allowedElementTypesNote next to every contentTypeKey.enum. Shallow / lazy by design — the LLM follows up via get_content_type_schema(elementTypeKey) when it actually needs property schemas, mirroring the CMS's own $ref lazy-resolution philosophy. Defensive deep-clone before mutating prevents JsonNode parent-tracking serialisation failures.

Entity Context schema embedding

• CmsEntityFormatHelper.FormatCmsEntity now accepts the published-content-type cache + schema service and inlines per-property editor, current value, and input shape (JSON Schema) into the Entity Context system prompt. Document/Media/MemberEntityAdapter inject the services from DI. Existing fall-back rendering preserved when services are absent.

set_value description

• Drops the "Only supports TextBox and TextArea" gate. Calls out the schema-first workflow for non-string editors and the read-merge-write workflow for collections (block list, block grid, multi-node tree picker, multi-url picker, multi-image media picker, tags). Tighter shape after experimentation showed Sonnet treats long prescriptive text as workflow documentation.

Failure visibility

• AIToolFunction<TArgs>.InvokeCoreAsync wraps tool execution in try/catch that logs (Umbraco.AI.Tools.<toolName>) and returns a ToolInvocationError { success, toolName, errorType, message } record instead of bubbling. Replaces the opaque MEAI default of [unknown:ErrorContent] with no body.
• AGUIStreamingService.StreamCoreAsync handles MEAI ErrorContent (logs + emits inline as a [Provider error <code>: <message>] text chunk so the run continues per MEAI docs), explicitly no-ops TextContent to avoid double-emit with update.Text, and logs unhandled content types at debug.
• New diagnostic logs at AIToolReorderingChatClient (buffered tool names + registered frontend tools), AGUIStreamingService (every FunctionCallContent received), and AIFrontendToolFunction (when the frontend-tool function fires + a warning when FunctionInvokingChatClient.CurrentContext is null).

Test plan

• CI green
• Manual smoke on the demo site against a doc type with a block list / block grid property:
  • get_content_type_schema returns each property's ValueSchema and embeds x-allowedElementTypes on block properties
  • Entity Context system prompt shows editor / current value / input shape per property
  • LLM (GPT) authors and stages a block-list write through set_value without manually reading the schema
  • Negative path: an editor without IValueSchemaProvider still renders correctly with ValueSchema: null
• When stable 17.4.0 ships, relax floors per the comment in Directory.Packages.props

Commits

• feat(core,copilot,deps): schema-aware property tooling
• fix(core): GUID input + throwing-cache resilience on get_content_type_schema
• feat(core): block list/grid x-allowedElementTypes enrichment (initially full)
• fix(core): deep-clone inner element-type schemas before attaching
• refactor(core): shallow + lazy enrichment (drops recursive property schemas)
• fix(copilot): replace + read-merge requirements on set_value
• fix(core): surface tool exceptions instead of [unknown:ErrorContent]
• fix(agent): surface MEAI ErrorContent in AG-UI stream
• fix(copilot): tighten set_value description for Sonnet
• chore(agent): diagnostic logging at every hop in the tool-call streaming pipeline