feat: Property value operation handlers + generic AI tools#160
Merged
Conversation
Introduces the backend infrastructure for AI-driven property value mutations: IAIPropertyValueHandler (the public C# plugin surface for editor-specific operations), AIPropertyValueDispatcher (path walker + operation router), and the supporting models (request, result, path segments, args, errors). The dispatcher is stateless: it never reads or writes data, operating purely on the root value supplied by the caller. This is what makes it shareable between frontend tools (which transport the workspace's staged value over HTTP) and future server-side tools (which will pass the database-read value in-process). Path segments alternate between property aliases and block-key selectors, supporting arbitrary nesting depth. Handlers are auto-discovered via IDiscoverable and registered through builder.AIPropertyValueHandlers(). The default-value provider is currently a stub returning null until CMS exposes a unified server-side preset surface (filed as a CMS-side proposal in the plan). Tests cover dispatcher behaviour at depth 1-3, error propagation paths (no-handler, invalid-path, validation failure), and the path JSON converter's round-trip serialisation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds POST /umbraco/ai/management/api/v1/property-value-operation as the HTTP transport that frontend tools use to invoke the property value dispatcher. The endpoint is a stateless transformation: the caller supplies the root value (the workspace's staged state), the dispatcher mutates it in-memory, and the response carries the new root value back. The endpoint never reads from or writes to the database. A controller-level test asserts the staged-value rule directly: any mutation that has the controller swap the supplied rootValue for a freshly-fetched DB read silently breaks the design that lets frontend tools transport unsaved staged changes through this endpoint, so the test fails loudly if regressed. Auth: SectionAccessAI (matching the rest of the AI management API). Group: ai-management (so the existing OpenAPI client picks it up after regeneration in Phase 5). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships handlers for the five complex Umbraco editors that need AI authoring support: BlockList, BlockGrid (root-level only in v1), RichText, MediaPicker3, and MultiUrlPicker. Block-shaped editors share BlockEnvelopeOps + ExposeBuilder for layout/contentData/expose mutation; pickers manage flat item arrays directly. Behaviour notes per the plan: - BlockGrid validates against any 'Extra' fields (gridArea, columnSpan, etc.) so the LLM gets a clear OperationNotSupported error rather than corrupting the layout. Reserved for v2. - RichText rejects AddItem unconditionally — adding to value.blocks without a corresponding markup placeholder produces an orphan. Other ops walk into value.blocks transparently. - MediaPicker3 has no item-level properties; SetItemPropertyValue throws on any alias. - MultiUrlPicker accepts SetItemPropertyValue for the editable scalar fields (name, target, queryString, url) so the dispatcher can target nested edits. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a tool stages a fully-built envelope as an object (e.g. the value returned by the property value operation dispatcher), the existing JSON.parse / RichText-wrap branches silently corrupted it: the parse threw and got swallowed, then the RichText branch stringified the still- intact object, mangling the envelope on its way to the workspace. The guard short-circuits when the supplied value is a non-null object and the editor schema alias is one of the known block-shaped editors (BlockList, BlockGrid, RichText). All other editor / type combinations flow through the existing path unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds UAI_ENTITY_ADAPTER_CONTEXT in the copilot package and provides it from UaiCopilotContext alongside the existing chat / hitl / entity contexts. Tools that need structured operations (the property value operation tools introduced for AI-driven authoring) consume this token to read full property envelopes and apply value changes through the existing workspace staging path. The token lives in the copilot package rather than @umbraco-ai/core because cross-package context tokens hit a TypeScript resolution mismatch — UmbContextToken's #private field tracks each workspace's local @umbraco-cms/backoffice resolution separately, so a token exported from core is nominally a different type when consumed in copilot. Hosting the token in the package that both provides and consumes it sidesteps that. The thinner UAI_ENTITY_CONTEXT contract in agent-ui still serves chat surfaces that don't need structured operations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ation types Picks up the new POST /property-value-operation endpoint added in the preceding commit, producing the typed PropertyValueOperationsService and its request/response models. Also adds the relevant types to the public exports so consuming packages (Umbraco.AI.Agent.Copilot) can import them directly from @umbraco-ai/core rather than reaching into internal paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships the five generic LLM-visible tools that consume the dispatcher endpoint: add_item, remove_item, move_item, set_value, clear_value. Each tool is a thin wrapper around PropertyValueOperationToolBase, which handles the common flow: - Resolve the entity adapter context. - Read the staged root value from the workspace's selected entity. - Build the dispatch request with active variant + document metadata. - POST via the typed PropertyValueOperationsService (hey-api client) so bearer-token auth, 401 retry, and standard error envelope all flow through tryExecute unchanged. - Apply the returned new root value via applyValueChange (the existing workspace staging path - the user's typing and our staging share the same write surface, so dirty-state and undo history keep working). set_value is fully replaced with a path-based implementation; the prior single-string-path arg shape is intentionally dropped (internal AI tool arg shapes are not under any back-compat guarantee). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six small cleanups surfaced by code review, none affecting the public plugin surface beyond making BlockEditorHandlerBase a public extension point for third-party block-shaped editors: - Extract BlockEditorHandlerBase as the shared base for BlockList / BlockGrid handlers. The two concrete handlers now hold only the editor schema alias, the layout key, and a layout-entry builder; everything else (add/remove/move/clear, expose, item-property accessors, content-type and editor-alias resolution) lives in one place. - Picker handlers' TryGetGuid now delegates to BlockEnvelopeOps.GetGuid (promoted from private to internal) — same body in three places becomes one. - Move parsePath / readVariant from add-item.api.ts to internal/path-args.ts so the four sibling tools no longer cross-import from each other (a leaky API). - Replace the local uuidv4() in entity-adapter/value-preparation.ts with crypto.randomUUID(), matching the convention already used in 8+ places across the client codebase. - Drop the redundant upfront path-shape validation loop in the dispatcher. Per-segment shape is enforced by the JSON converter at deserialization; the descent's casts surface any other mismatch as a structured InvalidPath. The remaining check (leaf must be a property alias) is the only constraint the converter alone can't express. - Trim a handful of narrative comments that restated the surrounding code (file:line markers like "Resolve handler", "Snapshot the frame so we can ascend later", etc.). Kept the WHY comments. Build and test: 794 unit tests pass; all five frontend packages build clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The MediaPicker3 thumbnail subcomponent fetches the rendered media on
mount and doesn't re-fetch when its mediaKey prop changes within an
existing picker entry. So when an LLM swaps an image via set_value, the
underlying value is correct (right mediaKey, right entry shape) but the
visible thumbnail keeps showing the previous image until the user saves
and reloads.
Lit re-mounts a subcomponent only when its surrounding entry key
changes, so to force a fresh fetch we re-mint the entry's `key`
(distinct from `mediaKey`) when its content actually changed. The prior
hand-rolled `[0].key = uuidv4()` only fired on string-parsed input and
only touched the first entry; the new path:
- Compares each incoming entry to the same-keyed entry in the current
staged value.
- Regenerates the key only when the entry exists in the old value AND
its non-key content differs. This forces re-mount for replaced
entries.
- Leaves keys alone when content is identical (no churn for entries the
LLM didn't touch — focal points, crops, etc. stay stable).
- Leaves keys alone for entries whose key isn't in the old value at all
— those are genuinely new (e.g. minted by `add_item`'s handler) and
the LLM may reference them via `remove_item({ blockKey })` later, so
we must not change them.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the hardcoded editor-alias branches in
prepareValueForEditor with a manifest-driven extension type so
third-party editors can plug in their own AI value-preparation logic
without core changes.
Public surface:
- ManifestUaiPropertyValuePreparer with forPropertyEditorSchemaAlias
- UaiPropertyValuePreparerApi (single prepare(value, currentValue) method)
- UAI_PROPERTY_VALUE_PREPARER_EXTENSION_TYPE constant
- resolveAndPrepareValue(value, editorAlias, currentValue)
First-party preparers, registered as manifests:
- BlockEnvelopePreparer for Umbraco.BlockList and Umbraco.BlockGrid
(short-circuits already-built object envelopes; falls back to
JSON.parse for string input)
- RichTextPreparer for Umbraco.RichText (object guard plus the
{markup, blocks} wrap for bare markup strings, preserving the
workspace's existing blocks envelope)
- MediaPicker3Preparer for Umbraco.MediaPicker3 (smart entry-key
regen against the staged value; force re-mount on content change,
preserve handler-minted keys for genuinely new entries)
Editors with no registered preparer fall back to a permissive default
(attempt JSON.parse on string input, return as-is).
The three adapter callers (document, media, block) now await the
async resolver; prepareValueForEditor and its hardcoded list are
deleted.
Build and test: 794 unit tests pass; all five frontend packages
build clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The valuePreparerManifests array was created in entity-adapter/value-preparers/manifests.ts and re-exported via entity-adapter/manifests.ts, but the root manifests.ts only imports entityAdapterManifests by name and spreads that — so the preparer manifests never made it into the bundle's exported manifests array and weren't registered with the extension registry at runtime. Net effect: every editor fell through to the default JSON.parse fallback, which regressed the MediaPicker3 thumbnail refresh fix and disabled the block-envelope and rich-text guards. Add the explicit import and spread alongside entityAdapterManifests so the preparers actually reach the registry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rty-value-handlers # Conflicts: # Directory.Packages.props # Umbraco.AI.Amazon/src/Umbraco.AI.Amazon/packages.lock.json # Umbraco.AI.Anthropic/src/Umbraco.AI.Anthropic/packages.lock.json # Umbraco.AI.Google/src/Umbraco.AI.Google/packages.lock.json # Umbraco.AI.MicrosoftFoundry/src/Umbraco.AI.MicrosoftFoundry/packages.lock.json # Umbraco.AI.OpenAI/src/Umbraco.AI.OpenAI/packages.lock.json # package-lock.json
…-rc2 Lowers Microsoft.EntityFrameworkCore.Design floor to 10.0.6 to match the EF Core version Umbraco.Cms 17.4.0-rc2 transitively bundles. The previous 10.0.7 pin was ahead of any published CMS release and surfaced as `Could not load Microsoft.EntityFrameworkCore, Version=10.0.7.0` at runtime when the migrations DLLs referenced assemblies the demo host couldn't resolve. Pins Umbraco.Templates::17.4.0-rc2 in the install script so the demo bundles the same CMS minor the AI Core packages compile against. Latest-stable currently resolves to 17.3, which lacks IPropertyEditorSchemaService. Refreshes provider lockfiles to the aligned transitive graph. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
BlockListModel / BlockGridModel / RichTextBlockModel aren't IPublishedContent or IPublishedElement, so FormatElementOrDefault returned them unchanged for the JSON serializer to walk. The serializer then traversed `IPublishedElement.ContentType.PropertyTypes[].ContentType` indefinitely and tripped the depth=64 cap with `A possible object cycle was detected`. Detect IBlockReference<IPublishedElement, IPublishedElement> (covers all three model collections plus strongly-typed subclasses like BlockListItem<MyHero>) and format Content / Settings / grid Areas recursively through the safe FormatElement path. Adds a depth cap as a defensive backstop. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CMS date picker stores values as `yyyy-MM-dd HH:mm:ss` and its #formatValue splits on a literal space, rendering the input blank for anything with `T`. LLMs almost always emit ISO 8601 (often with timezone suffix or millisecond fractions), and date-only strings are common too. The new preparer converts the `T` separator to a space, strips trailing `Z` / `±HH:mm` timezone offsets and fractional seconds (the editor is timezone-naive and second-precision), and pads date-only input to midnight so the editor's split-on-space invariant holds. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nstead of via the wire Frontend tools cannot read an editor alias for a property the workspace hasn't staged yet (`getValues()` only lists properties whose value has been touched). A brand-new document with empty fields therefore sent `rootEditorSchemaAlias: ""` and the endpoint rejected it with `The RootEditorSchemaAlias field is required` — a 400 the LLM had no way to correct. The dispatcher already resolves editor aliases via IContentTypeService during the descent walk for nested properties. Canonicalise root resolution the same way: derive the alias from (documentMetadata.contentTypeKey, path[0].alias) and drop the wire field from AIPropertyValueDispatchRequest, PropertyValueOperationRequestModel, and the frontend client interface. 47 dispatcher/controller tests rewritten to set up content types via a small BuildContentTypeServiceWithTypes helper. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
9 tasks
The set_value / add_item / remove_item / move_item / clear_value tools only stage changes in the workspace - the user still has to hit Save to persist them. HITL approval is being moved to the save and save_and_publish tools (on feature/cms-1740-ai-save-tools) where it gates the actual commit to the database. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A renderer's job is to display a tool call in chat (icon, label, custom result UI). Whether a tool needs HITL approval is an execution concern, not a rendering concern, so it belongs on the frontend tool manifest. - ManifestUaiAgentFrontendTool.meta gains approval?: UaiAgentToolApprovalConfig - ManifestUaiAgentToolRenderer.meta loses approval (purely UI now) - Executor reads approval from the frontend tool manifest - hitl-approval element looks up the frontend tool manifest for static config - UaiAgentToolApprovalConfig dropped elementAlias - custom approval elements are deferred; if a backend tool needs a custom approval UI it can embed the config inline in its interrupt payload - confirm_action example moved its approval config from renderer to tool Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings in the CMS 17.4.0 stable bump from #153 and rolls 17.4.0-rc2 references in this branch forward to 17.4.0. Updates the EF Core comment to reflect that 17.4.0 stable still bundles EF Core 10.0.6 (same as rc2).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on top of #153 — must not merge before #153 lands. The diff here is only the changes added on top.
Summary
Replaces the single, coarse
set_valuefrontend tool with a family of five generic LLM-visible tools (add_item,remove_item,move_item,set_value,clear_value) backed by a schema-aware backend dispatcher and a C# property value handler plugin surface (IAIPropertyValueHandler). Sonnet 4.6 reliably bails into narration when asked to author block-list / block-grid / RTE envelopes (cross-referenced layout/contentData/expose arrays with fresh GUIDs threaded through); these tools shift envelope assembly to the server so the LLM only has to produce 1-4 short fields per call.The shape mirrors what Sanity ships in production with their Content Agent (operation primitives + path-with-key-predicates + schema validation at dispatch). Same dispatcher, same handlers will be reused by future server-side tools that mutate content not currently in any user's UI.
Architecture
IAIPropertyValueDispatcher) walks an[propertyAlias, { blockKey: 'X' }, ...]path, calls handlers, returns the mutated root value. Stateless: never reads or writes data — the caller suppliesrootValue, the caller persists the result.POST /umbraco/ai/management/api/v1/property-value-operation) is the HTTP surface frontend tools consume. Locked behindSectionAccessAI. The endpoint never touches the DB. This lets frontend tools transport the workspace's staged value (preserving the user's unsaved edits) through the dispatcher and back.IAIPropertyValueHandler) is C#, auto-discovered viaIDiscoverable, registered throughbuilder.AIPropertyValueHandlers(). Third-party editor authors add a handler to gain AI authoring capability with no further plumbing — same lift as registering a custom property editor.Umbraco.BlockList,Umbraco.BlockGrid(root-level only in v1),Umbraco.RichText(rejectsAddItem— adding tovalue.blockswithout a markup placeholder produces an orphan),Umbraco.MediaPicker3,Umbraco.MultiUrlPicker.PropertyValueOperationsService(hey-api), apply returned value viaapplyValueChange(the existing workspace staging path — the user's typing and our staging share the same write surface, so dirty-state and undo history keep working).Load-bearing guarantees
A controller-level test asserts this directly: any future change that has the controller swap the supplied
rootValuefor a freshly-fetched DB read silently breaks the design that lets frontend tools transport unsaved staged changes. The test fails loudly if regressed.The matching frontend guard —
prepareValueForEditorskip-path for pre-built block envelopes (commit0a8b175f) — protects the staged result from being silently corrupted on the way back into the workspace.What's intentionally limited in v1
gridAreaparameter is reserved on the contract for v2 row/area/span work; supplying it today returns a structuredoperation-not-supportederror so the LLM can correct.AddItem: rejected. Edit RTE markup viaset_valueto insert a placeholder, thenset_valueagainst the embedded block's properties via path.nulluntil CMS exposes a unified server-sidepropertyValuePresetequivalent (CMS-side proposal). In practice, handlers expose canonical empty representations themselves.Tests
Umbraco.AI.Tests.Unit/PropertyValueOperations/(dispatcher, path JSON converter, envelope ops, BlockList / BlockGrid / RichText / MediaPicker3 / MultiUrlPicker handlers).Test plan / remaining validation (out of this PR's autonomous scope)
The plan called for three follow-up phases that need an interactive browser session against the demo. They aren't blocking this PR's review but should land before the work goes to release:
add_itemonly, verify Sonnet 4.6 reliably constructs paths for a depth-2 nested operation across ~5 phrasings (e.g. "add a hero block to the inner blocks of the row block"). If reliability is low, tighten tool descriptions and add apathexamples section to the entity context preamble. Worst case: fall back to active-entity convention with anopen_blocktool — flagged so we surface it before demo prep.AllowedToolScopeIds: ["entity-write"]not explicit IDs (if explicit, add the new tool IDs to the seed, or new tools are invisible in demo).docs/public/add-ons/agent/permissions.mdwith the new tool IDs + migration path for explicit allow-lists.docs/public/add-ons/agent/property-value-handlers.md— third-party C# handler authoring guide.add_item(path: ["contentRows"], elementType: "hero", values: { title: "X" })add_item(returns rowKey) → 2×add_item(path: ["rows", { blockKey: rowKey }, "innerBlocks"], …)set_value(path: ["contentRows", { blockKey }, "title"], value: "Z")remove_item(path: ["contentRows"], blockKey: <key>)add_item(path: ["heroImage"], values: { mediaKey: "XYZ" })against MediaPicker3clear_value(path: ["contentRows"]){ success: false, error: { code: "operation-not-supported", … } }CMS-side proposals (separate issues, not blocking)
To file against
umbraco/Umbraco.CMSafter this lands:x-allowedElementTypesenrichment on block schemas — currently we patch in aliases viaBlockSchemaEnricher.createWithPresetsexposure — block managers' assembly logic is editor-component-scoped; a callable service would let our handlers shrink and stay in lock-step with CMS internals.IPropertyValueDefaultProvider— defaults are scattered acrossIDataValueEditor.DefaultValue,IPropertyType.GetDefaultValue(), etc. A unified surface would let ourDefaultValueProviderbe a thin wrapper.🤖 Generated with Claude Code