Skip to content

feat: Property value operation handlers + generic AI tools#160

Merged
mattbrailsford merged 20 commits into
devfrom
feature/cms-1740-property-value-handlers
May 14, 2026
Merged

feat: Property value operation handlers + generic AI tools#160
mattbrailsford merged 20 commits into
devfrom
feature/cms-1740-property-value-handlers

Conversation

@mattbrailsford
Copy link
Copy Markdown
Contributor

Stacked on top of #153 — must not merge before #153 lands. The diff here is only the changes added on top.

Summary

Replaces the single, coarse set_value frontend tool with a family of five generic LLM-visible tools (add_item, remove_item, move_item, set_value, clear_value) backed by a schema-aware backend dispatcher and a C# property value handler plugin surface (IAIPropertyValueHandler). Sonnet 4.6 reliably bails into narration when asked to author block-list / block-grid / RTE envelopes (cross-referenced layout/contentData/expose arrays with fresh GUIDs threaded through); these tools shift envelope assembly to the server so the LLM only has to produce 1-4 short fields per call.

The shape mirrors what Sanity ships in production with their Content Agent (operation primitives + path-with-key-predicates + schema validation at dispatch). Same dispatcher, same handlers will be reused by future server-side tools that mutate content not currently in any user's UI.

Architecture

  • Backend dispatcher (IAIPropertyValueDispatcher) walks an [propertyAlias, { blockKey: 'X' }, ...] path, calls handlers, returns the mutated root value. Stateless: never reads or writes data — the caller supplies rootValue, the caller persists the result.
  • Stateless transformation endpoint (POST /umbraco/ai/management/api/v1/property-value-operation) is the HTTP surface frontend tools consume. Locked behind SectionAccessAI. The endpoint never touches the DB. This lets frontend tools transport the workspace's staged value (preserving the user's unsaved edits) through the dispatcher and back.
  • Handler plugin surface (IAIPropertyValueHandler) is C#, auto-discovered via IDiscoverable, registered through builder.AIPropertyValueHandlers(). Third-party editor authors add a handler to gain AI authoring capability with no further plumbing — same lift as registering a custom property editor.
  • First-party handlers ship for Umbraco.BlockList, Umbraco.BlockGrid (root-level only in v1), Umbraco.RichText (rejects AddItem — adding to value.blocks without a markup placeholder produces an orphan), Umbraco.MediaPicker3, Umbraco.MultiUrlPicker.
  • Frontend tools are thin HTTP consumers: read root value from workspace, POST through the typed PropertyValueOperationsService (hey-api), apply returned value via applyValueChange (the existing workspace staging path — the user's typing and our staging share the same write surface, so dirty-state and undo history keep working).

Load-bearing guarantees

The dispatcher endpoint never reads the property value itself. The caller always supplies it.

A controller-level test asserts this directly: any future change that has the controller swap the supplied rootValue for a freshly-fetched DB read silently breaks the design that lets frontend tools transport unsaved staged changes. The test fails loudly if regressed.

The matching frontend guard — prepareValueForEditor skip-path for pre-built block envelopes (commit 0a8b175f) — protects the staged result from being silently corrupted on the way back into the workspace.

What's intentionally limited in v1

  • Block-grid: root-level operations only. gridArea parameter is reserved on the contract for v2 row/area/span work; supplying it today returns a structured operation-not-supported error so the LLM can correct.
  • RTE-with-blocks AddItem: rejected. Edit RTE markup via set_value to insert a placeholder, then set_value against the embedded block's properties via path.
  • Default-value provider: stub returning null until CMS exposes a unified server-side propertyValuePreset equivalent (CMS-side proposal). In practice, handlers expose canonical empty representations themselves.

Tests

  • 42 new unit tests under Umbraco.AI.Tests.Unit/PropertyValueOperations/ (dispatcher, path JSON converter, envelope ops, BlockList / BlockGrid / RichText / MediaPicker3 / MultiUrlPicker handlers).
  • 1 controller test asserting the staged-value rule.
  • Full unit suite: 794 tests pass, no regressions.
  • All five frontend packages build clean (core → prompt → agent → agent-ui → copilot).

Test plan / remaining validation (out of this PR's autonomous scope)

The plan called for three follow-up phases that need an interactive browser session against the demo. They aren't blocking this PR's review but should land before the work goes to release:

  • Phase 6 — Sonnet path-construction spike: with the BlockList handler + add_item only, verify Sonnet 4.6 reliably constructs paths for a depth-2 nested operation across ~5 phrasings (e.g. "add a hero block to the inner blocks of the row block"). If reliability is low, tighten tool descriptions and add a path examples section to the entity context preamble. Worst case: fall back to active-entity convention with an open_block tool — flagged so we surface it before demo prep.
  • Phase 7 — Demo seed + docs:
    • Verify the seeded "Content Assistant" agent uses AllowedToolScopeIds: ["entity-write"] not explicit IDs (if explicit, add the new tool IDs to the seed, or new tools are invisible in demo).
    • Update docs/public/add-ons/agent/permissions.md with the new tool IDs + migration path for explicit allow-lists.
    • New docs/public/add-ons/agent/property-value-handlers.md — third-party C# handler authoring guide.
  • Phase 8 — End-to-end verification matrix on demo (Sonnet 4.6 + GPT-4):
    • "Add a hero block to contentRows with title X" → add_item(path: ["contentRows"], elementType: "hero", values: { title: "X" })
    • "Add a row with two heroes inside" → 1× add_item (returns rowKey) → 2× add_item(path: ["rows", { blockKey: rowKey }, "innerBlocks"], …)
    • "Change the title of the hero block to Z" → set_value(path: ["contentRows", { blockKey }, "title"], value: "Z")
    • "Remove the third block" → remove_item(path: ["contentRows"], blockKey: <key>)
    • "Set the hero image to media XYZ" → add_item(path: ["heroImage"], values: { mediaKey: "XYZ" }) against MediaPicker3
    • "Clear the contentRows" → clear_value(path: ["contentRows"])
    • "Add a hero block" against an RTE property → expect { success: false, error: { code: "operation-not-supported", … } }
    • Stretch: multi-variant document, segmented document, depth-2 nested edit, AI mutation while user has unsaved staged changes (verify staged changes survive).

CMS-side proposals (separate issues, not blocking)

To file against umbraco/Umbraco.CMS after this lands:

  1. Native x-allowedElementTypes enrichment on block schemas — currently we patch in aliases via BlockSchemaEnricher.
  2. Headless createWithPresets exposure — block managers' assembly logic is editor-component-scoped; a callable service would let our handlers shrink and stay in lock-step with CMS internals.
  3. Server-side IPropertyValueDefaultProvider — defaults are scattered across IDataValueEditor.DefaultValue, IPropertyType.GetDefaultValue(), etc. A unified surface would let our DefaultValueProvider be a thin wrapper.

🤖 Generated with Claude Code

mattbrailsford and others added 16 commits May 7, 2026 10:01
Introduces the backend infrastructure for AI-driven property value mutations:
IAIPropertyValueHandler (the public C# plugin surface for editor-specific
operations), AIPropertyValueDispatcher (path walker + operation router), and
the supporting models (request, result, path segments, args, errors).

The dispatcher is stateless: it never reads or writes data, operating purely
on the root value supplied by the caller. This is what makes it shareable
between frontend tools (which transport the workspace's staged value over
HTTP) and future server-side tools (which will pass the database-read value
in-process). Path segments alternate between property aliases and block-key
selectors, supporting arbitrary nesting depth.

Handlers are auto-discovered via IDiscoverable and registered through
builder.AIPropertyValueHandlers(). The default-value provider is currently a
stub returning null until CMS exposes a unified server-side preset surface
(filed as a CMS-side proposal in the plan).

Tests cover dispatcher behaviour at depth 1-3, error propagation paths
(no-handler, invalid-path, validation failure), and the path JSON converter's
round-trip serialisation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds POST /umbraco/ai/management/api/v1/property-value-operation as the
HTTP transport that frontend tools use to invoke the property value
dispatcher. The endpoint is a stateless transformation: the caller supplies
the root value (the workspace's staged state), the dispatcher mutates it
in-memory, and the response carries the new root value back. The endpoint
never reads from or writes to the database.

A controller-level test asserts the staged-value rule directly: any
mutation that has the controller swap the supplied rootValue for a
freshly-fetched DB read silently breaks the design that lets frontend
tools transport unsaved staged changes through this endpoint, so the
test fails loudly if regressed.

Auth: SectionAccessAI (matching the rest of the AI management API).
Group: ai-management (so the existing OpenAPI client picks it up after
regeneration in Phase 5).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships handlers for the five complex Umbraco editors that need AI authoring
support: BlockList, BlockGrid (root-level only in v1), RichText, MediaPicker3,
and MultiUrlPicker. Block-shaped editors share BlockEnvelopeOps + ExposeBuilder
for layout/contentData/expose mutation; pickers manage flat item arrays
directly.

Behaviour notes per the plan:
- BlockGrid validates against any 'Extra' fields (gridArea, columnSpan, etc.)
  so the LLM gets a clear OperationNotSupported error rather than corrupting
  the layout. Reserved for v2.
- RichText rejects AddItem unconditionally — adding to value.blocks without a
  corresponding markup placeholder produces an orphan. Other ops walk into
  value.blocks transparently.
- MediaPicker3 has no item-level properties; SetItemPropertyValue throws on
  any alias.
- MultiUrlPicker accepts SetItemPropertyValue for the editable scalar fields
  (name, target, queryString, url) so the dispatcher can target nested edits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a tool stages a fully-built envelope as an object (e.g. the value
returned by the property value operation dispatcher), the existing
JSON.parse / RichText-wrap branches silently corrupted it: the parse
threw and got swallowed, then the RichText branch stringified the still-
intact object, mangling the envelope on its way to the workspace.

The guard short-circuits when the supplied value is a non-null object
and the editor schema alias is one of the known block-shaped editors
(BlockList, BlockGrid, RichText). All other editor / type combinations
flow through the existing path unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds UAI_ENTITY_ADAPTER_CONTEXT in the copilot package and provides it
from UaiCopilotContext alongside the existing chat / hitl / entity
contexts. Tools that need structured operations (the property value
operation tools introduced for AI-driven authoring) consume this token
to read full property envelopes and apply value changes through the
existing workspace staging path.

The token lives in the copilot package rather than @umbraco-ai/core
because cross-package context tokens hit a TypeScript resolution
mismatch — UmbContextToken's #private field tracks each workspace's
local @umbraco-cms/backoffice resolution separately, so a token
exported from core is nominally a different type when consumed in
copilot. Hosting the token in the package that both provides and
consumes it sidesteps that. The thinner UAI_ENTITY_CONTEXT contract
in agent-ui still serves chat surfaces that don't need structured
operations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ation types

Picks up the new POST /property-value-operation endpoint added in the
preceding commit, producing the typed PropertyValueOperationsService and
its request/response models. Also adds the relevant types to the public
exports so consuming packages (Umbraco.AI.Agent.Copilot) can import them
directly from @umbraco-ai/core rather than reaching into internal paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships the five generic LLM-visible tools that consume the dispatcher
endpoint: add_item, remove_item, move_item, set_value, clear_value.
Each tool is a thin wrapper around PropertyValueOperationToolBase, which
handles the common flow:

- Resolve the entity adapter context.
- Read the staged root value from the workspace's selected entity.
- Build the dispatch request with active variant + document metadata.
- POST via the typed PropertyValueOperationsService (hey-api client) so
  bearer-token auth, 401 retry, and standard error envelope all flow
  through tryExecute unchanged.
- Apply the returned new root value via applyValueChange (the existing
  workspace staging path - the user's typing and our staging share the
  same write surface, so dirty-state and undo history keep working).

set_value is fully replaced with a path-based implementation; the prior
single-string-path arg shape is intentionally dropped (internal AI tool
arg shapes are not under any back-compat guarantee).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six small cleanups surfaced by code review, none affecting the public
plugin surface beyond making BlockEditorHandlerBase a public extension
point for third-party block-shaped editors:

- Extract BlockEditorHandlerBase as the shared base for BlockList /
  BlockGrid handlers. The two concrete handlers now hold only the editor
  schema alias, the layout key, and a layout-entry builder; everything
  else (add/remove/move/clear, expose, item-property accessors,
  content-type and editor-alias resolution) lives in one place.
- Picker handlers' TryGetGuid now delegates to BlockEnvelopeOps.GetGuid
  (promoted from private to internal) — same body in three places becomes
  one.
- Move parsePath / readVariant from add-item.api.ts to
  internal/path-args.ts so the four sibling tools no longer cross-import
  from each other (a leaky API).
- Replace the local uuidv4() in entity-adapter/value-preparation.ts with
  crypto.randomUUID(), matching the convention already used in 8+ places
  across the client codebase.
- Drop the redundant upfront path-shape validation loop in the dispatcher.
  Per-segment shape is enforced by the JSON converter at deserialization;
  the descent's casts surface any other mismatch as a structured
  InvalidPath. The remaining check (leaf must be a property alias) is the
  only constraint the converter alone can't express.
- Trim a handful of narrative comments that restated the surrounding code
  (file:line markers like "Resolve handler", "Snapshot the frame so we
  can ascend later", etc.). Kept the WHY comments.

Build and test: 794 unit tests pass; all five frontend packages build
clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The MediaPicker3 thumbnail subcomponent fetches the rendered media on
mount and doesn't re-fetch when its mediaKey prop changes within an
existing picker entry. So when an LLM swaps an image via set_value, the
underlying value is correct (right mediaKey, right entry shape) but the
visible thumbnail keeps showing the previous image until the user saves
and reloads.

Lit re-mounts a subcomponent only when its surrounding entry key
changes, so to force a fresh fetch we re-mint the entry's `key`
(distinct from `mediaKey`) when its content actually changed. The prior
hand-rolled `[0].key = uuidv4()` only fired on string-parsed input and
only touched the first entry; the new path:

- Compares each incoming entry to the same-keyed entry in the current
  staged value.
- Regenerates the key only when the entry exists in the old value AND
  its non-key content differs. This forces re-mount for replaced
  entries.
- Leaves keys alone when content is identical (no churn for entries the
  LLM didn't touch — focal points, crops, etc. stay stable).
- Leaves keys alone for entries whose key isn't in the old value at all
  — those are genuinely new (e.g. minted by `add_item`'s handler) and
  the LLM may reference them via `remove_item({ blockKey })` later, so
  we must not change them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the hardcoded editor-alias branches in
prepareValueForEditor with a manifest-driven extension type so
third-party editors can plug in their own AI value-preparation logic
without core changes.

Public surface:
- ManifestUaiPropertyValuePreparer with forPropertyEditorSchemaAlias
- UaiPropertyValuePreparerApi (single prepare(value, currentValue) method)
- UAI_PROPERTY_VALUE_PREPARER_EXTENSION_TYPE constant
- resolveAndPrepareValue(value, editorAlias, currentValue)

First-party preparers, registered as manifests:
- BlockEnvelopePreparer for Umbraco.BlockList and Umbraco.BlockGrid
  (short-circuits already-built object envelopes; falls back to
  JSON.parse for string input)
- RichTextPreparer for Umbraco.RichText (object guard plus the
  {markup, blocks} wrap for bare markup strings, preserving the
  workspace's existing blocks envelope)
- MediaPicker3Preparer for Umbraco.MediaPicker3 (smart entry-key
  regen against the staged value; force re-mount on content change,
  preserve handler-minted keys for genuinely new entries)

Editors with no registered preparer fall back to a permissive default
(attempt JSON.parse on string input, return as-is).

The three adapter callers (document, media, block) now await the
async resolver; prepareValueForEditor and its hardcoded list are
deleted.

Build and test: 794 unit tests pass; all five frontend packages
build clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The valuePreparerManifests array was created in
entity-adapter/value-preparers/manifests.ts and re-exported via
entity-adapter/manifests.ts, but the root manifests.ts only imports
entityAdapterManifests by name and spreads that — so the preparer
manifests never made it into the bundle's exported manifests array
and weren't registered with the extension registry at runtime.

Net effect: every editor fell through to the default JSON.parse
fallback, which regressed the MediaPicker3 thumbnail refresh fix
and disabled the block-envelope and rich-text guards.

Add the explicit import and spread alongside entityAdapterManifests
so the preparers actually reach the registry.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rty-value-handlers

# Conflicts:
#	Directory.Packages.props
#	Umbraco.AI.Amazon/src/Umbraco.AI.Amazon/packages.lock.json
#	Umbraco.AI.Anthropic/src/Umbraco.AI.Anthropic/packages.lock.json
#	Umbraco.AI.Google/src/Umbraco.AI.Google/packages.lock.json
#	Umbraco.AI.MicrosoftFoundry/src/Umbraco.AI.MicrosoftFoundry/packages.lock.json
#	Umbraco.AI.OpenAI/src/Umbraco.AI.OpenAI/packages.lock.json
#	package-lock.json
…-rc2

Lowers Microsoft.EntityFrameworkCore.Design floor to 10.0.6 to match the EF Core
version Umbraco.Cms 17.4.0-rc2 transitively bundles. The previous 10.0.7 pin was
ahead of any published CMS release and surfaced as `Could not load
Microsoft.EntityFrameworkCore, Version=10.0.7.0` at runtime when the migrations
DLLs referenced assemblies the demo host couldn't resolve.

Pins Umbraco.Templates::17.4.0-rc2 in the install script so the demo bundles the
same CMS minor the AI Core packages compile against. Latest-stable currently
resolves to 17.3, which lacks IPropertyEditorSchemaService.

Refreshes provider lockfiles to the aligned transitive graph.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
BlockListModel / BlockGridModel / RichTextBlockModel aren't IPublishedContent or
IPublishedElement, so FormatElementOrDefault returned them unchanged for the
JSON serializer to walk. The serializer then traversed
`IPublishedElement.ContentType.PropertyTypes[].ContentType` indefinitely and
tripped the depth=64 cap with `A possible object cycle was detected`.

Detect IBlockReference<IPublishedElement, IPublishedElement> (covers all three
model collections plus strongly-typed subclasses like BlockListItem<MyHero>)
and format Content / Settings / grid Areas recursively through the safe
FormatElement path. Adds a depth cap as a defensive backstop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CMS date picker stores values as `yyyy-MM-dd HH:mm:ss` and its #formatValue
splits on a literal space, rendering the input blank for anything with `T`.
LLMs almost always emit ISO 8601 (often with timezone suffix or millisecond
fractions), and date-only strings are common too.

The new preparer converts the `T` separator to a space, strips trailing `Z` /
`±HH:mm` timezone offsets and fractional seconds (the editor is timezone-naive
and second-precision), and pads date-only input to midnight so the editor's
split-on-space invariant holds.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nstead of via the wire

Frontend tools cannot read an editor alias for a property the workspace hasn't
staged yet (`getValues()` only lists properties whose value has been touched).
A brand-new document with empty fields therefore sent
`rootEditorSchemaAlias: ""` and the endpoint rejected it with `The
RootEditorSchemaAlias field is required` — a 400 the LLM had no way to correct.

The dispatcher already resolves editor aliases via IContentTypeService during
the descent walk for nested properties. Canonicalise root resolution the same
way: derive the alias from (documentMetadata.contentTypeKey, path[0].alias)
and drop the wire field from AIPropertyValueDispatchRequest,
PropertyValueOperationRequestModel, and the frontend client interface.

47 dispatcher/controller tests rewritten to set up content types via a small
BuildContentTypeServiceWithTypes helper.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mattbrailsford and others added 4 commits May 12, 2026 10:25
The set_value / add_item / remove_item / move_item / clear_value tools
only stage changes in the workspace - the user still has to hit Save to
persist them. HITL approval is being moved to the save and save_and_publish
tools (on feature/cms-1740-ai-save-tools) where it gates the actual commit
to the database.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A renderer's job is to display a tool call in chat (icon, label, custom
result UI). Whether a tool needs HITL approval is an execution concern,
not a rendering concern, so it belongs on the frontend tool manifest.

- ManifestUaiAgentFrontendTool.meta gains approval?: UaiAgentToolApprovalConfig
- ManifestUaiAgentToolRenderer.meta loses approval (purely UI now)
- Executor reads approval from the frontend tool manifest
- hitl-approval element looks up the frontend tool manifest for static config
- UaiAgentToolApprovalConfig dropped elementAlias - custom approval elements
  are deferred; if a backend tool needs a custom approval UI it can embed the
  config inline in its interrupt payload
- confirm_action example moved its approval config from renderer to tool

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings in the CMS 17.4.0 stable bump from #153 and rolls 17.4.0-rc2

references in this branch forward to 17.4.0. Updates the EF Core comment

to reflect that 17.4.0 stable still bundles EF Core 10.0.6 (same as rc2).
@mattbrailsford mattbrailsford changed the base branch from feature/cms-1740-schema-aware-tools to dev May 14, 2026 07:36
@mattbrailsford mattbrailsford merged commit 2c37037 into dev May 14, 2026
16 of 23 checks passed
@mattbrailsford mattbrailsford deleted the feature/cms-1740-property-value-handlers branch May 14, 2026 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant