Use toggle UI, auto-save, and inline DataForm settings for experiments by jorgefilipecosta · Pull Request #376 · WordPress/ai

jorgefilipecosta · 2026-04-07T16:30:22Z

Summary

Replace checkbox controls with toggle switches on the AI settings page
Auto-save settings on each toggle change instead of requiring a manual "Save Changes" button click
Show contextual snackbar notices (e.g. "AI enabled.", "Content Classification disabled.")
Allow experiments to declare custom settings fields via get_settings_fields() on Abstract_Feature, rendered as inline DataForms below the experiment toggle when enabled
Migrate Content Classification's custom settings (taxonomy strategy, max suggestions) from server-rendered HTML to DataForm fields with show_in_rest support
Custom experiment settings use a separate Save button that only appears when changes are pending

Test plan

Visit the AI settings page and verify toggles render instead of checkboxes
Toggle a setting and verify it auto-saves with a contextual snackbar message
Disable the global AI toggle and verify feature toggles become disabled
Enable Content Classification and verify the strategy and max suggestions fields appear inline below the toggle
Disable Content Classification and verify the sub-settings disappear
Change the taxonomy strategy, verify the Save button appears, click it, and confirm the snackbar and value persistence on reload
Verify /wp/v2/settings/ REST API returns wpai_feature_content-classification_field_strategy and wpai_feature_content-classification_field_max_suggestions
Run E2E tests: npm run test:e2e -- --grep "Plugin settings" and npm run test:e2e -- --grep "Content Classification"

Use the DataForm 'toggle' field type and ToggleControl component for a more intuitive on/off UI on the AI settings page.

Remove manual save button clicks from test helpers since settings now auto-save on toggle.

github-actions · 2026-04-07T16:30:34Z

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: jorgefilipecosta <jorgefilipecosta@git.wordpress.org>
Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

codecov · 2026-04-07T16:34:03Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.81%. Comparing base (ffd1eb4) to head (e0bddc2).
⚠️ Report is 58 commits behind head on develop.

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #376      +/-   ##
=============================================
+ Coverage      62.63%   63.81%   +1.17%     
- Complexity       680      684       +4     
=============================================
  Files             49       49              
  Lines           3501     3529      +28     
=============================================
+ Hits            2193     2252      +59     
+ Misses          1308     1277      -31

Flag	Coverage Δ
unit	`63.81% <100.00%> (+1.17%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

dkotter

Tested and works well for me, noting this also seems to solve the problem reported in #352, which is great.

There are E2E failures here with Content Classification, I'm assuming because the individual settings it needs no longer display, so I think fine to ignore that until we have a PR up to fix that.

That said, I do wonder how this will scale for features that need their own settings (like Content Classification). Is the plan that anytime you change an individual setting we save immediately, similar to how these toggles work? Seems like in that scenario having a save button still may be best

jeffpaul

Tests real nicely, the toggles and autosave are great to avoid the extra button clicks, thanks! Will await @dkotter review before approval+merge.

jeffpaul · 2026-04-07T17:33:10Z

lol Darin 🥇, JP 🥈

jeffpaul · 2026-04-07T17:37:28Z

As a reference, here's what the Content Classification settings bits look like:

ToggleControl associates its label differently than CheckboxControl in the accessibility tree, so getByRole('checkbox', { name }) fails. Use getByLabel which works reliably with both control types.

Allow experiments to declare custom settings field definitions via get_settings_fields() for DataForm rendering. Add get_settings_fields_metadata() to resolve short field IDs to full WordPress option names for REST API consumption.

Replace render_settings_fields() HTML with get_settings_fields() field definitions. Add show_in_rest with proper schemas to register_settings() so custom fields are accessible via the WordPress REST API and core-data store.

Pass each feature's custom settings field definitions to the settings page script data so the frontend can render DataForms for experiment-specific settings.

Add InlineFeatureSettings component that renders a DataForm with custom fields below the experiment toggle when enabled. Uses local state with a Save button that only appears when changes are pending. The sub-settings appear inline within the same card as the toggle.

Navigate to the new settings page URL and use label-based selectors with snackbar confirmation instead of the old form submission approach.

…ggleWithSettings

jorgefilipecosta · 2026-04-08T17:14:49Z

Hi @jeffpaul, @dkotter,
I included a change in this PR to add additional settings field:

When there are changes it renders a save button for the setting with inner changes:

Enabling disabling is still immediate.

Cover get_settings_fields(), get_settings_fields_metadata(), and settingsFields in bootstrap feature metadata output. Includes tests for Content_Classification field declarations and show_in_rest.

The enableExperiment helper uses page.getByLabel() which matches against the toggle's display label, not the slug. Updated the constant to use 'Content Classification' matching all other tests.

Move the settings data lookup to a ref so the callback identity stays stable across saves, avoiding unnecessary DataForm re-renders caused by the data dependency changing on every settings update.

Reset localEdits when the feature id changes so stale keys from a previous field definition do not persist into the new one.

Replace coordinated mutable state (savingRef, dataRef, localEdits, editComponentsRef) with core-data's built-in dirty tracking. Toggle auto-save now uses __experimentalSaveSpecifiedEntityEdits to persist only the toggled key, so rapid clicks are never dropped and inline drafts are never flushed. InlineFeatureSettings writes directly to the entity record store and saves only its own field IDs. Update E2E helpers to assert snackbar text instead of test ID.

jeffpaul · 2026-04-08T19:29:08Z

Content Classification settings work as expected, though the horizontal line and ALL CAPS for sub-experiment sections feels off from a UI perspective. Any chance we can remove the separator, switch to Title Case, and left align the sub-experiment settings with the experiment title & description?

The inline settings Save button was clickable while saving because it only had isBusy without disabled. Also, isSavingEntityRecord('root', 'site') fired for any site entity save including toggle auto-saves, causing false spinner activation. Replace the store-level isSaving with local useState so the button only reflects the inline settings save, and add disabled={isSaving} to prevent concurrent requests.

…a11y Output DataForm-compatible field definitions from PHP (use 'text' type, nest min/max under isValid) so the frontend can pass them through without transformation. Remove parseSettingsField and the type-mapping useMemo. Replace the module-level Map component cache with a single stable FeatureToggleWithSettings component that looks up its feature via an immutable FEATURES_BY_SETTING Map. Add aria-label to the inline Save button so screen readers can distinguish between multiple feature settings forms.

…dent - Remove border-top separator between toggle and sub-settings - Override Emotion text-transform: uppercase on BaseControl and InputControl labels with !important - Indent sub-settings with padding-left to align under toggle label - Constrain form width with max-width to prevent overly wide inputs

jorgefilipecosta · 2026-04-08T20:24:09Z

Content Classification settings work as expected, though the horizontal line and ALL CAPS for sub-experiment sections feels off from a UI perspective. Any chance we can remove the separator, switch to Title Case, and left align the sub-experiment settings with the experiment title & description?

Hi @jeffpaul, I tried to address your feedback:

Extract MIN_SUGGESTIONS and MAX_SUGGESTIONS class constants in Content_Classification to eliminate duplicated magic numbers across register_settings, get_settings_fields, and sanitize_max_suggestions. Move the DisabledToggle no-op onChange handler to a module-level constant to avoid allocating a new function on every render.

dkotter

There may still be work being done here but just noting there's a failing eslint check and E2E tests are also still failing

includes/bootstrap.php

jorgefilipecosta · 2026-04-09T09:05:25Z

There may still be work being done here but just noting there's a failing eslint check and E2E tests are also still failing

Hi @dkotter it was from in progress work, everything is green now.

jorgefilipecosta added 3 commits April 7, 2026 17:30

Switch settings page controls from checkboxes to toggles

c62c394

Use the DataForm 'toggle' field type and ToggleControl component for a more intuitive on/off UI on the AI settings page.

Remove save button styles from settings page

ed9a3a8

Update E2E helpers for auto-save behavior

93268ad

Remove manual save button clicks from test helpers since settings now auto-save on toggle.

jeffpaul added this to the 0.7.0 milestone Apr 7, 2026

jeffpaul added this to WordPress AI Planning & Roadmap Apr 7, 2026

jeffpaul moved this to Needs review in WordPress AI Planning & Roadmap Apr 7, 2026

jeffpaul requested review from dkotter and jeffpaul April 7, 2026 16:55

jeffpaul assigned jorgefilipecosta Apr 7, 2026

dkotter previously approved these changes Apr 7, 2026

View reviewed changes

jeffpaul reviewed Apr 7, 2026

View reviewed changes

Update E2E selectors from getByRole to getByLabel for toggles

ab9e199

ToggleControl associates its label differently than CheckboxControl in the accessibility tree, so getByRole('checkbox', { name }) fails. Use getByLabel which works reliably with both control types.

jorgefilipecosta dismissed dkotter’s stale review via ab9e199 April 8, 2026 15:10

theskinnyghost mentioned this pull request Apr 8, 2026

Allow bulk enabling/disabling experiments in groups #379

Closed

jorgefilipecosta added 7 commits April 8, 2026 16:14

lint fix

d9b47d9

Add get_settings_fields API to Abstract_Feature

786cb6c

Allow experiments to declare custom settings field definitions via get_settings_fields() for DataForm rendering. Add get_settings_fields_metadata() to resolve short field IDs to full WordPress option names for REST API consumption.

Migrate Content Classification settings to DataForm fields

e3538a1

Replace render_settings_fields() HTML with get_settings_fields() field definitions. Add show_in_rest with proper schemas to register_settings() so custom fields are accessible via the WordPress REST API and core-data store.

Include settingsFields in feature metadata for frontend

9ea2b2f

Pass each feature's custom settings field definitions to the settings page script data so the frontend can render DataForms for experiment-specific settings.

Update E2E setStrategy helper for DataForms settings page

42efcb9

Navigate to the new settings page URL and use label-based selectors with snackbar confirmation instead of the old form submission approach.

Add missing JSDoc params to InlineFeatureSettings and createFeatureTo…

4d95d11

…ggleWithSettings

jorgefilipecosta changed the title ~~Use toggle UI and auto-save for settings page~~ Use toggle UI, auto-save, and inline DataForm settings for experiments Apr 8, 2026

jorgefilipecosta added 2 commits April 8, 2026 18:33

Add PHP tests for settings fields API and bootstrap metadata

215197c

Cover get_settings_fields(), get_settings_fields_metadata(), and settingsFields in bootstrap feature metadata output. Includes tests for Content_Classification field declarations and show_in_rest.

Fix content-classification E2E tests to use display label

a77607e

The enableExperiment helper uses page.getByLabel() which matches against the toggle's display label, not the slug. Updated the constant to use 'Content Classification' matching all other tests.

jeffpaul requested a review from dkotter April 8, 2026 17:57

jorgefilipecosta added 3 commits April 8, 2026 19:36

Stabilize handleChange by reading data from a ref

2b84cf0

Move the settings data lookup to a ref so the callback identity stays stable across saves, avoiding unnecessary DataForm re-renders caused by the data dependency changing on every settings update.

Clear inline settings pending edits when feature definition changes

38f3309

Reset localEdits when the feature id changes so stale keys from a previous field definition do not persist into the new one.

jeffpaul mentioned this pull request Apr 8, 2026

Improve AI Experiments page UI with header, accordion, and entry points #153

Closed

7 tasks

jeffpaul mentioned this pull request Apr 8, 2026

Release version 0.7.0 #383

Open

31 tasks

jorgefilipecosta added 2 commits April 8, 2026 21:14

fix alignment

6385943

jeffpaul previously approved these changes Apr 8, 2026

View reviewed changes

jorgefilipecosta dismissed jeffpaul’s stale review via fc72a08 April 8, 2026 20:37

jorgefilipecosta added 3 commits April 8, 2026 21:50

fixes

65ae809

lint fixes

7a77733

lint fix

574c656

dkotter requested changes Apr 8, 2026

View reviewed changes

includes/bootstrap.php Outdated Show resolved Hide resolved

jorgefilipecosta added 5 commits April 8, 2026 22:25

lint fix try 2

9e679fc

feedback

d3be6e9

ent to end fixes

689c3a5

e2e fixes

32301c3

lint fix

e0bddc2

dkotter approved these changes Apr 9, 2026

View reviewed changes

dkotter merged commit aa6409b into develop Apr 9, 2026
19 of 20 checks passed

dkotter deleted the feature/settings-toggle-autosave branch April 9, 2026 13:33

github-project-automation bot moved this from Needs review to Done in WordPress AI Planning & Roadmap Apr 9, 2026

This was referenced Apr 9, 2026

Try and get LLMs to return content matching the language of the original content #357

Merged

feat: Content Provenance experiment (C2PA 2.3 §A.7 text authentication) #294

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use toggle UI, auto-save, and inline DataForm settings for experiments#376

Use toggle UI, auto-save, and inline DataForm settings for experiments#376
dkotter merged 46 commits intodevelopfrom
feature/settings-toggle-autosave

jorgefilipecosta commented Apr 7, 2026 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

codecov bot commented Apr 7, 2026 •

edited

Loading

Uh oh!

dkotter left a comment

Uh oh!

jeffpaul left a comment

Uh oh!

jeffpaul commented Apr 7, 2026

Uh oh!

jeffpaul commented Apr 7, 2026

Uh oh!

jorgefilipecosta commented Apr 8, 2026

Uh oh!

jeffpaul commented Apr 8, 2026

Uh oh!

jorgefilipecosta commented Apr 8, 2026

Uh oh!

dkotter left a comment

Uh oh!

Uh oh!

jorgefilipecosta commented Apr 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jorgefilipecosta commented Apr 7, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

github-actions bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dkotter left a comment

Choose a reason for hiding this comment

Uh oh!

jeffpaul left a comment

Choose a reason for hiding this comment

Uh oh!

jeffpaul commented Apr 7, 2026

Uh oh!

jeffpaul commented Apr 7, 2026

Uh oh!

jorgefilipecosta commented Apr 8, 2026

Uh oh!

jeffpaul commented Apr 8, 2026

Uh oh!

jorgefilipecosta commented Apr 8, 2026

Uh oh!

dkotter left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jorgefilipecosta commented Apr 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jorgefilipecosta commented Apr 7, 2026 •

edited by github-actions bot

Loading

github-actions bot commented Apr 7, 2026 •

edited

Loading

codecov bot commented Apr 7, 2026 •

edited

Loading