Skip to content

Commit b6b6dff

Browse files
gsoldevilacursoragent
gsoldevila
and
cursoragent
authored
[docs] Update SO validate troubleshooting section for new PR comment format (#270927)
## Summary Updates `docs/extend/saved-objects/validate.md` to reflect the structured error reporting introduced in #268469 and the additional validation rules added in subsequent PRs. ### What changed **Format**: The CI check now posts a structured PR comment (`**[rule-id]** Message. _Fix:_ …`) instead of raw `❌` terminal output. The troubleshooting intro is updated to explain the new format and show how to reproduce findings locally. **New rules documented** (were missing from the original section): | Rule ID | Introduced in | |---|---| | `existing-type/schema-breaking-changes` | #268630 | | `existing-type/schema-undiffable-legacy-hash` | #268630 | | `existing-type/new-mappings-not-in-model-version` | #268630 | | `existing-type/keyword-missing-ignore-above` | #268630 | | `existing-type/invalid-name-title-field-type` | #268630 | | `new-type/missing-initial-model-version` | #268469 | | `new-type/legacy-migrations` | #268469 | | `new-type/keyword-missing-ignore-above` | #270541 | | `new-type/invalid-name-title-field-type` | #270541 | | `model-version/mappings-not-in-schema` | #268630 | | `model-version/mapping-index-false` | #268630 | | `model-version/mapping-enabled-false` | #268630 | | `model-version/fixture-missing` | #270541 | | `model-version/fixture-invalid` | #270541 | | `documents/fixture-mismatch` | #270541 | **Structure**: Rules are now grouped into categories (existing-type / new-type / model-version / documents / removed-type) with stable anchor IDs on every rule heading, so `([docs](link))` references in PR comments land directly on the right entry. ## Test plan - [ ] Visual review of rendered markdown in the Elastic docs preview Made with [Cursor](https://cursor.com) --------- Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent d99c401 commit b6b6dff

7 files changed

Lines changed: 241 additions & 85 deletions

File tree

‎.buildkite/scripts/steps/checks/notify_saved_objects_changes.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ export interface SavedObjectsCheckReport {
7171

7272
const COMMENT_CONTEXT = 'saved-objects-check';
7373
const DOCS_BASE_URL = 'https://www.elastic.co/docs/extend/kibana/saved-objects';
74-
const TROUBLESHOOTING_URL = `${DOCS_BASE_URL}/validate#troubleshooting`;
74+
const TROUBLESHOOTING_URL = `${DOCS_BASE_URL}/troubleshooting`;
7575
const MODEL_VERSIONS_URL = `${DOCS_BASE_URL}#defining-model-versions`;
7676

7777
function hasSoChanges(report: SavedObjectsCheckReport): boolean {

‎docs/extend/saved-objects.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,8 @@ This documentation is organized into the following sections:
6363
* [Structure](saved-objects/structure.md) — Parts of a Saved Object type definition (name, index pattern, mappings, model versions) and the structure of a model version.
6464
* [Create a type](saved-objects/create.md) — Register a new Saved Object type, define mappings and references, and define the initial model version.
6565
* [Update a type](saved-objects/update.md) — Upgrade existing Saved Object types (legacy transition and new model versions).
66-
* [Validate type changes](saved-objects/validate.md) — Test model versions, ensure safe type definition changes, and troubleshoot validation failures.
66+
* [Validate type changes](saved-objects/validate.md) — Test model versions and ensure safe type definition changes.
67+
* [Troubleshooting validation](saved-objects/troubleshooting.md) — Resolve CI failures and startup errors from Saved Object type checks.
6768
* [Delete a type](saved-objects/delete.md) — Remove a Saved Object type registration.
6869
* [CRUD operations](saved-objects/use.md) — Perform CRUD on Saved Object instances via the Core service (create, get, find, update, delete). Do not use the deprecated HTTP API.
6970
* [Export and import](saved-objects/export.md) — Exporting and importing Saved Object documents.

‎docs/extend/saved-objects/troubleshooting.md‎

Lines changed: 233 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
---
2+
navigation_title: Troubleshooting validation
3+
---
4+
5+
This page helps you resolve failures from the Saved Object type validation check in CI and at {{kib}} startup.
6+
7+
# CI is failing for a PR [saved-objects-troubleshooting-ci]
8+
9+
When the *Check changes in saved objects* CI step fails, a comment is posted directly on the PR. Each violation is grouped by SO type and shows a **rule ID**, a message, a fix hint, and a link to the relevant docs section:
10+
11+
```markdown
12+
### `<soType>`
13+
- **[rule-id]** Message. _Fix:_ Fix hint. ([docs](link))
14+
```
15+
16+
You can reproduce all findings locally:
17+
18+
```shell
19+
# Get the merge-base with main (or the base branch of your PR)
20+
git merge-base HEAD main
21+
22+
# Run the check. Add "--fix" to auto-generate missing fixture templates
23+
node scripts/check_saved_objects --baseline <mergeBase> --fix
24+
```
25+
26+
Use the rule IDs below to identify the problem and the appropriate fix.
27+
28+
## Existing type rules
29+
30+
### `existing-type/mutated-migrations` [existing-type-mutated-migrations]
31+
32+
**Problem:** The deprecated `migrations` property has been modified. It must remain unchanged so that old documents can still be imported.
33+
34+
**Fix:** Revert any changes to `<type>.migrations`. If you need to change migration behavior, add a new model version instead.
35+
36+
### `existing-type/mutated-existing-model-version` [existing-type-mutated-model-version]
37+
38+
**Problem:** A structural change was made to an already-defined model version (mappings, changes block, or similar). Existing model versions are immutable once merged.
39+
40+
**Scenario 1:** You are fixing a bug in an existing model version. Once it is released (for example, in Serverless), other nodes may already be running against it. Add a new model version instead of editing the existing one.
41+
42+
**Scenario 2:** You only added one new version but CI still reports a mutation. Validation uses two baselines: the PR merge-base and the **current Serverless release**. If the Serverless release includes a version that differs from your local copy, the check sees a mutation. Rebase onto the latest `main` to pick up any updates.
43+
44+
**Fix:** Revert the structural change and add a new model version to capture the update.
45+
46+
### `existing-type/schema-breaking-changes` [existing-type-schema-breaking-changes]
47+
48+
**Problem:** A breaking schema change was detected in an existing model version. For example, a field was removed, its type changed, or an optional field was made required. These changes would break documents already stored against that version.
49+
50+
**Fix:** Revert the breaking schema change. If the update is necessary, introduce it in a new model version.
51+
52+
### `existing-type/schema-undiffable-legacy-hash` [existing-type-schema-undiffable]
53+
54+
**Problem:** A schema-only change was detected in an existing model version, but the baseline snapshot stores the schema as a legacy SHA-256 hash rather than a structured object. Detailed diffing is not possible against the old format.
55+
56+
**Fix:** Rebase onto the latest `main` to obtain a baseline snapshot with the new format, then re-run the check.
57+
58+
### `existing-type/deleted-model-versions` [existing-type-deleted-model-versions]
59+
60+
**Problem:** One or more model versions were deleted from the type. Your branch may be behind the current Serverless baseline and missing recent versions.
61+
62+
**Fix:** Restore the missing model version(s). Existing model versions cannot be deleted.
63+
64+
### `existing-type/too-many-new-model-versions` [existing-type-too-many-model-versions]
65+
66+
**Problem:** A single PR is adding more than one model version for the same type. Only one new model version per type per PR is allowed to support safe, incremental Serverless rollouts.
67+
68+
**Scenario 1:** You have several unrelated changes that each require a model version. Ship them in separate PRs.
69+
70+
**Scenario 2:** You only added one version but CI still reports two. Validation uses two baselines: your PR merge-base and the **current Serverless release**. If the Serverless release was recently rolled back, the "current release" baseline may make your branch look like it defines two new versions. Wait for the release state to normalize, or contact the {{kib}} Core team.
71+
72+
**Fix:** Split the change so that each PR adds exactly one new model version.
73+
74+
### `existing-type/mappings-changed-without-new-model-version` [existing-type-mappings-without-model-version]
75+
76+
**Problem:** The type's mappings were modified without adding a new model version. Mapping changes must be declared in a model version so the migration algorithm can keep the index up to date.
77+
78+
**Fix:** Add a new model version with a `mappings_addition` change block and the corresponding `schemas.forwardCompatibility` (and `schemas.create` if applicable).
79+
80+
### `existing-type/new-mappings-not-in-model-version` [existing-type-new-mappings-not-in-model-version]
81+
82+
**Problem:** The new model version's `mappings_addition` change does not include all of the newly introduced mapping fields.
83+
84+
**Fix:** Add the missing fields to the `mappings_addition` change in the new model version.
85+
86+
### `existing-type/removed-mapped-properties` [existing-type-removed-mapped-properties]
87+
88+
**Problem:** One or more mapped properties were removed from the type. {{es}} does not allow removing fields from a live index without a full reindex.
89+
90+
**Fix:** Restore the removed mapped properties. To stop writing to a field, leave it in the mappings but stop populating it from application code.
91+
92+
### `existing-type/virtual-version-downgrade` [existing-type-virtual-version-downgrade]
93+
94+
**Problem:** The type's computed virtual version is lower than it was in the baseline. This is usually caused by removing a model version.
95+
96+
**Fix:** Restore the missing model version(s) so the virtual version is at least as high as the baseline.
97+
98+
### `existing-type/keyword-missing-ignore-above` [existing-type-keyword-ignore-above]
99+
100+
**Problem:** A newly introduced `keyword` or `flattened` mapping field is missing an `ignore_above` limit. Without it, {{es}} silently drops strings that exceed the default limit.
101+
102+
**Fix:** Add `ignore_above: 1024` (or another appropriate limit) to each affected field.
103+
104+
### `existing-type/invalid-name-title-field-type` [existing-type-invalid-name-title]
105+
106+
**Problem:** A `name` or `title` field in the type's mappings uses a type other than `text`. The Saved Objects Search API relies on these fields being `text` for full-text search.
107+
108+
**Fix:** Change the field mapping type to `text`. If the field already exists in production, this requires a full reindex.
109+
110+
## New type rules
111+
112+
### `new-type/missing-initial-model-version` [new-type-missing-initial-model-version]
113+
114+
**Problem:** A brand-new SO type does not define model version `1`.
115+
116+
**Fix:** Add a `modelVersions` entry with key `1` containing `schemas.create` and `schemas.forwardCompatibility`.
117+
118+
### `new-type/legacy-migrations` [new-type-legacy-migrations]
119+
120+
**Problem:** A new SO type is using the deprecated `migrations` property.
121+
122+
**Fix:** Remove `migrations` and replace it with `modelVersions`, starting at version `1`.
123+
124+
### `new-type/keyword-missing-ignore-above` [new-type-keyword-ignore-above]
125+
126+
Same cause and fix as [`existing-type/keyword-missing-ignore-above`](#existing-type-keyword-ignore-above), but for a brand-new type.
127+
128+
### `new-type/invalid-name-title-field-type` [new-type-invalid-name-title]
129+
130+
Same cause and fix as [`existing-type/invalid-name-title-field-type`](#existing-type-invalid-name-title), but for a brand-new type.
131+
132+
## Model version rules
133+
134+
These rules apply to both new and existing types.
135+
136+
### `model-version/initial-must-be-schema-only` [model-version-initial-schema-only]
137+
138+
**Problem:** The initial model version (`1`) of a new type defines a `changes` block (for example, `mappings_addition`). For backward-compatibility reasons, the first model version can only contain `schemas`.
139+
140+
**Fix:** Remove `changes` from model version `1` and keep only `schemas`.
141+
142+
### `model-version/numbers-must-be-consecutive` [model-version-consecutive]
143+
144+
**Problem:** Model version keys must be consecutive positive integers starting at `1`. Either a key is not a number or there is a gap in the sequence.
145+
146+
**Fix:** Rename version keys to form an unbroken sequence: `1`, `2`, `3`, …
147+
148+
### `model-version/missing-schemas` [model-version-missing-schemas]
149+
150+
**Problem:** A new model version is missing the `schemas` definition entirely.
151+
152+
**Fix:** Add a `schemas` object with both `create` and `forwardCompatibility` sub-schemas.
153+
154+
### `model-version/missing-forward-compatibility` [model-version-missing-fwd-compat]
155+
156+
**Problem:** A new model version is missing `schemas.forwardCompatibility`.
157+
158+
**Fix:** Add `schemas.forwardCompatibility` to the model version.
159+
160+
### `model-version/missing-create-schema` [model-version-missing-create]
161+
162+
**Problem:** A new model version is missing `schemas.create`.
163+
164+
**Fix:** Add `schemas.create` to the model version.
165+
166+
### `model-version/mappings-not-in-schema` [model-version-mappings-not-in-schema]
167+
168+
**Problem:** The type has mapping fields that are not present in the latest model version's `create` schema. All mapped fields must be covered by the schema.
169+
170+
**Fix:** Add the missing fields to the `create` schema of the latest model version.
171+
172+
### `model-version/mapping-index-false` [model-version-mapping-index-false]
173+
174+
**Problem:** A new mapping field uses `index: false`. This option cannot be reverted without a full reindex, making it a risky long-term commitment.
175+
176+
**Fix:** Use `dynamic: false` on the parent object to prevent {{es}} from indexing unknown sub-fields, or omit the mapping entirely for fields that do not need to be searchable.
177+
178+
### `model-version/mapping-enabled-false` [model-version-mapping-enabled-false]
179+
180+
**Problem:** A new mapping field uses `enabled: false`. Like `index: false`, this cannot be undone without a reindex.
181+
182+
**Fix:** Use `dynamic: false` on the parent object instead, or omit the mapping entirely.
183+
184+
### `model-version/fixture-missing` [model-version-fixture-missing]
185+
186+
**Problem:** The type has a new model version but the required rollback test fixtures are missing. Fixtures are needed to validate upgrade and rollback behavior during CI.
187+
188+
**Fix:** Run `node scripts/check_saved_objects --baseline <mergeBase> --fix` to generate the fixture template, then populate it with representative sample documents. See [Ensuring robust serverless rollbacks](validate.md#ensuring-robust-serverless-rollbacks).
189+
190+
### `model-version/fixture-invalid` [model-version-fixture-invalid]
191+
192+
**Problem:** A fixture file exists but its contents are malformed. The file must be a JSON object with one key per version (`<previousVersion>` and `<newVersion>`), each holding a non-empty array of documents.
193+
194+
**Fix:** Correct the fixture file so it matches the expected format, or delete it and re-run with `--fix` to regenerate a valid template.
195+
196+
## Document rules
197+
198+
### `documents/fixture-mismatch` [documents-fixture-mismatch]
199+
200+
**Problem:** During the automated rollback test, a document read from {{es}} did not match any entry in the fixture. Either the migration produced an unexpected document shape, or the fixture is out of date.
201+
202+
**Fix:** Update the fixture to match the actual document structure produced by the migration, or fix the migration transformation to produce the expected output.
203+
204+
## Removed type rules
205+
206+
### `removed-type/registry-needs-update` [removed-type-registry-needs-update]
207+
208+
**Problem:** A Saved Object type is no longer registered but `removed_types.json` has not been updated to record the removal.
209+
210+
**Fix:** Run `node scripts/check_saved_objects --baseline <mergeBase> --fix`, then commit the updated `removed_types.json`. See [Delete](delete.md) for how to get the merge-base.
211+
212+
### `removed-type/name-reused` [removed-type-name-reused]
213+
214+
**Problem:** The type name was previously removed and recorded in `removed_types.json`. Type names in that file are permanently reserved and cannot be reused.
215+
216+
**Fix:** Choose a different name for the new type.
217+
218+
# Kibana fails to start: WIP type not in `allowWipTypes` [saved-objects-troubleshooting-wip-startup]
219+
220+
```shell
221+
Kibana cannot start because the following WIP saved object types are registered but not listed in 'migrations.allowWipTypes': [<soType>].
222+
```
223+
224+
**Problem:** A type listed in `wip_types.json` is registered by a plugin but has not been explicitly allowed in the Kibana configuration.
225+
226+
**Fix:** Add the type name to `migrations.allowWipTypes` in `kibana.yml` for every environment where the plugin is enabled:
227+
228+
```yaml
229+
migrations.allowWipTypes:
230+
- <soType>
231+
```
232+
233+
If you did not intend to register a WIP type, check whether the plugin should be turned off in this environment. See [Scenario A](validate.md#saved-objects-wip-types-scenario-a) for the full workflow.

0 commit comments

Comments
 (0)