Handling removed typed automatically (elastic#241880)

Resolves [issue](elastic/kibana-team#1936)

## Summary

This PR updates our CI checks for Saved Object (SO) types to
automatically handle removed types. The removed type is detected and
then handled by adding to our list of `removed_types` and also prevents
future re-registration of the same type name. This helps in reducing the
need for manual reviews in PRs that remove types.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

Author: hammad-nasir-elastic

dev_docs/tutorials/saved_objects.mdx

@@ -355,6 +355,78 @@ export const foo: SavedObjectsType = {
 
 [2] Set this to `true` to build your own HTTP API and have complete control over the route handler.
 
+## Removing a Saved Object type
+
+When you need to remove a Saved Object type from your plugin, there are important steps to follow to ensure the type name cannot be accidentally reused in the future.
+
+### Why do we track removed types?
+
+Once a Saved Object type has been registered and used in production, its name becomes "reserved" for that purpose. If we allowed the same name to be reused for a different type later:
+
+- **Migration conflicts**: Old documents with the removed type name could interfere with new documents using the same name
+- **Upgrade failures**: Users upgrading from older versions could experience migration failures if type names are reused
+
+To prevent these issues, Kibana maintains a list of all removed type names in `packages/kbn-check-saved-objects-cli/removed_types.json`. This ensures type names are never reused.
+
+### How to remove a Saved Object type
+
+#### Step 1: Remove the type registration
+
+First, remove the type registration from your plugin. Also remove the type definition file and any references to it.
+
+#### Step 2: Run the automated check
+
+Kibana includes an automated check that detects when types are removed. Run it locally to update the `removed_types.json` file:
+
+```bash
+# Get your current commit ID
+git log -n 1
+
+# Get the merge-base commit with main
+git merge-base -a <currentCommitSha> main
+
+# Run the check with the baseline
+node scripts/check_saved_objects --baseline <mergeBase> --fix
+```
+
+Replace `<currentCommitSha>` with your actual commit SHA from the first command, and `<mergeBase>` with the merge-base commit SHA from the second command.
+
+This command will:
+1. Compare the current registered types against the baseline (merge-base with main)
+2. Detect any types that were removed
+3. Automatically add the removed type name(s) to `removed_types.json`
+4. Exit with an error message indicating which types were added
+
+The error message will look like:
+
+```
+❌ The following SO types are no longer registered: 'my-removed-type'. 
+Updated 'removed_types.json' to prevent the same names from being reused in the future.
+```
+
+#### Step 3: Commit the changes
+
+Include both your code changes and the updated `removed_types.json` file in your commit.
+
+### Manual update (alternative)
+
+If you prefer to update `removed_types.json` manually, you can:
+
+1. Open `packages/kbn-check-saved-objects-cli/removed_types.json`
+2. Add your removed type name to the array in alphabetical order
+3. Save the file
+
+### Important considerations
+
+<DocCallOut color="warning" title="Type names cannot be reused">
+  Once a type name is added to `removed_types.json`, it cannot be used again for a new Saved Object type. Choose type names carefully when creating new types.
+</DocCallOut>
+
+Before removing a type, ensure:
+- **No references exist**: Check that no other Saved Object types reference the type you're removing
+- **Data migration is complete**: If users have documents of this type, ensure they've been migrated to a new type or are no longer needed
+- **Coordinate with stakeholders**: Confirm with your team and any dependent teams that the removal is expected
+
 ## Troubleshooting
 
 ### CI is failing for my PR
@@ -406,3 +478,15 @@ These errors are pretty obvious. Model versions must be defined as consecutive n
 * The current serverless release. If a high-severity issue occurs, Serverless might be rolled back to a previous version. This could in turn impact CI pipelines for PRs that would otherwise be fine. In other words, _"from the current serverless standpoint, your PR is not releasable at the moment"_ . This should only happen exceptionally. Please take a look at the `#kibana-mission-control` channel to see if something is going on.
   * **Scenario 2.1**: Kibana has been rolled back. Please wait until the situation goes back to normal, i.e. an emergency release is performed after the rollback, and Kibana gets to a normal release state.
   * **Scenario 2.2**: Kibana has NOT been rolled back. Reach out to `#kibana-core` and we will help you figure out what's going on.
+
+```
+❌ The following SO types are no longer registered: '<soType>'. Please run with --fix to update 'removed_types.json'.
+```
+**Problem:** You removed a Saved Object type but didn't update the `removed_types.json` file to track the removal.
+**Solution:** Run `node scripts/check_saved_objects --baseline <mergeBase> --fix` locally to automatically update the `removed_types.json` file, then commit the changes. See the <DocLink id="kibDevTutorialSavedObject" section="removing-a-saved-object-type" text="Removing a Saved Object type"/> section for detailed instructions on how to determine the correct merge-base commit.
+
+```
+❌ Cannot re-register previously removed type(s): <soType>. Please use a different name.
+```
+**Problem:** You're trying to create a new Saved Object type using a name that was previously used and removed. Type names cannot be reused to prevent migration conflicts.
+**Solution:** Choose a different name for your Saved Object type. Once a type name is added to `packages/kbn-check-saved-objects-cli/removed_types.json`, it's permanently reserved and cannot be reused.

packages/kbn-check-saved-objects-cli/index.ts

@@ -9,3 +9,4 @@
 
 export { runCheckSavedObjectsCli } from './src/commands/run_check_saved_objects_cli';
 export { runCheckMappingsUpdateCli } from './src/commands/run_check_mappings_update_cli';
+export { getRemovedTypes } from './src/migrations/removed_types/get_removed_types';

packages/kbn-check-saved-objects-cli/removed_types.json

@@ -0,0 +1,27 @@
+[
+  "apm-services-telemetry",
+  "application_usage_transactional",
+  "background-session",
+  "cases-sub-case",
+  "csp_rule",
+  "endpoint:user-artifact",
+  "file-upload-telemetry",
+  "fleet-agent-actions",
+  "fleet-agent-events",
+  "fleet-agents",
+  "fleet-enrollment-api-keys",
+  "guided-onboarding-guide-state",
+  "guided-onboarding-plugin-state",
+  "guided-setup-state",
+  "investigation",
+  "maps-telemetry",
+  "ml-telemetry",
+  "osquery-usage-metric",
+  "server",
+  "siem-detection-engine-rule-execution-info",
+  "siem-detection-engine-rule-status",
+  "timelion-sheet",
+  "tsvb-validation-telemetry",
+  "ui-counter",
+  "upgrade-assistant-telemetry"
+]

packages/kbn-check-saved-objects-cli/src/commands/run_check_saved_objects_cli.ts

@@ -13,6 +13,7 @@ import { startServers, stopServers } from '../util/servers';
 import type { TaskContext } from './types';
 import {
   automatedRollbackTests,
+  checkRemovedTypes,
   getSnapshots,
   validateNewTypes,
   validateUpdatedTypes,
@@ -37,6 +38,8 @@ export function runCheckSavedObjectsCli() {
         gitRev,
         newTypes: [],
         updatedTypes: [],
+        currentRemovedTypes: [],
+        newRemovedTypes: [],
         fixtures: {},
         fix,
       };
@@ -51,6 +54,10 @@ export function runCheckSavedObjectsCli() {
             title: 'Get type registry snapshots',
             task: getSnapshots,
           },
+          {
+            title: 'Check removed SO types',
+            task: checkRemovedTypes,
+          },
           {
             title: 'Validate new SO types',
             task: validateNewTypes,

packages/kbn-check-saved-objects-cli/src/commands/tasks/check_removed_types.ts

@@ -0,0 +1,44 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the "Elastic License
+ * 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
+ * Public License v 1"; you may not use this file except in compliance with, at
+ * your election, the "Elastic License 2.0", the "GNU Affero General Public
+ * License v3.0 only", or the "Server Side Public License, v 1".
+ */
+
+import type { ListrTask } from 'listr2';
+import type { Task, TaskContext } from '../types';
+import {
+  detectConflictsWithRemovedTypes,
+  detectNewRemovedTypes,
+  getRemovedTypes,
+  updateRemovedTypes,
+} from '../../migrations/removed_types';
+
+export const checkRemovedTypes: Task = async (ctx, task) => {
+  const subtasks: ListrTask<TaskContext>[] = [
+    {
+      title: 'Detecting conflicts with removed types',
+      task: async () => {
+        ctx.currentRemovedTypes = await getRemovedTypes();
+        await detectConflictsWithRemovedTypes(ctx.to!, ctx.currentRemovedTypes);
+      },
+    },
+    {
+      title: `Detecting new removed types`,
+      task: () => {
+        ctx.newRemovedTypes = detectNewRemovedTypes(ctx.from!, ctx.to!, ctx.currentRemovedTypes);
+      },
+    },
+    {
+      title: `Updating removed types`,
+      task: async () => {
+        await updateRemovedTypes(ctx.newRemovedTypes, ctx.currentRemovedTypes, ctx.fix);
+      },
+      skip: () => ctx.newRemovedTypes.length === 0,
+    },
+  ];
+
+  return task.newListr<TaskContext>(subtasks);
+};

packages/kbn-check-saved-objects-cli/src/commands/tasks/index.ts

@@ -11,3 +11,4 @@ export { getSnapshots } from './get_snapshots';
 export { validateNewTypes } from './validate_new_types';
 export { validateUpdatedTypes } from './validate_updated_types';
 export { automatedRollbackTests } from './automated_rollback_tests';
+export { checkRemovedTypes } from './check_removed_types';

packages/kbn-check-saved-objects-cli/src/commands/types.ts

@@ -20,6 +20,8 @@ export interface TaskContext {
   to?: MigrationSnapshot;
   newTypes: string[];
   updatedTypes: string[];
+  currentRemovedTypes: string[];
+  newRemovedTypes: string[];
   fixtures: Record<
     string,
     {

packages/kbn-check-saved-objects-cli/src/compatibility/extract_mappings_from_plugins_worker.ts

@@ -12,7 +12,7 @@ import { set } from '@kbn/safer-lodash-set';
 import { buildTypesMappings } from '@kbn/core-saved-objects-migration-server-internal';
 import type { SavedObjectsTypeMappingDefinitions } from '@kbn/core-saved-objects-base-server-internal';
 import { PLUGIN_SYSTEM_ENABLE_ALL_PLUGINS_CONFIG_PATH } from '@kbn/core-plugins-server-internal/src/constants';
-import { REMOVED_TYPES } from '@kbn/core-saved-objects-server-internal';
+import { getRemovedTypes } from '../..';
 
 export interface Result {
   mappings: SavedObjectsTypeMappingDefinitions;
@@ -59,7 +59,8 @@ const cleanupRemovedTypes = (
   await root.preboot();
   const { savedObjects } = await root.setup();
   const mappings = buildTypesMappings(savedObjects.getTypeRegistry().getAllTypes());
-  cleanupRemovedTypes(mappings, REMOVED_TYPES);
+  const removedTypes = await getRemovedTypes();
+  cleanupRemovedTypes(mappings, removedTypes);
   const result: Result = { mappings };
   process.send(result);
 })().catch((error) => {

packages/kbn-check-saved-objects-cli/src/migrations/removed_types/constants.ts

@@ -0,0 +1,12 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the "Elastic License
+ * 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
+ * Public License v 1"; you may not use this file except in compliance with, at
+ * your election, the "Elastic License 2.0", the "GNU Affero General Public
+ * License v3.0 only", or the "Server Side Public License, v 1".
+ */
+
+import { resolve } from 'path';
+
+export const REMOVED_TYPES_JSON_PATH = resolve(__dirname, '../../../removed_types.json');

packages/kbn-check-saved-objects-cli/src/migrations/removed_types/detect_conflicts_removed_types.ts

@@ -0,0 +1,36 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the "Elastic License
+ * 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
+ * Public License v 1"; you may not use this file except in compliance with, at
+ * your election, the "Elastic License 2.0", the "GNU Affero General Public
+ * License v3.0 only", or the "Server Side Public License, v 1".
+ */
+
+import type { MigrationSnapshot } from '../../types';
+
+/**
+ * Detects new types in 'to' that conflict with previously removed types
+ */
+export async function detectConflictsWithRemovedTypes(
+  to: MigrationSnapshot,
+  currentRemovedTypes: string[]
+) {
+  const toTypes = Object.keys(to.typeDefinitions);
+
+  const conflictingTypes: string[] = [];
+
+  for (const type of toTypes) {
+    if (currentRemovedTypes.includes(type)) {
+      conflictingTypes.push(type);
+    }
+  }
+
+  if (conflictingTypes.length > 0) {
+    throw new Error(
+      `❌ Cannot re-register previously removed type(s): ${conflictingTypes.join(
+        ', '
+      )}. Please use a different name.`
+    );
+  }
+}