Adding rest actions to get and set data stream settings

[masseyke]

This builds on #127417, #126947, and #127282, adding rest actions to get and set data stream settings. The rest API changes are only visible if the logs_stream feature flag is enabled (i.e. you are running a snapshot, or the es.logs_stream_enabled system property is set to true).
Example usage:
Before setting any data stream settings,

GET _data_stream/my-data-stream/_settings

returns:

{
  "data_streams": [
    {
      "name": "my-data-stream",
      "settings": {},
      "effective_settings": {
        "index": {
          "lifecycle": {
            "name": "test-policy"
          },
          "mode": "standard",
          "number_of_replicas": "0"
        }
      }
    }
  ]
}

The settings field is empty because there are no settings directly on the my-data-stream data stream. The effective_settings field shows the result of merging this data stream's settings in with the settings it inherited from its template(s).
Now we can set settings on the data stream:

PUT _data_stream/my-data-stream/_settings
{
    "index.lifecycle.name" : "new-test-policy",
    "index.number_of_shards": 10
}

and the response is:

{
  "data_streams": [
    {
      "name": "my-data-stream",
      "applied_to_data_stream": true,
      "settings": {
        "index": {
          "lifecycle": {
            "name": "new-test-policy"
          },
          "number_of_shards": "10"
        }
      },
      "effective_settings": {
        "index": {
          "lifecycle": {
            "name": "new-test-policy"
          },
          "mode": "standard",
          "number_of_shards": "10",
          "number_of_replicas": "0"
        }
      },
      "index_settings_results": {
        "applied_to_data_stream_only": [
          "index.number_of_shards"
        ],
        "applied_to_data_stream_and_backing_indices": [
          "index.lifecycle.name"
        ]
      }
    }
  ]
}

The response lets the user know which settings were applied to all backing indices, and which were applied only to the data stream. Now doing a get settings API call shows these new data stream settings:

GET _data_stream/my-data-stream/_settings

returns

{
  "data_streams": [
    {
      "name": "my-data-stream",
      "settings": {
        "index": {
          "lifecycle": {
            "name": "new-test-policy"
          },
          "number_of_shards": "10"
        }
      },
      "effective_settings": {
        "index": {
          "lifecycle": {
            "name": "new-test-policy"
          },
          "mode": "standard",
          "number_of_shards": "10",
          "number_of_replicas": "0"
        }
      }
    }
  ]
}

The get data stream API now also returns the settings that have been applied to the current data stream. Using the example above:

GET _data_stream/my-data-stream

returns:

{
  "data_streams": [
    {
      "name": "my-data-stream",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-2025.05.07-000001",
          "index_uuid": "_2sdNBZZTGqkwj247VlGdg",
          "managed_by": "Index Lifecycle Management",
          "prefer_ilm": true,
          "ilm_policy": "new-test-policy",
          "index_mode": "standard"
        }
      ],
      "generation": 1,
      "status": "GREEN",
      "template": "my-template",
      "ilm_policy": "new-test-policy",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false,
      "index_mode": "standard",
      "settings": {
        "index": {
          "lifecycle": {
            "name": "new-test-policy"
          },
          "number_of_shards": "10"
        }
      },
      "failure_store": {
        "enabled": false,
        "rollover_on_write": true,
        "indices": []
      }
    }
  ]
}

The settings field contains only those settings that have been applied to this data stream, and does not incorporate settings from templates.

Note: A dry-run mode will come in a follow-up PR.

[masseyke]

Note: the serverless check is failing b/c the yaml rest test uses ILM settings that are not available in serverless. Before merging this PR, I'll exclude those tests from serverless in https://github.com/elastic/elasticsearch-serverless/pull/3890.

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[Copilot]

Pull Request Overview

Adds support for REST endpoints to get and set data stream settings behind a feature flag.

• Introduces RestGetDataStreamSettingsAction and RestUpdateDataStreamSettingsAction handlers registered when the logs_stream flag is enabled
• Updates DataStream.toXContent to include a settings field when the feature is on
• Adds YAML tests for the new GET/PUT settings APIs and extends existing Java tests to round-trip the settings

Reviewed Changes

Copilot reviewed 8 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
server/src/main/java/org/elasticsearch/cluster/metadata/DataStream.java	Adds a LOGS_STREAM_FEATURE_FLAG and gates settings output in XContent
server/src/main/java/org/elasticsearch/action/datastreams/GetDataStreamAction.java	Adds settings ParseField and emits data stream settings when flagged
server/src/test/java/org/elasticsearch/action/datastreams/GetDataStreamActionTests.java	Imports Settings and tests that settings round-trip through XContent
modules/data-streams/src/main/java/org/elasticsearch/datastreams/rest/RestGetDataStreamSettingsAction.java	New GET settings handler
modules/data-streams/src/main/java/org/elasticsearch/datastreams/rest/RestUpdateDataStreamSettingsAction.java	New PUT settings handler
modules/data-streams/src/main/java/org/elasticsearch/datastreams/DataStreamsPlugin.java	Registers the new handlers behind the feature flag
modules/data-streams/src/main/java/org/elasticsearch/datastreams/DataStreamFeatures.java	Exposes the logs_stream feature for tests
modules/data-streams/src/yamlRestTest/resources/rest-api-spec/test/data_stream/240_data_stream_settings.yml	Integration tests for the new settings endpoints

Files not reviewed (2)

• rest-api-spec/src/main/resources/rest-api-spec/api/indices.get_data_stream_settings.json: Language not supported
• rest-api-spec/src/main/resources/rest-api-spec/api/indices.put_data_stream_settings.json: Language not supported

Comments suppressed due to low confidence (1)

modules/data-streams/src/main/java/org/elasticsearch/datastreams/rest/RestGetDataStreamSettingsAction.java:32

• Typo in action name: should be "get_data_stream_settings_action" for consistency with other handlers.

return "gett_data_stream_settings_action";

[Copilot]

[nitpick] Directly instantiating a raw FeatureFlag here duplicates the NodeFeature in DataStreamFeatures. Consider centralizing feature checks via the NodeFeature or a shared utility.

Suggested change

	public static final boolean LOGS_STREAM_FEATURE_FLAG = new FeatureFlag("logs_stream").isEnabled();
	public static final NodeFeature LOGS_STREAM_FEATURE = new NodeFeature("logs_stream");

[Copilot]

[nitpick] Using the static flag on DataStream couples plugin registration to the metadata class. Consider referencing the NodeFeature from DataStreamFeatures for a clearer separation of concerns.

[lukewhiting]

One small typo to fix but otherwise, looks good :-)

[lukewhiting]

Typo?

Suggested change

	return "gett_data_stream_settings_action";
	return "get_data_stream_settings_action";