Granting kibana_system reserved role access to "all" privileges to .adhoc.alerts* and .internal.adhoc.alerts* indices

[e40pud]

Parent ticket: https://github.com/elastic/security-team/issues/12484

Summary

We'd like to add privileges to a new set of indices to the kibana_system role. The reason for that is we need to have different naming schema for the manually generated attack discovery alerts index aliases and backing indices pointing to these aliases.

Adding for the new "Attack Discovery Scheduling" feature that utilizes alerts as data and a reserved index to write alerts. The attack discovery scheduling feature requires a possibility to generate alerts without running an existing (registered in alerting framework) rule and for that we are writing adhoc generated alerts to a separate index (than normal alerts) so they won't show up with standard .alerts* queries, but still need the same permissions as "normal" alert indices.

[elasticsearchmachine]

Pinging @elastic/es-security (Team:Security)

[elasticsearchmachine]

Hi @e40pud, I've created a changelog YAML for you.

[jeramysoucy]

Question - are these indices created and managed by Kibana, or another stack component (e.g. ES, an integration, etc.)?

Also, we prefer rather than assigning "all", that specific privileges are enumerated when augmenting the kibana_system role. For example, kibana_system likely does not need the ability to perform cross-cluster replication on these indices, and "cross_cluster_replication" is granted by "all". Ideally, the minimum required privileges should be granted.

The full list of index privileges for reference: https://www.elastic.co/docs/reference/elasticsearch/security-privileges#privileges-list-indices
Keep in mind also that some privileges cascade, like the "manage" index privilege which grants many of the other manage_* index privileges.

[e40pud]

Question - are these indices created and managed by Kibana, or another stack component (e.g. ES, an integration, etc.)?

Also, we prefer rather than assigning "all", that specific privileges are enumerated when augmenting the kibana_system role. For example, kibana_system likely does not need the ability to perform cross-cluster replication on these indices, and "cross_cluster_replication" is granted by "all". Ideally, the minimum required privileges should be granted.

The full list of index privileges for reference: https://www.elastic.co/docs/reference/elasticsearch/security-privileges#privileges-list-indices Keep in mind also that some privileges cascade, like the "manage" index privilege which grants many of the other manage_* index privileges.

@jeramysoucy thanks for the review. I updated the list of privileges to be able to create, manage, read and write the index.

[jeramysoucy]

@e40pud Thanks for updating this! The last thing I'd like to ask, is about the manage privilege, which inherently includes all of these privileges:

• manage_data_stream_lifecycle
• manage_follow_index
• manage_ilm
• manage_leader_index
• monitor
• maintenance
• auto_configure

Aiming for the minimum necessary privileges, could we include just a subset of these? If not, could you provide justification to include them? Sorry to have to ask. We like to be as thorough as we can when changing this role.

[e40pud]

I went through the ES apis that we use to work with this indices and here is the list of all of them:

1. Simulate an index (API, Usage in kibana)
2. Simulate index template (API, Usage in kibana)
3. Create or update an index template (API, Usage in kibana)
4. Get data streams (API, Usage in kibana)
5. Create a data stream (API, Usage in kibana)
6. Get aliases (API, Usage in kibana)
7. Get index information (API, Usage in kibana)
8. Create an index (API, Usage in kibana)
9. Create or update an alias (API, Usage in kibana)
10. Get index templates (API, Usage in kibana)
11. Update field mappings (API, Usage in kibana)
12. Create or update a lifecycle policy (API, Usage in kibana)
13. Create or update a component template (API, Usage in kibana)
14. Update index settings (API, Usage in kibana)
15. Search (API, Usage in kibana)
16. Bulk index or delete documents (API, Usage in kibana)

Regarding the manage privilege we need it because when we create the index we also specify the alias for it:

esClient.indices.create({
  index: indexPatterns.name,
  aliases: {
    [indexPatterns.alias]: {
      is_write_index: true,
    },
  },
})

and according to the create_index privilege documentation manage privilege is required.

Privilege to create an index or data stream. A create index request may contain aliases to be added to the index once created. In that case the request requires the manage privilege as well, on both the index and the aliases names.

[jeramysoucy]

@e40pud First, sorry for the delay. Second, sorry to ask for another change. I reviewed this with @slobodanadamovic, and while it is true that we do not have specific alias privileges, we are also not limited to using only end-user named privileges.

In this case, we can reduce the scope being granted by the manage privilege by referencing action names directly instead. Here's an example that Slobodan provided.

Does this seem like a better match here? We'd prefer this from a security perspective, since we can hone in on just the action needed without needing to grant any additional access via manage.

[e40pud]

Thank you for the response @jeramysoucy. I'm just not quite sure which actions I will need to use in this case. There are multiple APIs that will have access to the index and it's alias (16 mentioned above) and I do not really know what set of privileges will cover all of them. Is there some kind of map of API => minimum required privilege which I can use to create a correct minimum set of privileges?

I'm also not sure why the documentation states the required manage privilege and still we do not really need it? Is this specific to our use case only or the documentation does not accurately describe it?

[jeramysoucy]

I'm also not sure why the documentation states the required manage privilege and still we do not really need it? Is this specific to our use case only or the documentation does not accurately describe it?

My understanding is that the documentation is intended for end-users. As user, we would only be able to include named privileges in a role. But as a developer, we do not have this limitation.

IIUC, the example linked above will impart the privilege for the necessary alias actions. Regarding the others you listed, I'll need some help from @elastic/es-security (cc @slobodanadamovic)

Here is what I could identify so far...

• Simulate an index: ?
• Simulate index template: manage_index_templates cluster privilege?
• Create or update an index template: manage_index_templates cluster privilege?
• Get data streams: view_index_metadata
• Create a data stream: create_index
• Get aliases: view_index_metadata
• Get index information: view_index_metadata
• Create an index: create_index
• Create or update an alias: TransportIndicesAliasesAction.NAME (not sure if this also grants update)
• Get index templates: manage_index_templates cluster privilege?
• Update field mappings: write? Not sure on this one. Many of the privileges that are marked as granting this privilege also note that it is deprecated as of v8.0.
• Create or update a lifecycle policy: manage_ilm
• Create or update a component template: ?
• Update index settings: ?
• Search: read
• Bulk index or delete documents: write

[slobodanadamovic]

Basically, the privileges grant certain actions to be executed. We have named privileges (e.g. manage) that have a set of patterns or actions that they grant. In this particular case it’s the rollover action that was denied.
You can grant it to kibana_system role by adding the actual action name indices:admin/rollover to index privileges, or by referencing the action constant RolloverAction.NAME.

[e40pud]

I now updated Serverless roles to reflect this changes and I see the next error:

[2025-05-27T17:04:06.579+02:00][ERROR][plugins.ruleRegistry] Error creating data stream .adhoc.alerts-security.attack.discovery.alerts-default - security_exception
Root causes:
security_exception: action [indices:admin/data_stream/create] is unauthorized for service account [elastic/kibana] on indices [.adhoc.alerts-security.attack.discovery.alerts-default], this action is granted by the index privileges [create_index,manage,all]

Should I add CreateDataStreamAction.NAME (indices:admin/data_stream/create) to index privilege?

[slobodanadamovic]

Strange. This should be allowed. You have create_index over .adhoc.alerts* names, which includes .adhoc.alerts-security.attack.discovery.alerts-default and create_index grants indices:admin/data_stream/create. How did you test this? Is it possible that Elasticsearch changes from this PR were not included in your test?

[e40pud]

Yeah, that was me testing against not update ES. I checked it again with my changes and this error is gone.

Though, now I see a different one:

[2025-05-27T19:39:54.214+02:00][ERROR][plugins.ruleRegistry] ResponseError: {"errors":true,"took":0,"items":[{"create":{"_index":".adhoc.alerts-security.attack.discovery.alerts-default","_id":"0b666812-ac3a-4aeb-ad08-466789e49891","status":403,"failure_store":"not_enabled","error":{"type":"security_exception","reason":"action [indices:admin/refresh/unpromotable] is unauthorized for service account [elastic/kibana] on indices [.ds-.adhoc.alerts-security.attack.discovery.alerts-default-2025.05.27-000001], this action is granted by the index privileges [maintenance,manage,all]"}}}]}
at Object.bulk (rule_data_client.ts:247:27)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at createAttackDiscoveryAlerts (index.ts:84:20)
at generateAndUpdateAttackDiscoveries (generate_and_update_discoveries.ts:106:7)

We use refresh in the bulk API and I guess this is causing this error.

When I added "manage" privilege to the list the error is gone. @slobodanadamovic what would be an appropriate privilege to add to that list to cover the refresh=true?

[slobodanadamovic]

For allowing refresh, I suggest granting maintenance privilege.

[slobodanadamovic]

LGTM (from es-security side) 👍

[jeramysoucy]

@e40pud Thanks very much for your patience on this one!
And thank you, @slobodanadamovic for all of your help and guidance here!

[elasticsearchmachine]

💚 Backport successful

Status	Branch	Result
✅	8.19