[BUG] .opendistro-ism-config index cannot consume new user field custom_attributes #1471
Description
What is the bug?
As part of the work to solve opensearch-project/alerting#1829, the security plugin will serialize all user attributes (keys + values) whereas today common-utils only references the names (keys) of the attribute map with the field custom_attribute_names. When ISM creates its system indices it creates them with mappings checked into this repo and assumes that the user will have custom_attribute_names.
In common-utils we would like to make a change to serialize a field called custom_attributes (opposed to existing custom_attribute_names that would contain the full map of key, value pair attributes that can be injected back at monitor runtime to support use-cases where monitors are created by users mapped to roles that utilize DLS + user attribute substitution...a common pattern used in multi-tenant cases.
How can one reproduce the bug?
When I run the ISM tests with the respective changes in security + common-utils and set plugins.security.user_attribute_serialization.enabled to true, I get an error run ISM integ tests
✗ ./gradlew integTest -Dsecurity=true -Dhttps=true --tests IndexStateManagementHistoryIT -i
IndexStateManagementHistoryIT > test history shard settings FAILED
org.opensearch.client.ResponseException: method [PUT], host [https://127.0.0.1:53149], URI [/_plugins/_ism/policies/indexstatemanagementhistoryit_shard_settings_1?refresh=true], status line [HTTP/2.0 400 Bad Request]
{"error":{"root_cause":[{"type":"strict_dynamic_mapping_exception","reason":"mapping set to strict, dynamic introduction of [custom_attributes] within [policy.user] is not allowed"}],"type":"strict_dynamic_mapping_exception","reason":"mapping set to strict, dynamic introduction of [custom_attributes] within [policy.user] is not allowed"},"status":400}
at __randomizedtesting.SeedInfo.seed([51FAE56906895165:A23271502FAB3A14]:0)
at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:501)
at app//org.opensearch.client.RestClient.performRequest(RestClient.java:384)
at app//org.opensearch.client.RestClient.performRequest(RestClient.java:359)
at app//org.opensearch.indexmanagement.TestHelpersKt.makeRequest(TestHelpers.kt:84)
at app//org.opensearch.indexmanagement.TestHelpersKt.makeRequest$default(TestHelpers.kt:67)
at app//org.opensearch.indexmanagement.indexstatemanagement.IndexStateManagementRestTestCase.createPolicyJson(IndexStateManagementRestTestCase.kt:137)
at app//org.opensearch.indexmanagement.indexstatemanagement.IndexStateManagementRestTestCase.createPolicy(IndexStateManagementRestTestCase.kt:110)
at app//org.opensearch.indexmanagement.indexstatemanagement.IndexStateManagementRestTestCase.createPolicy$default(IndexStateManagementRestTestCase.kt:104)
at app//org.opensearch.indexmanagement.indexstatemanagement.action.IndexStateManagementHistoryIT.test history shard settings(IndexStateManagementHistoryIT.kt:186)
What is the expected behavior?
ISM should be able to consume the new field