[Security Solution] Add FTR tests for prebuilt rules OOM testing#236891
Conversation
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
jbudz
left a comment
There was a problem hiding this comment.
.buildkite/ftr_security_stateful_configs.yml LGTM
2c5a23a to
f75a6d1
Compare
nikitaindik
left a comment
There was a problem hiding this comment.
Reviewed the changes and tested by doing these:
- set up
qaflocally - deployed a 9.3.0 stack to prod cloud
- pointed Kibana to a custom EPR deployed GCP
- ran the tests against the deployment, trying to bootstrap with "good" and "bad" packages.
In the green scenario, the endpoints return 200. In a scenario with a huge package, the bootstrap endpoint returns 502 "Bad Gateway".
@maximpn With huge packages, do we see OOMs in the bootstrap endpoint already? I expected the tests to fail at "installation review" stage. I was using your package v99.32.0.
|
|
||
| ### ⚙️ Test Environment Setup | ||
|
|
||
| To effectively reproduce OOM-related behavior, the deployment should be created in Elastic Cloud with 1GB RAM limit for the Kibana instance and 2-4GB RAM limit for Elasticsearch instance. ML and Integration instances as well as cold and frozen tier Elasticsearch nodes aren't required. An example Elastic Cloud configuration applicable for internal testing framework QAF (QA Framework) looks like the following |
There was a problem hiding this comment.
Does RAM limit for ES matter for Kibana OOM tests? In the example plan below only 1GB is specified.
There was a problem hiding this comment.
I haven't observed any significant memory pressure in the ES instance while performing experiments. The same results are observed when ES instance has 4GB, 2GB and 1GB of memory. So it should be safe to have only 1GB of RAM for the ES instance.
There was a problem hiding this comment.
Thanks for checking this. Then we could also update the wording here to have 1GB for ES instead of 2-4GB.
There was a problem hiding this comment.
Yes. I've done it already.
|
|
||
| #### CI | ||
|
|
||
| The tests can be run via a Buildkite pipeline... |
There was a problem hiding this comment.
I think we can remove the unfinished sections for now and add them later once we have more info.
f75a6d1 to
fd41b78
Compare
|
@nikitaindik Thanks for your review! I've updated the README.md file to address your comments. Please have a look. |
nikitaindik
left a comment
There was a problem hiding this comment.
@maximpn Replied to a comment about ES RAM configuration. Please, take a look.
Thanks for the updates. LGTM now.
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
2 similar comments
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
…stic#236891) **Partially addresses:** elastic#188090 ## Summary This PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages. ## Running test instructions - Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal) - Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml` ```yaml --- name: {{ deployment_name }} settings: autoscaling_enabled: {{ autoscaling_enabled }} metadata: system_owned: false resources: elasticsearch: - region: {{ region }} settings: dedicated_masters_threshold: 6 plan: cluster_topology: - zone_count: 1 elasticsearch: node_attributes: data: hot instance_configuration_id: gcp.es.datahot.n2.68x10x45 node_roles: - master - ingest - remote_cluster_client - data_hot - transform - data_content id: hot_content size: value: 1024 resource: memory elasticsearch: version: {{ stack_version }} deployment_template: id: gcp-storage-optimized ref_id: main-elasticsearch enterprise_search: [] kibana: - elasticsearch_cluster_ref_id: main-elasticsearch region: {{ region }} plan: cluster_topology: - instance_configuration_id: gcp.kibana.n2.68x32x45 zone_count: 1 size: value: 1024 resource: memory kibana: version: {{ stack_version }} ref_id: main-kibana ``` - Create an ECH deployment by running the following command ```bash qaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing ``` - Run the tests by running the following command ```bash qaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts ``` where `<kibana-root>` is the absolute path to the Kibana's root folder.
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
2 similar comments
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
…stic#236891) **Partially addresses:** elastic#188090 ## Summary This PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages. ## Running test instructions - Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal) - Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml` ```yaml --- name: {{ deployment_name }} settings: autoscaling_enabled: {{ autoscaling_enabled }} metadata: system_owned: false resources: elasticsearch: - region: {{ region }} settings: dedicated_masters_threshold: 6 plan: cluster_topology: - zone_count: 1 elasticsearch: node_attributes: data: hot instance_configuration_id: gcp.es.datahot.n2.68x10x45 node_roles: - master - ingest - remote_cluster_client - data_hot - transform - data_content id: hot_content size: value: 1024 resource: memory elasticsearch: version: {{ stack_version }} deployment_template: id: gcp-storage-optimized ref_id: main-elasticsearch enterprise_search: [] kibana: - elasticsearch_cluster_ref_id: main-elasticsearch region: {{ region }} plan: cluster_topology: - instance_configuration_id: gcp.kibana.n2.68x32x45 zone_count: 1 size: value: 1024 resource: memory kibana: version: {{ stack_version }} ref_id: main-kibana ``` - Create an ECH deployment by running the following command ```bash qaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing ``` - Run the tests by running the following command ```bash qaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts ``` where `<kibana-root>` is the absolute path to the Kibana's root folder. (cherry picked from commit d99c5b4)
…stic#236891) **Partially addresses:** elastic#188090 ## Summary This PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages. ## Running test instructions - Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal) - Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml` ```yaml --- name: {{ deployment_name }} settings: autoscaling_enabled: {{ autoscaling_enabled }} metadata: system_owned: false resources: elasticsearch: - region: {{ region }} settings: dedicated_masters_threshold: 6 plan: cluster_topology: - zone_count: 1 elasticsearch: node_attributes: data: hot instance_configuration_id: gcp.es.datahot.n2.68x10x45 node_roles: - master - ingest - remote_cluster_client - data_hot - transform - data_content id: hot_content size: value: 1024 resource: memory elasticsearch: version: {{ stack_version }} deployment_template: id: gcp-storage-optimized ref_id: main-elasticsearch enterprise_search: [] kibana: - elasticsearch_cluster_ref_id: main-elasticsearch region: {{ region }} plan: cluster_topology: - instance_configuration_id: gcp.kibana.n2.68x32x45 zone_count: 1 size: value: 1024 resource: memory kibana: version: {{ stack_version }} ref_id: main-kibana ``` - Create an ECH deployment by running the following command ```bash qaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing ``` - Run the tests by running the following command ```bash qaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts ``` where `<kibana-root>` is the absolute path to the Kibana's root folder. (cherry picked from commit d99c5b4) # Conflicts: # .buildkite/ftr_security_stateful_configs.yml
…ng (#236891) (#240013) # Backport This will backport the following commits from `main` to `8.19`: - [[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)](#236891) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-10-10T12:27:05Z","message":"[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/188090\n\n## Summary\n\nThis PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.\n\n## Running test instructions\n\n- Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal)\n- Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml`\n```yaml\n---\nname: {{ deployment_name }}\nsettings:\n autoscaling_enabled: {{ autoscaling_enabled }}\nmetadata:\n system_owned: false\nresources:\n elasticsearch:\n - region: {{ region }}\n settings:\n dedicated_masters_threshold: 6\n plan:\n cluster_topology:\n - zone_count: 1\n elasticsearch:\n node_attributes:\n data: hot\n instance_configuration_id: gcp.es.datahot.n2.68x10x45\n node_roles:\n - master\n - ingest\n - remote_cluster_client\n - data_hot\n - transform\n - data_content\n id: hot_content\n size:\n value: 1024\n resource: memory\n elasticsearch:\n version: {{ stack_version }}\n deployment_template:\n id: gcp-storage-optimized\n ref_id: main-elasticsearch\n enterprise_search: []\n kibana:\n - elasticsearch_cluster_ref_id: main-elasticsearch\n region: {{ region }}\n plan:\n cluster_topology:\n - instance_configuration_id: gcp.kibana.n2.68x32x45\n zone_count: 1\n size:\n value: 1024\n resource: memory\n kibana:\n version: {{ stack_version }}\n ref_id: main-kibana\n```\n\n- Create an ECH deployment by running the following command\n\n```bash\nqaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing\n```\n\n- Run the tests by running the following command\n\n```bash\nqaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts\n```\n\nwhere `<kibana-root>` is the absolute path to the Kibana's root folder.","sha":"d99c5b4b7211c1510f18f2b6260e24ea327b7d5a","branchLabelMapping":{"^v9.3.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport missing","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v9.2.0","v9.3.0","v8.19.6","v9.0.9"],"title":"[Security Solution] Add FTR tests for prebuilt rules OOM testing","number":236891,"url":"https://github.com/elastic/kibana/pull/236891","mergeCommit":{"message":"[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/188090\n\n## Summary\n\nThis PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.\n\n## Running test instructions\n\n- Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal)\n- Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml`\n```yaml\n---\nname: {{ deployment_name }}\nsettings:\n autoscaling_enabled: {{ autoscaling_enabled }}\nmetadata:\n system_owned: false\nresources:\n elasticsearch:\n - region: {{ region }}\n settings:\n dedicated_masters_threshold: 6\n plan:\n cluster_topology:\n - zone_count: 1\n elasticsearch:\n node_attributes:\n data: hot\n instance_configuration_id: gcp.es.datahot.n2.68x10x45\n node_roles:\n - master\n - ingest\n - remote_cluster_client\n - data_hot\n - transform\n - data_content\n id: hot_content\n size:\n value: 1024\n resource: memory\n elasticsearch:\n version: {{ stack_version }}\n deployment_template:\n id: gcp-storage-optimized\n ref_id: main-elasticsearch\n enterprise_search: []\n kibana:\n - elasticsearch_cluster_ref_id: main-elasticsearch\n region: {{ region }}\n plan:\n cluster_topology:\n - instance_configuration_id: gcp.kibana.n2.68x32x45\n zone_count: 1\n size:\n value: 1024\n resource: memory\n kibana:\n version: {{ stack_version }}\n ref_id: main-kibana\n```\n\n- Create an ECH deployment by running the following command\n\n```bash\nqaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing\n```\n\n- Run the tests by running the following command\n\n```bash\nqaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts\n```\n\nwhere `<kibana-root>` is the absolute path to the Kibana's root folder.","sha":"d99c5b4b7211c1510f18f2b6260e24ea327b7d5a"}},"sourceBranch":"main","suggestedTargetBranches":["9.0"],"targetPullRequestStates":[{"branch":"9.2","label":"v9.2.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/238438","number":238438,"state":"OPEN"},{"branch":"main","label":"v9.3.0","branchLabelMappingKey":"^v9.3.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/236891","number":236891,"mergeCommit":{"message":"[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/188090\n\n## Summary\n\nThis PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.\n\n## Running test instructions\n\n- Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal)\n- Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml`\n```yaml\n---\nname: {{ deployment_name }}\nsettings:\n autoscaling_enabled: {{ autoscaling_enabled }}\nmetadata:\n system_owned: false\nresources:\n elasticsearch:\n - region: {{ region }}\n settings:\n dedicated_masters_threshold: 6\n plan:\n cluster_topology:\n - zone_count: 1\n elasticsearch:\n node_attributes:\n data: hot\n instance_configuration_id: gcp.es.datahot.n2.68x10x45\n node_roles:\n - master\n - ingest\n - remote_cluster_client\n - data_hot\n - transform\n - data_content\n id: hot_content\n size:\n value: 1024\n resource: memory\n elasticsearch:\n version: {{ stack_version }}\n deployment_template:\n id: gcp-storage-optimized\n ref_id: main-elasticsearch\n enterprise_search: []\n kibana:\n - elasticsearch_cluster_ref_id: main-elasticsearch\n region: {{ region }}\n plan:\n cluster_topology:\n - instance_configuration_id: gcp.kibana.n2.68x32x45\n zone_count: 1\n size:\n value: 1024\n resource: memory\n kibana:\n version: {{ stack_version }}\n ref_id: main-kibana\n```\n\n- Create an ECH deployment by running the following command\n\n```bash\nqaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing\n```\n\n- Run the tests by running the following command\n\n```bash\nqaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts\n```\n\nwhere `<kibana-root>` is the absolute path to the Kibana's root folder.","sha":"d99c5b4b7211c1510f18f2b6260e24ea327b7d5a"}},{"branch":"8.19","label":"v8.19.6","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/238437","number":238437,"state":"OPEN"},{"branch":"9.0","label":"v9.0.9","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"url":"https://github.com/elastic/kibana/pull/238467","number":238467,"branch":"9.1","state":"OPEN"}]}] BACKPORT-->
#236891) (#238467) # Backport This will backport the following commits from `main` to `9.1`: - [[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)](#236891) <!--- Backport version: 10.0.2 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-10-10T12:27:05Z","message":"[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/188090\n\n## Summary\n\nThis PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.\n\n## Running test instructions\n\n- Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal)\n- Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml`\n```yaml\n---\nname: {{ deployment_name }}\nsettings:\n autoscaling_enabled: {{ autoscaling_enabled }}\nmetadata:\n system_owned: false\nresources:\n elasticsearch:\n - region: {{ region }}\n settings:\n dedicated_masters_threshold: 6\n plan:\n cluster_topology:\n - zone_count: 1\n elasticsearch:\n node_attributes:\n data: hot\n instance_configuration_id: gcp.es.datahot.n2.68x10x45\n node_roles:\n - master\n - ingest\n - remote_cluster_client\n - data_hot\n - transform\n - data_content\n id: hot_content\n size:\n value: 1024\n resource: memory\n elasticsearch:\n version: {{ stack_version }}\n deployment_template:\n id: gcp-storage-optimized\n ref_id: main-elasticsearch\n enterprise_search: []\n kibana:\n - elasticsearch_cluster_ref_id: main-elasticsearch\n region: {{ region }}\n plan:\n cluster_topology:\n - instance_configuration_id: gcp.kibana.n2.68x32x45\n zone_count: 1\n size:\n value: 1024\n resource: memory\n kibana:\n version: {{ stack_version }}\n ref_id: main-kibana\n```\n\n- Create an ECH deployment by running the following command\n\n```bash\nqaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing\n```\n\n- Run the tests by running the following command\n\n```bash\nqaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts\n```\n\nwhere `<kibana-root>` is the absolute path to the Kibana's root folder.","sha":"d99c5b4b7211c1510f18f2b6260e24ea327b7d5a","branchLabelMapping":{"^v9.3.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v9.2.0","v9.3.0","v9.1.6","v8.18.9","v8.19.6","v9.0.9"],"title":"[Security Solution] Add FTR tests for prebuilt rules OOM testing","number":236891,"url":"https://github.com/elastic/kibana/pull/236891","mergeCommit":{"message":"[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/188090\n\n## Summary\n\nThis PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.\n\n## Running test instructions\n\n- Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal)\n- Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml`\n```yaml\n---\nname: {{ deployment_name }}\nsettings:\n autoscaling_enabled: {{ autoscaling_enabled }}\nmetadata:\n system_owned: false\nresources:\n elasticsearch:\n - region: {{ region }}\n settings:\n dedicated_masters_threshold: 6\n plan:\n cluster_topology:\n - zone_count: 1\n elasticsearch:\n node_attributes:\n data: hot\n instance_configuration_id: gcp.es.datahot.n2.68x10x45\n node_roles:\n - master\n - ingest\n - remote_cluster_client\n - data_hot\n - transform\n - data_content\n id: hot_content\n size:\n value: 1024\n resource: memory\n elasticsearch:\n version: {{ stack_version }}\n deployment_template:\n id: gcp-storage-optimized\n ref_id: main-elasticsearch\n enterprise_search: []\n kibana:\n - elasticsearch_cluster_ref_id: main-elasticsearch\n region: {{ region }}\n plan:\n cluster_topology:\n - instance_configuration_id: gcp.kibana.n2.68x32x45\n zone_count: 1\n size:\n value: 1024\n resource: memory\n kibana:\n version: {{ stack_version }}\n ref_id: main-kibana\n```\n\n- Create an ECH deployment by running the following command\n\n```bash\nqaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing\n```\n\n- Run the tests by running the following command\n\n```bash\nqaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts\n```\n\nwhere `<kibana-root>` is the absolute path to the Kibana's root folder.","sha":"d99c5b4b7211c1510f18f2b6260e24ea327b7d5a"}},"sourceBranch":"main","suggestedTargetBranches":["9.1","8.18","9.0"],"targetPullRequestStates":[{"branch":"9.2","label":"v9.2.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/238438","number":238438,"state":"OPEN"},{"branch":"main","label":"v9.3.0","branchLabelMappingKey":"^v9.3.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/236891","number":236891,"mergeCommit":{"message":"[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/188090\n\n## Summary\n\nThis PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.\n\n## Running test instructions\n\n- Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal)\n- Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml`\n```yaml\n---\nname: {{ deployment_name }}\nsettings:\n autoscaling_enabled: {{ autoscaling_enabled }}\nmetadata:\n system_owned: false\nresources:\n elasticsearch:\n - region: {{ region }}\n settings:\n dedicated_masters_threshold: 6\n plan:\n cluster_topology:\n - zone_count: 1\n elasticsearch:\n node_attributes:\n data: hot\n instance_configuration_id: gcp.es.datahot.n2.68x10x45\n node_roles:\n - master\n - ingest\n - remote_cluster_client\n - data_hot\n - transform\n - data_content\n id: hot_content\n size:\n value: 1024\n resource: memory\n elasticsearch:\n version: {{ stack_version }}\n deployment_template:\n id: gcp-storage-optimized\n ref_id: main-elasticsearch\n enterprise_search: []\n kibana:\n - elasticsearch_cluster_ref_id: main-elasticsearch\n region: {{ region }}\n plan:\n cluster_topology:\n - instance_configuration_id: gcp.kibana.n2.68x32x45\n zone_count: 1\n size:\n value: 1024\n resource: memory\n kibana:\n version: {{ stack_version }}\n ref_id: main-kibana\n```\n\n- Create an ECH deployment by running the following command\n\n```bash\nqaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing\n```\n\n- Run the tests by running the following command\n\n```bash\nqaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts\n```\n\nwhere `<kibana-root>` is the absolute path to the Kibana's root folder.","sha":"d99c5b4b7211c1510f18f2b6260e24ea327b7d5a"}},{"branch":"9.1","label":"v9.1.6","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.9","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.6","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/238437","number":238437,"state":"OPEN"},{"branch":"9.0","label":"v9.0.9","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
#236891) (#240012) # Backport This will backport the following commits from `main` to `9.2`: - [[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)](#236891) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-10-10T12:27:05Z","message":"[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/188090\n\n## Summary\n\nThis PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.\n\n## Running test instructions\n\n- Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal)\n- Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml`\n```yaml\n---\nname: {{ deployment_name }}\nsettings:\n autoscaling_enabled: {{ autoscaling_enabled }}\nmetadata:\n system_owned: false\nresources:\n elasticsearch:\n - region: {{ region }}\n settings:\n dedicated_masters_threshold: 6\n plan:\n cluster_topology:\n - zone_count: 1\n elasticsearch:\n node_attributes:\n data: hot\n instance_configuration_id: gcp.es.datahot.n2.68x10x45\n node_roles:\n - master\n - ingest\n - remote_cluster_client\n - data_hot\n - transform\n - data_content\n id: hot_content\n size:\n value: 1024\n resource: memory\n elasticsearch:\n version: {{ stack_version }}\n deployment_template:\n id: gcp-storage-optimized\n ref_id: main-elasticsearch\n enterprise_search: []\n kibana:\n - elasticsearch_cluster_ref_id: main-elasticsearch\n region: {{ region }}\n plan:\n cluster_topology:\n - instance_configuration_id: gcp.kibana.n2.68x32x45\n zone_count: 1\n size:\n value: 1024\n resource: memory\n kibana:\n version: {{ stack_version }}\n ref_id: main-kibana\n```\n\n- Create an ECH deployment by running the following command\n\n```bash\nqaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing\n```\n\n- Run the tests by running the following command\n\n```bash\nqaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts\n```\n\nwhere `<kibana-root>` is the absolute path to the Kibana's root folder.","sha":"d99c5b4b7211c1510f18f2b6260e24ea327b7d5a","branchLabelMapping":{"^v9.3.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport missing","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v9.2.0","v9.3.0","v8.19.6","v9.0.9"],"title":"[Security Solution] Add FTR tests for prebuilt rules OOM testing","number":236891,"url":"https://github.com/elastic/kibana/pull/236891","mergeCommit":{"message":"[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/188090\n\n## Summary\n\nThis PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.\n\n## Running test instructions\n\n- Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal)\n- Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml`\n```yaml\n---\nname: {{ deployment_name }}\nsettings:\n autoscaling_enabled: {{ autoscaling_enabled }}\nmetadata:\n system_owned: false\nresources:\n elasticsearch:\n - region: {{ region }}\n settings:\n dedicated_masters_threshold: 6\n plan:\n cluster_topology:\n - zone_count: 1\n elasticsearch:\n node_attributes:\n data: hot\n instance_configuration_id: gcp.es.datahot.n2.68x10x45\n node_roles:\n - master\n - ingest\n - remote_cluster_client\n - data_hot\n - transform\n - data_content\n id: hot_content\n size:\n value: 1024\n resource: memory\n elasticsearch:\n version: {{ stack_version }}\n deployment_template:\n id: gcp-storage-optimized\n ref_id: main-elasticsearch\n enterprise_search: []\n kibana:\n - elasticsearch_cluster_ref_id: main-elasticsearch\n region: {{ region }}\n plan:\n cluster_topology:\n - instance_configuration_id: gcp.kibana.n2.68x32x45\n zone_count: 1\n size:\n value: 1024\n resource: memory\n kibana:\n version: {{ stack_version }}\n ref_id: main-kibana\n```\n\n- Create an ECH deployment by running the following command\n\n```bash\nqaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing\n```\n\n- Run the tests by running the following command\n\n```bash\nqaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts\n```\n\nwhere `<kibana-root>` is the absolute path to the Kibana's root folder.","sha":"d99c5b4b7211c1510f18f2b6260e24ea327b7d5a"}},"sourceBranch":"main","suggestedTargetBranches":["9.0"],"targetPullRequestStates":[{"branch":"9.2","label":"v9.2.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/238438","number":238438,"state":"OPEN"},{"branch":"main","label":"v9.3.0","branchLabelMappingKey":"^v9.3.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/236891","number":236891,"mergeCommit":{"message":"[Security Solution] Add FTR tests for prebuilt rules OOM testing (#236891)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/188090\n\n## Summary\n\nThis PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.\n\n## Running test instructions\n\n- Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal)\n- Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml`\n```yaml\n---\nname: {{ deployment_name }}\nsettings:\n autoscaling_enabled: {{ autoscaling_enabled }}\nmetadata:\n system_owned: false\nresources:\n elasticsearch:\n - region: {{ region }}\n settings:\n dedicated_masters_threshold: 6\n plan:\n cluster_topology:\n - zone_count: 1\n elasticsearch:\n node_attributes:\n data: hot\n instance_configuration_id: gcp.es.datahot.n2.68x10x45\n node_roles:\n - master\n - ingest\n - remote_cluster_client\n - data_hot\n - transform\n - data_content\n id: hot_content\n size:\n value: 1024\n resource: memory\n elasticsearch:\n version: {{ stack_version }}\n deployment_template:\n id: gcp-storage-optimized\n ref_id: main-elasticsearch\n enterprise_search: []\n kibana:\n - elasticsearch_cluster_ref_id: main-elasticsearch\n region: {{ region }}\n plan:\n cluster_topology:\n - instance_configuration_id: gcp.kibana.n2.68x32x45\n zone_count: 1\n size:\n value: 1024\n resource: memory\n kibana:\n version: {{ stack_version }}\n ref_id: main-kibana\n```\n\n- Create an ECH deployment by running the following command\n\n```bash\nqaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing\n```\n\n- Run the tests by running the following command\n\n```bash\nqaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts\n```\n\nwhere `<kibana-root>` is the absolute path to the Kibana's root folder.","sha":"d99c5b4b7211c1510f18f2b6260e24ea327b7d5a"}},{"branch":"8.19","label":"v8.19.6","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/238437","number":238437,"state":"OPEN"},{"branch":"9.0","label":"v9.0.9","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"url":"https://github.com/elastic/kibana/pull/238467","number":238467,"branch":"9.1","state":"OPEN"}]}] BACKPORT-->
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
…stic#236891) **Partially addresses:** elastic#188090 ## Summary This PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages. ## Running test instructions - Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal) - Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml` ```yaml --- name: {{ deployment_name }} settings: autoscaling_enabled: {{ autoscaling_enabled }} metadata: system_owned: false resources: elasticsearch: - region: {{ region }} settings: dedicated_masters_threshold: 6 plan: cluster_topology: - zone_count: 1 elasticsearch: node_attributes: data: hot instance_configuration_id: gcp.es.datahot.n2.68x10x45 node_roles: - master - ingest - remote_cluster_client - data_hot - transform - data_content id: hot_content size: value: 1024 resource: memory elasticsearch: version: {{ stack_version }} deployment_template: id: gcp-storage-optimized ref_id: main-elasticsearch enterprise_search: [] kibana: - elasticsearch_cluster_ref_id: main-elasticsearch region: {{ region }} plan: cluster_topology: - instance_configuration_id: gcp.kibana.n2.68x32x45 zone_count: 1 size: value: 1024 resource: memory kibana: version: {{ stack_version }} ref_id: main-kibana ``` - Create an ECH deployment by running the following command ```bash qaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing ``` - Run the tests by running the following command ```bash qaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts ``` where `<kibana-root>` is the absolute path to the Kibana's root folder.
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
…stic#236891) **Partially addresses:** elastic#188090 ## Summary This PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to **Detection Prebuilt Rules**. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages. ## Running test instructions - Set up QAF by following to the [instructions](https://docs.elastic.dev/appex-qa/qaf/getting-started) (internal) - Place the following Elastic Cloud plan in `~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.yml` ```yaml --- name: {{ deployment_name }} settings: autoscaling_enabled: {{ autoscaling_enabled }} metadata: system_owned: false resources: elasticsearch: - region: {{ region }} settings: dedicated_masters_threshold: 6 plan: cluster_topology: - zone_count: 1 elasticsearch: node_attributes: data: hot instance_configuration_id: gcp.es.datahot.n2.68x10x45 node_roles: - master - ingest - remote_cluster_client - data_hot - transform - data_content id: hot_content size: value: 1024 resource: memory elasticsearch: version: {{ stack_version }} deployment_template: id: gcp-storage-optimized ref_id: main-elasticsearch enterprise_search: [] kibana: - elasticsearch_cluster_ref_id: main-elasticsearch region: {{ region }} plan: cluster_topology: - instance_configuration_id: gcp.kibana.n2.68x32x45 zone_count: 1 size: value: 1024 resource: memory kibana: version: {{ stack_version }} ref_id: main-kibana ``` - Create an ECH deployment by running the following command ```bash qaf elastic-cloud deployments create --stack-version 9.3.0 --version-validation --deployment-name prebuilt-rules-oom-test-9.3.0 --environment production --no-autoscaling --no-sso --region gcp-us-west2 --plan prebuilt_rules_oom_testing ``` - Run the tests by running the following command ```bash qaf kibana ftr run-config --ec-deployment-name prebuilt-rules-oom-test-9.3.0 --kibana-repo-root <kibana-root> <kibana-root>/x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/oom_testing/configs/ess_basic_license.config.ts ``` where `<kibana-root>` is the absolute path to the Kibana's root folder.
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
1 similar comment
|
Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync. |
Partially addresses: #188090
Summary
This PR contains FTR tests designed to expose potential Out of Memory (OOM) issues in Kibana when performing memory-intensive operations related to Detection Prebuilt Rules. The tests open a possibility to test against any Prebuilt Rules fleet package version including prerelease packages.
Running test instructions
~/.qaf/config/cloud_plans/prebuilt_rules_oom_testing.ymlwhere
<kibana-root>is the absolute path to the Kibana's root folder.