Clarify hardware profile and instance configuration related docs for ECH (elastic#2039)

## Background

There are a bunch of hardware profile and instance configuration related
ECH docs that are not clear enough and have caused some confusions to
users and support so far.
After syncing with @yuvielastic @maggieghamry @stefnestor and @jakommo,
we decided to make some doc updates to clarify.

Details in this internal ticket
(elastic/support-tech-lead#1586). Below is a
quick recap of what we would like to change.

## For reviewers

- Requested @yuvielastic your kind review from PM's perspective
- Requested @claudia-correia your kind review from engineering's
perspective
- Requested @maggieghamry your kind review from support's perspective
- Requested @eedugon or @shainaraskas your kind review from docs team's
perspective

## Recap of what to change

### **:: [1]**


https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile
- Change title to `Manage hardware profiles`
- Add that version <= 7.10 can't do hardware profile migration

### **:: [2]**


https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/change-hardware
- Update the doc from mentioning "Change hardware" to "Customize
instance configuration" and make this update all through this doc page.
<= It's technically more proper to say "instance configuration" but not
just "hardware"
- Add "Hardware profile is also referenced as deployment templates (DTs)
in ECH to make it more logically connectable"
- Add an "In addition" paragraph to explain some details, that has
confused users and support so far. We have already confirmed with Yuvi
from technical perspective.

### **:: [3]**


https://www.elastic.co/docs/deploy-manage/production-guidance/optimize-performance/search-speed
- Reference regarding "hardware profile" is not correct. It should be
the real hardware profile one as mentioned in [1].

### **:: [4]**


https://www.elastic.co/docs/deploy-manage/production-guidance/optimize-performance/indexing-speed
- Reference regarding "hardware profile" is not correct. It should be
the real hardware profile one as mentioned in [1].

### :: [5]

Move
https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/change-hardware
- "customize IC" doc under
https://www.elastic.co/docs/deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile
"manage hardware profile" to make it more logically clear.

---

## 🔍 Preview links for changed docs
-
[deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile.md](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/2039/deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile)
-
[deploy-manage/deploy/elastic-cloud/change-hardware.md](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/2039/deploy-manage/deploy/elastic-cloud/change-hardware)
-
[deploy-manage/production-guidance/optimize-performance/indexing-speed.md](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/2039/deploy-manage/production-guidance/optimize-performance/indexing-speed)
-
[deploy-manage/production-guidance/optimize-performance/search-speed.md](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/2039/deploy-manage/production-guidance/optimize-performance/search-speed)



---

cc @maggieghamry @stefnestor @jakommo

---------

Co-authored-by: Cláudia Correia <[email protected]>
Co-authored-by: Edu González de la Herrán <[email protected]>

Author: 3 people

deploy-manage/deploy/elastic-cloud/change-hardware.md

@@ -8,9 +8,13 @@ products:
   - id: cloud-hosted
 ---
 
-# Change hardware [ec-change-hardware-for-a-specific-resource]
+# Customize instance configuration [ec-change-instance-configuration]
 
-The virtual hardware on which {{stack}} deployments run is defined by instance configurations. To learn more about what an instance configuration is, refer to [Instance configurations](cloud://reference/cloud-hosted/hardware.md#ec-getting-started-configurations).
+This document explains how to modify the instance configurations used by specific components of your deployment without changing the overall hardware profile assigned to the deployment. This advanced configuration scenario is useful in situations where you need to migrate an Elasticsearch tier or stateless resource to a different hardware type.
+
+## Consideration [ec-considerations-on-changing-ic]
+
+{{stack}} deployments run on virtual hardware defined by instance configurations. For more details, refer to [Hardware profiles](./ec-change-hardware-profile.md#ec-hardware-profile) and [Instance configurations](cloud://reference/cloud-hosted/hardware.md#ec-getting-started-configurations) documents.
 
 When a deployment is created, each {{es}} tier and stateless resource (e.g., Kibana) gets an instance configuration assigned to it, based on the hardware profile used. The combination of instance configurations defined within each hardware profile is designed to provide the best possible outcome for each use case. Therefore, it is not advisable to use instance configurations that are not specified on the hardware profile, except in specific situations in which we may need to migrate an {{es}} tier or stateless resource to a different hardware type. An example of such a scenario is when a cloud provider stops supporting a hardware type in a specific region.
 
@@ -89,6 +93,30 @@ Having an instance configuration mismatch between the deployment and the hardwar
 
 ## Deprecated instance configurations (ICs) and deployment templates (DTs) [ec-deprecated-icdt]
 
-A list of deprecated and valid ICs/DTs can be found on the [Available regions, deployment templates and instance configurations](cloud://reference/cloud-hosted/ec-regions-templates-instances.md) page, as well as through the API, using `hide_deprecated` to return valid ICs/DTs. For example, to return valid ICs/DTs the following request can be used: `https://api.elastic-cloud.com/api/v1/deployments/templates?region=us-west-2&hide_deprecated=true`. To list only the deprecated ones, this can be used: `https://api.elastic-cloud.com/api/v1/deployments/templates?region=us-west-2&metadata=legacy:true`.
+Hardware profile is also referenced as deployment templates in {{ecloud}}. 
+
+You can find a list of deprecated and valid instance configurations (ICs) and deployment templates (DTs) in two ways:
+
+### Public documentation page
+
+Visit the [Available regions, deployment templates and instance configurations](cloud://reference/cloud-hosted/ec-regions-templates-instances.md) page for detailed information.
+
+### API access
+
+Use the [Get deployment templates API](https://www.elastic.co/docs/api/doc/cloud/operation/operation-get-deployment-templates-v2) with query parameters like `hide_deprecated` to retrieve valid ICs and DTs. This API request returns a list of DTs along with the respective ICs referenced within each DT.
+
+For example, 
+* To return valid ICs/DTs the following request can be used: `https://api.elastic-cloud.com/api/v1/deployments/templates?region=us-west-2&hide_deprecated=true`. 
+* To list only the deprecated ones, this can be used: `https://api.elastic-cloud.com/api/v1/deployments/templates?region=us-west-2&metadata=legacy:true`.
 
 If a deprecated IC/DT is already in use, it can continue to be used. However, creating or migrating to a deprecated IC/DT is no longer possible and will result in a plan failing. In order to migrate to a valid IC/DT, navigate to the **Edit hardware profile** option in the Cloud UI or use the [Deployment API](https://www.elastic.co/docs/api/doc/cloud/operation/operation-migrate-deployment-template).
+
+::::{note}
+Deployments using {{stack}} versions prior to 7.10 do not support changing the hardware profile through the {{ecloud}} console or API. To change the hardware profile, first upgrade to version 7.10 or later.
+::::
+
+In addtion, you can refer to below information about how these terminologies are referenced. 
+* _Deprecated_ is also referenced as _legacy_. 
+* Using the `metadata=legacy:true` query parameter will return only legacy/deprecated DTs.
+* Using the `hide_deprecated=true` query parameter will return only valid DTs.
+* Not using any of the query parameters above will return all DTs. In this case, check the presence of `legacy: true` in the `metadata` entries within the API response, to verify if an IC/DT is deprecated or not.

deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile.md

@@ -8,7 +8,9 @@ products:
   - id: cloud-hosted
 ---
 
-# Change hardware profiles [ec-change-hardware-profile]
+# Manage hardware profiles [ec-change-hardware-profile]
+
+## Hardware profile [ec-hardware-profile]
 
 Deployment hardware profiles deploy the {{stack}} on virtual hardware. Each hardware profile has a different blend of storage, RAM, and vCPU.
 
@@ -24,6 +26,11 @@ The {{ecloud}} console indicates when a new version of a hardware profile is ava
 
 ## Change the hardware profile using the {{ecloud}} console [ec_change_the_hardware_profile_using_the_elastic_cloud_console]
 
+::::{note}
+Deployments using {{stack}} versions prior to 7.10 do not support changing the hardware profile through the {{ecloud}} console or API. To change the hardware profile, first upgrade to version 7.10 or later.
+::::
+
+
 ### Upgrade to the newest version of your current hardware profile [ec_upgrade_to_the_newest_version_of_your_current_hardware_profile]
 
 Note that if there’s no indication that a newer version is available, that means that your deployment is already running on the latest version of that hardware profile.
@@ -72,6 +79,11 @@ If your deployment is configured for high availability, the hardware profile cha
 
 ## Change the hardware profile using the API [ec_change_the_hardware_profile_using_the_api]
 
+::::{note}
+Deployments using {{stack}} versions prior to 7.10 do not support changing the hardware profile through the {{ecloud}} console or API. To change the hardware profile, first upgrade to version 7.10 or later.
+::::
+
+
 Prerequisites:
 
 * A valid {{ecloud}} [API key](../../api-keys/elastic-cloud-api-keys.md) (`$EC_API_KEY`)
@@ -176,21 +188,28 @@ Consider this configuration for ingestion use cases with 1-4 days of data availa
 
 ### CPU optimized (ARM) [ec-profiles-compute-optimized-arm]
 
-This profile is similar to CPU optimized profile but is powered by AWS Graviton2 instances. You can find the exact storage, memory, and vCPU allotment on the [hardware details page](cloud://reference/cloud-hosted/hardware.md#ec-getting-started-configurations) for each cloud provider.
+This profile is similar to CPU optimized profile but powered by ARM instances. Currently, we offer ARM instances on AWS. You can find the exact storage, memory, and vCPU allotment on the [hardware details page](cloud://reference/cloud-hosted/hardware.md#ec-getting-started-configurations) for each cloud provider.
 
 **Ideal use case**
 
 Consider this configuration for ingestion use cases with 1-4 days of data available for fast access and for search use cases with indexing and querying workloads. Provides the most CPU resources per unit of RAM.
 
-
-### Vector search optimized (ARM) [ec-profiles-vector-search]
+### Vector search optimized [ec-profiles-vector-search]
 
 This profile is suited for Vector search, Generative AI and Semantic search optimized workloads. You can find the exact storage, memory, and vCPU allotment on the [hardware details page](cloud://reference/cloud-hosted/hardware.md#ec-getting-started-configurations) for each cloud provider.
 
 **Ideal use case**
 
 Optimized for applications that leverage Vector Search and/or Generative AI. Also the optimal choice for utilizing ELSER for semantic search applications. Broadly suitable for all semantic search, text embedding, image search, and other Vector Search use cases.
 
+### Vector search optimized (ARM) [ec-profiles-vector-search-arm]
+
+This profile is suited for Vector search, Generative AI and Semantic search optimized workloads powered by ARM instances. Currently, we offer ARM instances on AWS. You can find the exact storage, memory, and vCPU allotment on the [hardware details page](cloud://reference/cloud-hosted/hardware.md#ec-getting-started-configurations) for each cloud provider.
+
+**Ideal use case**
+
+Optimized for applications that leverage Vector Search and/or Generative AI. Also the optimal choice for utilizing ELSER for semantic search applications. Broadly suitable for all semantic search, text embedding, image search, and other Vector Search use cases.
+
 
 ### General purpose [ec-profiles-general-purpose]
 
@@ -203,7 +222,7 @@ Suitable for ingestion use cases with 5-7 days of data available for fast access
 
 ### General purpose (ARM) [ec-profiles-general-purpose-arm]
 
-This profile is similar to the General purpose profile but is powered by AWS Graviton2 instances. You can find the exact storage, memory, and vCPU allotment on the [hardware details page](cloud://reference/cloud-hosted/hardware.md#ec-getting-started-configurations) for each cloud provider.
+This profile is similar to General purpose profile but powered by ARM instances. Currently, we offer ARM instances on AWS. You can find the exact storage, memory, and vCPU allotment on the [hardware details page](cloud://reference/cloud-hosted/hardware.md#ec-getting-started-configurations) for each cloud provider.
 
 **Ideal use case**
 

deploy-manage/production-guidance/optimize-performance/indexing-speed.md

@@ -133,7 +133,7 @@ If indexing is I/O-bound, consider increasing the size of the filesystem cache (
 Stripe your index across multiple SSDs by configuring a RAID 0 array. Remember that it will increase the risk of failure since the failure of any one SSD destroys the index. However this is typically the right tradeoff to make: optimize single shards for maximum performance, and then add replicas across different nodes so there’s redundancy for any node failures. You can also use [snapshot and restore](../../tools/snapshot-and-restore.md) to backup the index for further insurance.
 
 ::::{note}
-In {{ech}} and {{ece}}, you can choose the underlying hardware by selecting different hardware profiles or deployment templates. Refer to [ECH > Change hardware](/deploy-manage/deploy/elastic-cloud/change-hardware.md) and [ECE deployment templates](/deploy-manage/deploy/cloud-enterprise/configure-deployment-templates.md) for more details.
+In {{ech}} and {{ece}}, you can choose the underlying hardware by selecting different hardware profiles or deployment templates. Refer to [ECH > Manage hardware profiles](/deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile.md) and [ECE > Manage deployment templates](/deploy-manage/deploy/cloud-enterprise/configure-deployment-templates.md) for more details.
 ::::
 
 ### Local vs. remote storage [_local_vs_remote_storage]

deploy-manage/production-guidance/optimize-performance/search-speed.md

@@ -66,7 +66,7 @@ If your searches are I/O-bound, consider increasing the size of the filesystem c
 If your searches are CPU-bound, consider using a larger number of faster CPUs.
 
 ::::{note}
-In {{ech}} and {{ece}}, you can choose the underlying hardware by selecting different hardware profiles or deployment templates. Refer to [ECH > Change hardware](/deploy-manage/deploy/elastic-cloud/change-hardware.md) and [ECE deployment templates](/deploy-manage/deploy/cloud-enterprise/configure-deployment-templates.md) for more details.
+In {{ech}} and {{ece}}, you can choose the underlying hardware by selecting different hardware profiles or deployment templates. Refer to [ECH > Manage hardware profiles](/deploy-manage/deploy/elastic-cloud/ec-change-hardware-profile.md) and [ECE > Manage deployment templates](/deploy-manage/deploy/cloud-enterprise/configure-deployment-templates.md) for more details.
 ::::
 
 ### Local vs. remote storage [_local_vs_remote_storage_2]

deploy-manage/toc.yml

@@ -45,6 +45,8 @@ toc:
                   - file: deploy/elastic-cloud/configure.md
                     children:
                       - file: deploy/elastic-cloud/ec-change-hardware-profile.md
+                        children:
+                          - file: deploy/elastic-cloud/change-hardware.md 
                       - file: deploy/elastic-cloud/ec-customize-deployment-components.md
                       - file: deploy/elastic-cloud/edit-stack-settings.md
                       - file: deploy/elastic-cloud/add-plugins-extensions.md
@@ -57,7 +59,6 @@ toc:
                       - file: deploy/elastic-cloud/switch-from-apm-to-integrations-server-payload.md
                   - file: deploy/elastic-cloud/find-cloud-id.md
               - file: deploy/elastic-cloud/ec-vcpu-boost-instance.md
-              - file: deploy/elastic-cloud/change-hardware.md
               - file: deploy/elastic-cloud/manage-deployments-using-elastic-cloud-api.md
               - file: deploy/elastic-cloud/keep-track-of-deployment-activity.md
               - file: deploy/elastic-cloud/restrictions-known-problems.md