OpenSearch 3 to OpenSearch 3 Migrations #1611
Description
Is your feature request related to a problem?
The lack of OpenSearch 3 as a source prevents users from using the migration tooling for same-version migrations, such as to address document reindexing or transformations.
What solution would you like?
Migration Assistant and all products within the migration repo will support migrating from OpenSearch 3 to OpenSearch 3. This should include compatibility for features introduced in OpenSearch 3, such as star-tree indices (GA in 3.1) as well as the ability to perform metadata migrations, existing data migrations with RFS, and Capture and Replay functionalities.
Additionally, while not all transformations required for metadata, existing data, and live traffic will be supported "out of the box," there should be a mechanism for users performing such migrations to add custom transformations to the OpenSearch-Migrations transformation framework.
What alternatives have you considered?
Performing snapshot and restore, which while efficient, lacks support for transformation and reindexing capabilities.
Do you have any additional context?
Migrating from OpenSearch 3 introduces several considerations that were not as relevant for earlier versions:
- Lucene Version: OpenSearch 3 uses a newer version of Lucene (Lucene 10). The migration process must ensure that index data from this Lucene version is handled correctly in OpenSearch. This will require taking a dependency on JDK 21.
- JDK Version: Lucene 10 requires JDK 21 support, which may cause compatibility issues with the existing tooling which should be addressed and validated.
The feature should account for these differences and ensure that indices and data created in OpenSearch 3 are properly handled in same-version migrations without the need for significant manual intervention.