Large snapshot generator test

[jugal-chauhan]

Description

This PR implements an automated test framework to generate large-scale snapshot datasets for testing Migration Assistant's Reindex From Snapshot (RFS) functionality. This pipeline enables repeatable testing with configurable parameters, making it easier to generate large snapshots (more than 10TB) for testing. This test utilizes a multiplication JSON transformer that multiplies a single document N times.

Document Multiplier Test Workflow:

• Jenkins parameters are collected for configuring the test
• Environment Setup includes creation of unique test ID, defining source and migration CDK context, and setting up deployment environment
• Source data creation takes place by creating a test index with custom settings and inserting dummy test data in bulk batches. Each document contains detailed metadata and fields. An initial RFS snapshot is taken.
• Backfill process setup creates a transformer config file that includes multiplication parameter and rules for document replication
• Gathers initial cluster statistics such as document count and size
• Test execution takes off with initiating backfill with provided number of RFS workers from jenkins parameter. Backfill uses transformer for multiplication. Backfill uses the same index and provides detailed logs with progress status.
• Progress monitoring is enabled that detects any error when document count is stuck during backfill. Once backfill has been complete, test periodically checks for document count to ensure stability before stopping backfill.
• Records final cluster statistics and performance metric
• A large snapshot is created and stored in a specific S3 bucket
• Pipeline cleans up all deployed stacks after test completion

Issues Resolved

MIGRATIONS-2423

Testing

Jenkins Pipeline : https://migrations-ci-staging.opensearch.org/job/k8s-large-snapshot-generator/

Check List

• New functionality includes testing
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[peternied]

@jugal-chauhan This has been open for more than a month, need any help with next steps or should this be closed out?

[jugal-chauhan]

Work on this PR has been paused. Closing the PR for time being.