[BUG] Running the solution is getting null error in metadata migration

[rudney-souza]

What is the bug?

After running the snapshot command, and waiting till completion, ive ran the command for metadata evaluate and got an null error

How can one reproduce the bug?

making a snapshot from ES_6.7 and then try to get metadata evaluated

What is the expected behavior?

A normal migration from ES_6.7 to OS_2.17

What is your host/environment?

ES_6.7 to OS_2.17

Do you have any screenshots?

[peternied]

@rudney-souza Thanks for opening this issue. To help us understand what happened could you grab the metadata log file associated with failure, see Metadata-Migration#troubleshoot-issues for how to get this.

[rudney-souza]

Thanks for response @peternied, so i got those logs in here

[peternied]

@rudney-souza Thanks - I'm not seeing any [ERROR] entries or exception call stacks in what you've posted. Is it possible for you to share with me the full log contents privately?

I'm on the OpenSearch Slack instance as Peter Nied which might be a faster and private way to discover the root cause, see OpenSearch Slack. Alternatively you can email at petern@amazon.com

[peternied]

Extracted the full callstack from the log file:

2024-12-04 19:55:04,302 ERROR o.o.m.c.Evaluate [main] Unexpected failure
java.lang.NullPointerException: null
        at org.opensearch.migrations.transformation.rules.IndexMappingTypeRemoval.canApply(IndexMappingTypeRemoval.java:56) ~[transformation-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.transformation.rules.IndexMappingTypeRemoval.applyTransformation(IndexMappingTypeRemoval.java:73) ~[transformation-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.transformation.rules.IndexMappingTypeRemoval.applyTransformation(IndexMappingTypeRemoval.java:40) ~[transformation-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.bulkload.transformers.Transformer_ES_6_8_to_OS_2_11.lambda$transformIndex$3(Transformer_ES_6_8_to_OS_2_11.java:78) ~[RFS-0.1.0-SNAPSHOT.jar:?]
        at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
        at org.opensearch.migrations.bulkload.transformers.Transformer_ES_6_8_to_OS_2_11.transformIndex(Transformer_ES_6_8_to_OS_2_11.java:78) ~[RFS-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.bulkload.transformers.Transformer_ES_6_8_to_OS_2_11.lambda$transformGlobalMetadata$1(Transformer_ES_6_8_to_OS_2_11.java:41) ~[RFS-0.1.0-SNAPSHOT.jar:?]
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) ~[?:?]
        at org.opensearch.migrations.bulkload.transformers.Transformer_ES_6_8_to_OS_2_11.transformGlobalMetadata(Transformer_ES_6_8_to_OS_2_11.java:38) ~[RFS-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.bulkload.transformers.CompositeTransformer.transformGlobalMetadata(CompositeTransformer.java:16) ~[RFS-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.bulkload.worker.MetadataRunner.migrateMetadata(MetadataRunner.java:25) ~[RFS-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.commands.MigratorEvaluatorBase.migrateGlobalMetadata(MigratorEvaluatorBase.java:115) ~[MetadataMigration-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.commands.MigratorEvaluatorBase.migrateAllItems(MigratorEvaluatorBase.java:86) ~[MetadataMigration-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.commands.Evaluate.execute(Evaluate.java:29) [MetadataMigration-0.1.0-SNAPSHOT.jar:?]
        at org.opensearch.migrations.MetadataMigration.main(MetadataMigration.java:77) [MetadataMigration-0.1.0-SNAPSHOT.jar:?]

[peternied]

@rudney-souza I've merged a fix for this issue, you can pick it up immediately by going onto the bootstrap box with the following steps

cd /opensearch-migrations/deployment/cdk/opensearch-service-migration
git fetch origin
git merge origin/main
./buildDockerImages.sh
# Confirm any cdk.context.json settings and then redeploy
./deploy.sh

[rudney-souza]

That sounds nice @peternied , thanks a lot!

[rudney-souza]

Now all working, thanks a lot @peternied