(ShadowManager): NullPointerException on shadow desired state field deletion

[edreanernst]

Describe the bug
When the local shadow on a Greengrass deployment is updated with a desired state of null for a nested field (the accepted method to remove the field's state from the shadow), Shadow Manager throws a NullPointerException, presumably when it attempts to calculate a delta.

To Reproduce
First, send a payload to Shadow Manager such as:
{"state": {"desired": {"mystate": {"mysub": "123"}}}}
Then send a payload after that:
{"state": {"desired": {"mystate": {"mysub": null}}}}

Expected behavior
The expectation is that the field mysub in my example should be removed from the shadow document without any errors.

Actual behavior
A NullPointerException is thrown and continues to be thrown whenever other subsequent shadow updates are attempted until the null is cleared from the document by another update to the shadow for field mysub with a value other than null.

The NullPointerException is:

[ERROR] (AwsEventLoop 1) com.aws.greengrass.ipc.common.ExceptionUtil: Unhandled exception in IPC. {}
java.lang.NullPointerException
        at com.aws.greengrass.shadowmanager.model.ShadowStateMetadata.buildMetadata(ShadowStateMetadata.java:294)
        at com.aws.greengrass.shadowmanager.model.ShadowStateMetadata.getDeltaMetadata(ShadowStateMetadata.java:254)
        at com.aws.greengrass.shadowmanager.model.ShadowDocument.getDelta(ShadowDocument.java:202)
        at com.aws.greengrass.shadowmanager.ipc.UpdateThingShadowRequestHandler.publishDeltaMessage(UpdateThingShadowRequestHandler.java:309)
        at com.aws.greengrass.shadowmanager.ipc.UpdateThingShadowRequestHandler.lambda$handleRequest$2(UpdateThingShadowRequestHandler.java:236)
        at com.aws.greengrass.ipc.common.ExceptionUtil.translateExceptions(ExceptionUtil.java:33)
        at com.aws.greengrass.shadowmanager.ipc.UpdateThingShadowRequestHandler.handleRequest(UpdateThingShadowRequestHandler.java:94)
        at com.aws.greengrass.shadowmanager.ipc.UpdateThingShadowIPCHandler.handleRequest(UpdateThingShadowIPCHandler.java:83)
        at com.aws.greengrass.shadowmanager.ipc.UpdateThingShadowIPCHandler.handleRequest(UpdateThingShadowIPCHandler.java:29)
        at software.amazon.awssdk.eventstreamrpc.OperationContinuationHandler.onContinuationMessage(OperationContinuationHandler.java:324)
        at software.amazon.awssdk.crt.eventstream.ServerConnectionContinuationHandler.onContinuationMessageShim(ServerConnectionContinuationHandler.java:53)

Environment

• OS: Raspbian 12 Bookworm
• JDK version: 11.0.28
• Nucleus version: 2.15.0
• Shadow Manager version: 2.3.11

Additional context
In my opinion this bug is quite significant, because when this condition is triggered all shadow management is impacted until the condition is cleared, which might be quite difficult to debug.

Take note that this issue does not occur if the top-level desired state is set to null (mystate in my example above). In that case it works as expected.

[AniruddhaKanhere]

@edreanernst thank you for taking the time to report this issue.

Let us take a deeper look and we will get back to you.

[kstribrnAmzn]

This issue is really fascinating. I'm not sure yet why it would throw the exception on ShadowStateMetadata.java:294. I suspect the metadataNode is Null which is causing the NPE.

I will be looking into this issue more and will start with research to understand why it would be Null.

[kstribrnAmzn]

Can you provide logs around the time this happens? I'm setting up an instance to try reproducing this.

[kstribrnAmzn]

Took some time but I got the reproduction steps figured out. The issue arises when you update a local-only shadow to remove a field before the periodic shadow sync occurs. The NPE will be continually thrown until the sync to the cloud completes.

Now that I know the path, I'll start looking into a fix.

[kstribrnAmzn]

Fix for this issue - #213