Is your feature request related to a problem? Please describe.

Currently, the event writer implements partial de-duplication to send only the most recent events for a resource. This prevents processing unnecessary events and also helps reduce the queue size. However, when running at scale, the principal may be overwhelmed by the number of incoming events from large number of agents. The rate of processing events might be slower than the rate of inbound events. This results in accumulation of events on the principal waiting to be processed leading to increased memory usage. We need to implement de-duplication on the principal for incoming events so that the principal only processes the most recent event. This results in lower memory usage and also reduces the number of events processed.

Describe the solution you'd like

The principal should only keep track and process the most recent incoming event for a resource.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.