You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Jan 13, 2021. It is now read-only.
Currently, the SQSPoller is invoked by the CW Event schedule and invokes the message processor to process batches of messages synchronously. The current algorithm works and is cost-efficient, but cannot support high throughput. An alternative would be if the SQSPoller and MessageDispatcher classes were put into separate lambda functions and the SQSPoller invoked the MessageDispatcher lambda function asynchronously to process a batch of messages, then continued on. The MessageDispatcher lambda function would handle updating the queue with results of the execution of its batch of messages. This would dramatically increase parallelism of message processing. One caveat is that a high volume queue could lead to too many lambdas being invoked, causing the account to hit its concurrency limit. However, setting a concurrency limit on the MessageProcessor function could be used to control how many concurrent invocations would be supported.
Currently, the SQSPoller is invoked by the CW Event schedule and invokes the message processor to process batches of messages synchronously. The current algorithm works and is cost-efficient, but cannot support high throughput. An alternative would be if the SQSPoller and MessageDispatcher classes were put into separate lambda functions and the SQSPoller invoked the MessageDispatcher lambda function asynchronously to process a batch of messages, then continued on. The MessageDispatcher lambda function would handle updating the queue with results of the execution of its batch of messages. This would dramatically increase parallelism of message processing. One caveat is that a high volume queue could lead to too many lambdas being invoked, causing the account to hit its concurrency limit. However, setting a concurrency limit on the MessageProcessor function could be used to control how many concurrent invocations would be supported.