Tidy methods to commit to state store #6520
Description
User Story
As a developer, I want the methods to commit to the state store to be in one place and clear about the distinction between them, so that I know whether an update to the state store will be done synchronously, asynchronously, or by a method set in configuration.
Description / Background
Currently to update the state store, you can either do it synchronously or asynchronously.
If you do it synchronously, you pass a transaction object to the state store object. For some transactions you need to do that with a method on the transaction object, in order for some of the logic that should happen during synchronous commit to happen. There are also methods on the state store interface to add a transaction, that look like you should call them directly. Those are actually only called directly by the state store committer, when it processes asynchronous commits and applies them to the state store.
If you want to update the state store asynchronously, you need to use a completely different class to send a commit to the state store commit queue. There's no method on the state store interface to do this.
We'd like to clean this up and make it clear to a developer looking at the state store interface which ways of committing there are, and how and when each should be used.
Technical Notes / Implementation Details
Current state
There's StateStore.addFilesTransaction and StateStore.addPartitionTransaction which are currently only used directly by the state store committer.
There's the synchronousCommit method on all the transaction objects, e.g. AddFilesTransaction.
There's SqsFifoStateStoreCommitRequestSender, which sends asynchronous commits.
Everywhere in the system that supports asynchronous commits has its own logic for whether to call the commit request sender or the synchronousCommit method.
Changes
The state store could have synchronousCommit and asynchronousCommit methods, and those should always be used. Alternatively these could be called commit and synchronousCommit so that commit is the one most people choose by default. We can explain why you won't want to use synchronousCommit most of the time, as it has an impact on throughput of the state store.
Since some extra parts will be needed inside the state store, we can adjust the builder and static constructors for DynamoDBTransactionLogStateStore to be specific about what you want to be able to do with it. For asynchronous commits we'll need to create a commit request sender. For synchronous commits where the transaction synchronousCommit method takes extra parameters, we could pass them in in order to be able to do those types of commit synchronously. The resulting state store should fail if the user attempts a type of commit that it's not deliberately configured to be able to do.