localTPSExceededError so sleep for a second spams the log output #40
Description
Is your feature request related to a problem? Please describe.
This logging is happening way too many times per second to be Info level. I understand that this is going to happen once per thread, so that maybe a contributing factor here, but it is very intense in the logging. If you send your logs to somewhere like Sumologic or DataDog, this could be very expensive.
See attached gist with the log output snippit.
Describe the solution you'd like
I think this backoff time should be configurable, and one second may not even be enough for the default. If it is configurable, I can pass in 2 or 5 seconds, and this will actually alleviate the throttling issue it is hitting. Looking closer in the code, and at AWS limits, this is mitigating the 5 transactions per second limit, and makes sense to wait the amount of time it is.
A better option: Just make these logs debug level. If someone wants to see them, they can change the log level.
Describe alternatives you've considered
I could put a sleep time in my ProcessRecords implementation, but I don't know if that is good, bad, or indifferent...
Additional context
Log showing this happening many times a second.
https://gist.github.com/komealy/039b6b5ff8be646aa5b68c3d164bf234#file-vmware-go-kcl-v2-log