Log Throttling - Feature Test Summary

[tonyreigns]

Feature: #31719

List of FAT projects affected

• com.ibm.ws.logging_2_fat

Test strategy

This upcoming logging feature introduces the capability to throttling logs based on the number of repeated log events(configurable) based on either the messageID or entire message over a 5 minute period(not configurable). This is a feature that will be enabled by default and is defaulted to 1000 messageIDs.

Test scenarios:
All the test scenarios unless otherwise specified, are using server.xml configuration.

Negative test cases:

• testInvalidLogThrottlingMaxMessagesConfig: Ensure that invalid throttleMaxMessagesPerWindow configuration is caught, set to the default and has a warning printed.
• testInvalidLogThrottlingMessageTypeConfig: Ensure that invalid throttleType configuration is caught, set to the default and has a warning printed.
• testCaseSensitiveThrottleTypeFullMessage: Ensure that configuration is not case sensitive for throttleType message
• testCaseSensitiveThrottletypeMessageID: Ensure that configuration is not case sensitive for throttleType messageID

Positive test cases:

• testLogThrottlingWarningTriggered: Ensure log throttling warning is printed when logs throttling is active and ensure it's only printed once.
• testLogThrottlingActiveLowOccurrence: Ensure that log throttling is activated immediately after threshold is met.
• testLogThrottlingActiveHighOccurrence: Ensure that log throttling is activated and remains active with a high volume of logs continuing to be logged.
• testLogThrottlingHighMaxMessages: Ensures that log throtting is not activated when the log threshold is not met.
• testLogThrottlingActiveFullMessage: Ensure that when using the message throttleType, applicable messages are throttled while other are not.
• testLogThrottlingDisabled: Ensure that throttling is not activated when throttling is disabled.
• testDefaultConfig: Test that the default configuration throttles correctly. throttleMaxMessagesPerWindow=1000, messageType=messageID
• testLogThrottlingActiveLowOccurrenceEnv: Test server.env configuration. Both throttleType and throttleMaxMessagesPerWindow are tested here.
• testLogThrottlingActiveLowOccurrenceBootstrap: Test bootstrap.properties configuration. Both throttleType and throttleMaxMessagesPerWindow are tested here.
• testLogThrottlingEmptyMaxMessages: Test an empty throttleMaxMessagesPerWindow config and ensure the default is used.
• testLogThrottlingCheckpoint: Test that throttling behaves correctly with InstantOn/Checkpoint.

Side note: This feature caused some test buckets in the FIPS Beta pipeline to start failing due to some buckets not expecting their logs to be throttled or for the warning to be printed. The majority of the failures were quickly fixed by updating the fatTestPorts.xml to disable throttling by default but some buckets weren't using that xml in their servers. The fatTestPorts changes were ported to WS-CD-Open and the bucket that wasn't using the default was also updated. I believe this issue has been resolved and is being tracked by: https://libh-proxy1.fyre.ibm.com/cognitive/defectAnalysis.html?defect=307247

No recent occurrences as of 08/12. SOE was completed with no further failures, defect has been closed.

Confidence Level

Collectively as a team you need to assess your confidence in the testing delivered based on the values below. This should be done as a team and not an individual to ensure more eyes are on it and that pressures to deliver quickly are absorbed by the team as a whole.

4 - We have delivered all automated testing we believe is needed for the golden paths of this feature and have good coverage of the error/outlying scenarios. While more testing of the error/outlying scenarios could be added we believe there is minimal risk here and the cost of providing these is considered higher than the benefit they would provide.