JMX Scraper: Refactoring integration tests assertion framework

[robsunday]

Component(s)

jmx-scraper

Is your feature request related to a problem? Please describe.

Current implementation of integration tests assertion framework (MetricAssertions class) reports assertion failures in a very verbose and non-optimal way.
In case of assertion failure it prints out all received metrics and assertions multiple times. It makes finding the issue really hard. The more metrics is checked the harder it is and often the only way to find out what went wrong is selectively commenting out the code.
Also some assertions are missing details forcing developer to guess what went wrong.

Describe the solution you'd like

New assertion framework should be created fulfilling the following requirements:

1. Print only the metric that failed the assertion
2. Provide meaningful description and the metric property details needed to identify root cause of assertion failure
3. Fail when any of expected metrics was not received
4. Fail when unexpected metric was received (there is no assertion defined for it). There should be mechanism to opt out form this default behavior. It may be useful when it is very hard/impossible to force tested target to report some metric.
5. API must be easy to use, concise and reliable (it should be hard to miss some metric checks)

New framework will coexist for some time with the current one, so currently implemented integration tests may be ported incrementally in separate PRs.
Once all tests are ported the old framework will be removed from the code

Describe alternatives you've considered

No response

Additional context

No response

[SylvainJuge]

#1566 relates to this.

The migration work will likely be spread over many PRs.

[SylvainJuge]

Remaining steps to close this:

• migration of wildfly integration tests [jmx-scraper] wildfly refactor assertions #1587
• migration of tomcat integration tests [jmx-scraper] Assertions refactoring - Tomcat integration test converted #1589
• remove hasTypedDataPoints and waitAndAssertMetrics methods.

[SylvainJuge]

@robsunday is there anything left to do here or can we close this ? I can see that we still have a few usages of waitAndAssertMetrics method for example.

I expect your current work with kafka might definitely bring a few more things to change related to assertions.

[robsunday]

New framework is implemented