Add an option to retry based on error message

[Noordsestern]

Requirement

Short Story

I have flaky test cases that have 60% chance to fail, but only in test automation, not in manual testing. Retrying helps. All test cases fail with almost the same error message. I would like to rerun those test cases that fail with an expected error message. All other errors are actual failures and I do not want to retry them.

Long Story

I have browser test cases running against a SUT implemented in MaterialUI. MaterialUI in itself seems to be flaky when being used too fast (even in manual testing). However, I can't change the SUT, so test cases have to work on MaterialUI, too. As the error messages all hold similar cause (only details in the stacktrace differ), i would like to retry these flaky test cases automatically. All other errors shall count as failure without a retry.

Suggestion

*** Test Cases ***
Test Super Flaky MaterialUI
    [Tags]    test:retry(3)(MaterialPopover)    # flaky error message always contains "MaterialPopover"
    Login User
    ....