Skip to content

Conversation

@mikepenz
Copy link
Owner

Copilot AI review requested due to automatic review settings December 30, 2025 09:06
@github-actions
Copy link

TestsPassed ☑️SkippedFailed ❌️
Example Pytest Report4 ran2 passed0 skipped2 failed
TestResult
Example Pytest Report
test_sample.test_which_succeeds✅ passed
test_sample.test_which_fails❌ failure
test_sample.test_with_error❌ failure
test_sample.test_with_flaky_success✅ passed

@github-actions
Copy link

TestsPassed ☑️Skipped ⚠️Failed ❌️
Example Ungrouped Test Report | test_10.xml3 ran3 passed0 skipped0 failed
Example Ungrouped Test Report | test_11.xml3 ran2 passed0 skipped1 failed
Example Ungrouped Test Report | test_12.xml3 ran2 passed1 skipped0 failed

@github-actions
Copy link

TestsPassed ✅SkippedFailed
Example Nested JUnit Test Report3 ran3 passed0 skipped0 failed
TestResult
Example Nested JUnit Test Report
ABC-0199: XMPP Ping
PingIntegrationTest.pingAsync (Normal)✅ passed
PingIntegrationTest.pingServer (Normal)✅ passed
ABC-0045: Multi-User Chat
MultiUserIntegrationTest.mucRoleTestForReceivingModerator (Normal)✅ passed

@mikepenz mikepenz merged commit a67ac83 into main Dec 30, 2025
12 of 18 checks passed
@mikepenz mikepenz deleted the fix/1480_1463 branch December 30, 2025 09:10
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances the handling of flaky test retries and fixes a critical bug where tests with the same name but different classnames/files were incorrectly merged. The solution introduces a compound key strategy and improves the retry detection logic.

Key changes:

  • Implements compound key (name|classname|file) to uniquely identify tests and prevent incorrect merging
  • Refactors retry handling logic to properly track flaky tests in all scenarios (success after failure, failure after success, all failures)
  • Adds comprehensive test coverage for flaky test scenarios

Reviewed changes

Copilot reviewed 5 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test_results/flaky_retries/flaky_with_classname_file.xml Test data file demonstrating flaky test with multiple failures then success, plus separate tests with same name but different classnames
test_results/flaky_retries/flaky_success_first.xml Test data file for edge case where success occurs before failures in execution order
test_results/flaky_retries/flaky_all_failures.xml Test data file for scenario where a flaky test never succeeds across all retries
src/testParser.ts Core fix implementing compound key strategy and improved retry handling logic
dist/index.js Compiled JavaScript matching the TypeScript source changes
__tests__/testParser.test.ts Four new comprehensive test cases validating the flaky test handling improvements

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

2 participants