[cpullvm] Don't stop on first test failure in Linux builds, write test results to xml logs#360
Draft
jonathonpenix wants to merge 1 commit into
Draft
Conversation
…t results to xml logs With automerge now enabled, we have had more consistent failures, even in Linux x86 builds. And, we've now seen failures that have been masked by other failures since we stop after the first one. So let's try what seems to be done in ATfE and use lit's `--ignore-fail` and dump all results into xml. A few things to note: 1. Windows builds are not impacted by this--they will still fail at the first error. We don't have QEMU testing enabled there, so we're really only running the lit tests anyway, so I don't think this is as large of an issue. We can re-evaluate this as needed. 2. We might need to think about how this works for ex: PRs. As-is, if there are failures I *think* the precheckin build will still succeed. So either we need an extra step, or people need to be diligent about checking the test reports. 3. I'm also not sure how this will work with eld? Given that its lit tests are split with different defaults. There might be some work needed to make this work well. Note that the code added here is from or is derived from ATfE. All license/copyright info we already have in the modified files should already be correct/appropriate. Signed-off-by: Jonathon Penix <jpenix@qti.qualcomm.com>
da5d9fb to
336c8b9
Compare
Contributor
Author
|
This might need more work as mentioned in the description, but just creating this for discussion about what Arm does, to see what the results would look like, etc. Edit: Also confirming that we're going to need to do something for eld to handle the whole "invoke lit multiple times" thing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With automerge now enabled, we have had more consistent failures, even in Linux x86 builds. And, we've now seen failures that have been masked by other failures since we stop after the first one.
So let's try what seems to be done in ATfE and use lit's
--ignore-failand dump all results into xml.A few things to note:
Note that the code added here is from or is derived from ATfE. All license/copyright info we already have in the modified files should already be correct/appropriate.