-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
type:featureNew capability or enhancementNew capability or enhancement
Description
Description:
Currently, the NUnit results when there is a failure are plotted like this (example taken from here):
The test failing here is only one (BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest), but there are other 2 tests that were run. Additionally, those two test may have some RecordMessage log that could be useful to show (see #420).
After discussing with @FraserGreenroyd , we agreed on this example structure for the results plotting:
Inner results:
BH.Tests.Engine.Base.Query.UnpackTests.ErrorReportingTest (Passed):
Messages:
Some error logged via BH.Engine.Base.Compute.RecordError
Some error logged via BH.Engine.Base.Compute.RecordWarning
Some error logged via BH.Engine.Base.Compute.RecordNote
BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest (Failed):
Shouldly.ShouldAssertException : Comparing object equivalence, at path:
result.OfType<BHoMObject>().FirstOrDefault() [BH.oM.Base.BHoMObject]
BHoM_Guid [System.Guid]
Expected value to be
a1380fbb-b004-4bac-b28c-0d45780a5e9c
but was
4fa87007-d24a-d44b-9d3e-aa9d743b7059
at BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest() in C:\ProgramData\BHoMGitHub\BHoM_Engine\.ci\unit-tests\Base_Engine_Tests\Query\Unpack.cs:line 143
Error:
Some warning logged via BH.Engine.Base.Compute.RecordError
Warnings:
Some warning logged via BH.Engine.Base.Compute.RecordWarning
BH.Tests.Engine.Base.Query.UnpackTests.WarningReportingTest (Passed):
Warnings:
Some warning logged via BH.Engine.Base.Compute.RecordWarning
BH.Tests.Engine.Base.Query.UnpackTests.NoteReportingTest (Passed):
Messages:
Some note logged via BH.Engine.Base.Compute.RecordNote
The information from the RecordMessage log is shown differently depending on whether the test has passed or failed:
- when the test has passed, the recorded messages are clustered under a "Messages" category (no differentiation btw Errors/Warnings/Note). This was suggested by @FraserGreenroyd to avoid confusing readers.
- when the test has failed, the recorded messages are grouped as normal (Errors/Warnings/Notes).
For each test, the recorded messages should appear after any message from NUnit.
Metadata
Metadata
Assignees
Labels
type:featureNew capability or enhancementNew capability or enhancement
