Skip to content

Improve NUnit result plotting #421

@alelom

Description

@alelom

Description:

Currently, the NUnit results when there is a failure are plotted like this (example taken from here):

image

The test failing here is only one (BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest), but there are other 2 tests that were run. Additionally, those two test may have some RecordMessage log that could be useful to show (see #420).

After discussing with @FraserGreenroyd , we agreed on this example structure for the results plotting:

Inner results:
    BH.Tests.Engine.Base.Query.UnpackTests.ErrorReportingTest (Passed):
        Messages: 
            Some error logged via BH.Engine.Base.Compute.RecordError
            Some error logged via BH.Engine.Base.Compute.RecordWarning
            Some error logged via BH.Engine.Base.Compute.RecordNote

    BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest (Failed):
        Shouldly.ShouldAssertException : Comparing object equivalence, at path:
        result.OfType<BHoMObject>().FirstOrDefault() [BH.oM.Base.BHoMObject]
            BHoM_Guid [System.Guid]
            Expected value to be
        a1380fbb-b004-4bac-b28c-0d45780a5e9c
            but was
        4fa87007-d24a-d44b-9d3e-aa9d743b7059
        at BH.Tests.Engine.Base.Query.UnpackTests.PurposedlyFailingTest() in C:\ProgramData\BHoMGitHub\BHoM_Engine\.ci\unit-tests\Base_Engine_Tests\Query\Unpack.cs:line 143

        Error:
            Some warning logged via BH.Engine.Base.Compute.RecordError

        Warnings:
            Some warning logged via BH.Engine.Base.Compute.RecordWarning
    
    BH.Tests.Engine.Base.Query.UnpackTests.WarningReportingTest (Passed):
        Warnings:
            Some warning logged via BH.Engine.Base.Compute.RecordWarning
    
    BH.Tests.Engine.Base.Query.UnpackTests.NoteReportingTest (Passed):
        Messages:
            Some note logged via BH.Engine.Base.Compute.RecordNote

The information from the RecordMessage log is shown differently depending on whether the test has passed or failed:

  • when the test has passed, the recorded messages are clustered under a "Messages" category (no differentiation btw Errors/Warnings/Note). This was suggested by @FraserGreenroyd to avoid confusing readers.
  • when the test has failed, the recorded messages are grouped as normal (Errors/Warnings/Notes).

For each test, the recorded messages should appear after any message from NUnit.

Metadata

Metadata

Assignees

Labels

type:featureNew capability or enhancement

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions