Results tree as files hierarchy on disk

[SebMichaud]

Would it be feasible to have an option which allows Run Test (Scalar Path).vi to output results in the same hierarchy as the hierarchy of tests VIs on disk ?

[francois-normandin]

Hi @SebMichaud ,

Expanding on your suggestion, I've had a quick brainstorm session with @jimkring yesterday and we compiled a list of grouping types that different use cases would warrant. I'm adding the list here so anyone can reflect and comment on it.

Auto-Grouping

Auto-grouping would be selectable for Test Suites.

none

Current workflow, no change. Sections are added to the report in the order they are processed, and based on the call-chain hierarchy. No sorting. The ordering/grouping is determined by execution order, which is non-deterministic. This would remain the default grouping for calling a Test VI directly, or for calling the RunTests method on a single VI file path.

project hierarchy

Tests are organized with the same structure as the Virtual Folders in the project environment. The developer chose to organize its unit tests in a virtual hierarchy, and the report will mimic the structure. This would be the default grouping for calling the RunTests method on a project reference.

disk hierarchy

Tests are organized with the same structure as the on-disk folder structure. The developer chose to organize its unit tests in a folder hierarchy, and the report will mimic the structure. This would be the default grouping for calling the RunTests method on a folder path.

namespace hierarchy

Tests are loosely organized in different ways (single VIs, part of classes, part of lvlib). The developer wants the test report to be determinisitic but no clear organization is defined. This grouping would classify the assertion results based on the namespacing, such that individual VIs are at root, and those in libraries would organize based on the library name.

For example, the test VI myTestLib.lvlib:class1.lvclass:TestA.vi running three assertions would report:

-- myTestLib
    -- class1
        -- TestA
            -- assert1
            -- assert2
            -- assert3

[SebMichaud]

That sounds really good, thanks for this abstract.