process: adjust testing levels and traceability

[pahmann]

see #821

[github-actions]

License Check Results

🚀 The license check preparation job ran successfully.

Status: ⚠️ Needs Review

Click to expand output

2025/03/31 12:32:57 Downloading https://releases.bazel.build/7.4.0/release/bazel-7.4.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: docs
Loading: 0 packages loaded
    currently loading: docs
Analyzing: target //docs:license.check.python (1 packages loaded, 0 targets configured)
Analyzing: target //docs:license.check.python (1 packages loaded, 0 targets configured)

Analyzing: target //docs:license.check.python (96 packages loaded, 10 targets configured)

Analyzing: target //docs:license.check.python (124 packages loaded, 379 targets configured)

Analyzing: target //docs:license.check.python (131 packages loaded, 840 targets configured)

Analyzing: target //docs:license.check.python (144 packages loaded, 2465 targets configured)

Analyzing: target //docs:license.check.python (144 packages loaded, 2465 targets configured)

Analyzing: target //docs:license.check.python (147 packages loaded, 4487 targets configured)

Analyzing: target //docs:license.check.python (148 packages loaded, 4611 targets configured)

Analyzing: target //docs:license.check.python (148 packages loaded, 4611 targets configured)

INFO: Analyzed target //docs:license.check.python (149 packages loaded, 4736 targets configured).
[9 / 13] JavaToolchainCompileClasses external/rules_java~/toolchains/platformclasspath_classes; 0s processwrapper-sandbox ... (2 actions running)
[11 / 13] JavaToolchainCompileBootClasspath external/rules_java~/toolchains/platformclasspath.jar; 0s processwrapper-sandbox
INFO: Found 1 target...
Target //docs:license.check.python up-to-date:
  bazel-bin/docs/license.check.python
  bazel-bin/docs/license.check.python.jar
[13 / 13] no actions running
INFO: Elapsed time: 19.588s, Critical Path: 2.39s
INFO: 13 processes: 9 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 13 total actions
INFO: Running command line: bazel-bin/docs/license.check.python docs/formatted.txt -review -project automotive.score -repo https://github.com/eclipse-score/score -token otyhZ4eaRYK1tKLNNF-Y
[main] INFO Querying Eclipse Foundation for license data for 69 items.
[main] INFO Found 45 items.
[main] INFO Querying ClearlyDefined for license data for 25 items.
[main] INFO Found 25 items.
[main] INFO License information could not be automatically verified for the following content:
[main] INFO 
[main] INFO pypi/pypi/-/docutils/0.21.2
[main] INFO 
[main] INFO This content is either not correctly mapped by the system, or requires review.
[main] INFO A review is required for pypi/pypi/-/docutils/0.21.2.
[main] INFO A review request already exists https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/19880 .

[pahmann]

@aschemmel-tech this is the draft PR to rework the testing level and to unify component and component integration testing as discussed in the process WG meeting last week.

[github-actions]

The created documentation from the pull request is available at: docu-html

[aschemmel-tech]

Additional to inline comments: I really think we need to provide a complete example for a "unit test with gtest" and a "feature test with ITF".

[aschemmel-tech]

I would not say so. The detailed design describes the the units forming a component and their interactions - see https://eclipse-score.github.io/score/main/process/process_areas/implementation/_assets/detailed_design_example.html - so the interfaces and interactions between units can be used as a basis for testing. Maybe rather make the integration testing of component architecture optional (in case a component has no (sub-)components.

[aschemmel-tech]

I think what is still unclear: are the component integration tests 6-9 or 6-10? In the SW verification plan it seems to say that "level 1" is 6-9. Is this what you want to say? And I also think we need to explain better what a "Unit Tester" has to do in the guideline: 1. read the Detailed Desig, create a test for every interface of the unit showing at least every flow in dynamic diagrams, 2. follow the DD to the component requirements - test these 3. fill attributes including test description 4. link test to ??? (not component requirements)

[pahmann]

From WG working meeting 6-10 goes down to green "Component".
6-9 is for grey part.
Unclear is the handling of requirements.
Start with the simple use case (without Component architecture due to unit simplicity)
Detailed design is directly related to component requirements and implicitly covered.

[aschemmel-tech]

I was thinking if we not just also add a link from Unit Test to the Component Requirements. Reasoning is that this would be also easier to demonstrate a full requirement coverage by testing. But also we should keep the DD link possibility for unit test because we do want also the possibility to test DD instead have a Component requirement for everything - maybe best to say you have to have one or the other.