Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

process: verification guide for types and methods #721

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

pahmann
Copy link
Contributor

@pahmann pahmann commented Mar 18, 2025

As part of the interim audit, it was mentioned to better explain the testing methods and derivation techniques.

see #497

Copy link

github-actions bot commented Mar 18, 2025

License Check Results

🚀 The license check preparation job ran successfully.

Status: ⚠️ Needs Review

Click to expand output
2025/03/25 16:06:03 Downloading https://releases.bazel.build/7.4.0/release/bazel-7.4.0-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: docs
Loading: 0 packages loaded
    currently loading: docs
Analyzing: target //docs:license.check.python (1 packages loaded)
Analyzing: target //docs:license.check.python (1 packages loaded, 0 targets configured)
Analyzing: target //docs:license.check.python (1 packages loaded, 0 targets configured)

Analyzing: target //docs:license.check.python (90 packages loaded, 10 targets configured)

Analyzing: target //docs:license.check.python (94 packages loaded, 10 targets configured)

Analyzing: target //docs:license.check.python (106 packages loaded, 207 targets configured)

Analyzing: target //docs:license.check.python (130 packages loaded, 824 targets configured)

Analyzing: target //docs:license.check.python (140 packages loaded, 2425 targets configured)

Analyzing: target //docs:license.check.python (144 packages loaded, 2465 targets configured)

Analyzing: target //docs:license.check.python (144 packages loaded, 2465 targets configured)

Analyzing: target //docs:license.check.python (148 packages loaded, 4611 targets configured)

INFO: Analyzed target //docs:license.check.python (149 packages loaded, 4736 targets configured).
[7 / 13] Creating runfiles tree bazel-out/k8-opt-exec-ST-d57f47055a04/bin/tools/dash/formatters/dash_format_converter.runfiles [for tool]; 0s local
[9 / 13] JavaToolchainCompileClasses external/rules_java~/toolchains/platformclasspath_classes; 0s processwrapper-sandbox ... (2 actions running)
[12 / 13] [Prepa] Building docs/license.check.python.jar ()
INFO: Found 1 target...
Target //docs:license.check.python up-to-date:
  bazel-bin/docs/license.check.python
  bazel-bin/docs/license.check.python.jar
INFO: Elapsed time: 22.294s, Critical Path: 2.44s
INFO: 13 processes: 9 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 13 total actions
INFO: Running command line: bazel-bin/docs/license.check.python docs/formatted.txt -review -project automotive.score -repo https://github.com/eclipse-score/score -token otyhZ4eaRYK1tKLNNF-Y
[main] INFO Querying Eclipse Foundation for license data for 69 items.
[main] INFO Found 45 items.
[main] INFO Querying ClearlyDefined for license data for 25 items.
[main] INFO Found 25 items.
[main] INFO License information could not be automatically verified for the following content:
[main] INFO 
[main] INFO pypi/pypi/-/docutils/0.21.2
[main] INFO 
[main] INFO This content is either not correctly mapped by the system, or requires review.
[main] INFO A review is required for pypi/pypi/-/docutils/0.21.2.
[main] INFO A review request already exists https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/19880 .

Copy link

The created documentation from the pull request is available at: docu-html

@pahmann pahmann force-pushed the pahmann_verification_types_methods branch from ad727c3 to 465c4a8 Compare March 25, 2025 15:04
Additionally, addressing Typos.

see #497

Signed-off-by: Philipp Ahmann <[email protected]>
@pahmann pahmann force-pushed the pahmann_verification_types_methods branch from 465c4a8 to 655f13a Compare March 25, 2025 15:17
@pahmann pahmann marked this pull request as draft March 25, 2025 16:06
@pahmann pahmann linked an issue Mar 25, 2025 that may be closed by this pull request
@pahmann pahmann self-assigned this Mar 27, 2025
Copy link
Contributor

@aschemmel-tech aschemmel-tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see inline comments

* - Structural Branch Coverage (Code coverage)
- structural-branch-coverage
* - Structural Condition Coverage (Code coverage)
- structural-condition-coverage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should explain in text why we are deviating from Iso 26262 (our tooling is more reliable for this) and why it is ok (this is even more fine grained as branch coverage and we consider it in argumentation if not 100% are reached)

section :ref:`verification_concept_types_methods`. This guide helps to get an understanding what
the different methods and derivation techniques mean and how to create test cases using the same.

Methods
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please consider to put this in seperate document as we have defined the "gd_meth__" need for methods. And link here.

least once during the testing process.

Condition coverage is also creating branch coverage, so a 100% `structural-condition-coverage` will not
guarantee a 100% branch coverage. The actual branch coverage percentage depends on the actual code
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to check this statement

under test.

Fro C++ based unit testing gTest is the "software tool" which supports conditional coverage, but
has no direct option to create branch coverage. This has be considered, while writing unit tests.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

avtually it is gcov which provides the structural coverage, it supports both but it is more reliable for condition coverage

**4. Design Test Cases**

Create test cases using the representative values from each equivalence class.
Aim to cover all classes in your test suite.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing "fuzz-testing" mentioned in SW verification plan

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please align with new definition what is used in S-CORE and remove all TestTypes which are not tests: e.g. inspection, walkthrough, static code analysis, ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

The methods of ISO 26262 shall be explained
2 participants