Improvement: Verification Process enhancements

[pahmann]

Important

Make sure to link this issue with the PR for your improvement.

Following improvements in the verification process have been identified during the porting from incubator and adopting to the latest process model structure:

software_verification.rst (Verification Plan as part of PMP):

• Within the coverage of detailed design additional verification measures such as "data flow coverage" should be checked.
  

  score/docs/platform_management_plan/software_verification.rst

  Line 285 in 548fd89

  Coverage of detailed design

• Within the coverage of architectural design additional verification measures such as "sequence diagrams" should be checked. Question is also how to define the coverage of such measures.
  

  score/docs/platform_management_plan/software_verification.rst

  Line 297 in 548fd89

  Coverage of architectural design

verification_getstrt.rst (Verification Getting Started)

• Add a link to the automated test case creation currently in development by the testing community.
  

  score/docs/process/process_areas/verification/verification_getstrt.rst

  Line 64 in 548fd89

  Additionally, tools will be provided to automatically generate coverage tests based on the unit source code.

verification_guidance.rst (Guideline how to perform verification activities)

• It is hard to show full coverage of requirements, if everything is hidden in a PR review. A concept like "have a specification which links multiple tests of partially covered type" or similar is needed.
  

  score/docs/process/process_areas/verification/guidance/verification_guidance.rst

  Lines 81 to 85 in 548fd89

  	Two properties exists; one to show partial and one to show full coverage of a requirement.
  	For multiple test cases having a "partial coverage" a review has to be conducted to confirm
  	that a requirement is fully covered. The pull request description should indicate which requirements
  	are fully covered by the PR commits and which test cases are needed to fully cover the test case.
  	This is important, as multiple PRs may be needed to fully verify a single requirement.

• How to execute manual test locally, and report execution results need to be described.

• How to execute automated test cases locally (e.g. before creation of a PR) need to be described.
  

  score/docs/process/process_areas/verification/guidance/verification_guidance.rst

  Line 89 in 548fd89

  Test case execution

• Verification methods and types need to be described in more detail:
  

  score/docs/process/process_areas/verification/guidance/verification_guidance.rst

  Line 133 in 548fd89

  Verifiation types and methods

  • detail out what is meant by what is meant by e.g. “static code analysis” issue The methods of ISO 26262 shall be explained #497
  • the application of methods getting explained issue The application of the test methods must be recognizable #498

verification_process_toolreqs.rst (Verification requirements put to tooling)

• Add proper requirement link demanding independence of QA activities
  

  score/docs/process/process_areas/verification/guidance/verification_process_toolreqs.rst

  Line 144 in 548fd89

  The approver of a pull request shall differ from the author(s) of the pull request in all pull requests.

TODOs kept in source code:

Requires implementation of #599:

• score/docs/process/process_areas/verification/verification_concept.rst

  Line 153 in 548fd89

  For more information, see ``[TODO: Link to ITF documentation once available. Related feature request is #599]``.

Requires merge of PR #526:

• Improvement: Add links to implemenation work products in verification process #705
  

  score/docs/process/process_areas/verification/workflows.rst

  Line 31 in 548fd89

  ``[TODO: add :has: doc_concept__imp__concept, doc_getstrt__imp__getstrt after PR #526 is merged]``

• Improvement: Add links to implemenation work products in verification process #705
  

  score/docs/process/process_areas/verification/workproducts.rst

  Line 132 in 548fd89

  * : ref : ` implementation ` ``[TODO: Add ref once arch PR #526 is merged]`` implementing :need:`wp__sw_code_inspect`

TODOs from #441 comments:

• Add a defined workflow for justification in case of failing test cases. See:
  process: initial documentation of verification #441 (comment)
• within verification concept in test to requirements linking add a good example to the Process Requirements. See:
  process: initial documentation of verification #441 (comment)
• add guideline to link our tools to process requirements for tool qualification. See:
  process: initial documentation of verification #441 (comment)
• Test types and methods need more description and examples. See:
  process: initial documentation of verification #441 (comment)
  and process: initial documentation of verification #441 (comment)