Some general questions

[jenisys]

Requirement traceability has different types of relations, like:

• CASE 1: a component/software-part/function satisfies a requirement
• CASE 2: a test verifies a requirement
• CASE 3: a test verifies some aspect/behavior, but not any requirement

From what I understand by looking at the description, you cover:

• CASE 1 for function(s)
• CASE 2 / CASE 3 is detected through test coverage (or manual verification)
• BUT: You cannot distinguish between CASE 2 or CASE 3 !? (except in the manual case)

If my understanding is correct:

• Would it not be better to provide also a verify_req() macro/decorator to ensure that CASE 2 is covered ?!
• Would it not be better to use satisfy_req as term to distinguish which relation to a requirement is meant ?!
• Use these terms in the traceability report: satisfied instead of: traced, verified (for CASE 2) instead of covered and covered for CASE 3

SEE ALSO:

• https://mbse4u.com/wp-content/uploads/2023/02/SysMLv1.7-MBSE4U-Reference-Sheet.pdf (Requirements section)
• https://community.infineon.com/t5/Blogs/Requirements-Traceability-with-mantra/ba-p/864822#.

[mhatzl]

Hi, thanks for checking out mantra :)

in general, my goal with mantra is to keep requirements traceability as simple as possible for developers, because they are the ones that have to add/maintain the traces in code.
That's why I went with having only the req macro. This way, as developer you only need to know/care about the req macro.
(the reqcov function one is a bit unfortunate, because you cannot define the same name for two proc-macros in the same crate...)

For the req attribute macro (and the doc-comment variant), I'd say if the macro is set on a function that has a #[test] attribute, then it verifies the requirement. Otherwise, it satisfies it.
This is likely a really big oversimplification, but automatically deciding wether a requirement is fully or just partially satisfied/verified by code/test is tricky to solve without cumbersome manual annotations. At least I haven't yet found an approach that I am happy with.

I do like your distinction between verified and satisfied for function like macros.
This way, you could "wrap" assertions in a verify_req!() macro, and other code blocks in satisfy_req!().

I currently don't consider/care about CASE 3, because if a test isn't covering a requirement, it's state won't affect the traceability report that much. You can still add the test result to mantra and do your manual checks, but you will need your own report template or direct access to the database.

Using covered for CASE 2 was mostly because I wanted to differentiate between automated tests and verification through manual reviews.
Not happy myself with it and I might change it to verified and manually-verified.

Thanks again for your remarks.
They are really helpful for the changes I am planning for mantra.

[mhatzl]

For traces to configuration files, being able to state verifies and satisfies explicit would also be helpful. so maybe I adapt the req attribute macro to allow specifying the variant if mantra is not able to detect it automatically. If mantra cannot detect it automatically and nothing is set, I'd go with satisfies by default and put out a warning.