[DOC] The usage example confuses me more than it actually helps

[sassman]

Describe what is/was unclear to you

I saw your talk at the rustweek, it was very cool big thanks for it!.
I thought your tool could replace a hand made one, so I gave it a try.

First on a complex workspace project then on a simple "hello-world" binary.

Both with the same result:

❯ mantra collect
[2025-05-16T09:27:21Z WARN ] No requirements were found.
No requirements found.

'2' requirements updated:
- `JIRAX-35604`
- `JIRAX-35605`

Then I looked closer to the usage example and I'm more puzzled that before.

Describe what would help you to better understand the documentaon

I would suggest the usage example(s) could take a total unbiased / unknown outsiders view, as if you hear the first time of the tool and you are not familiar with the config file or the individual .json files.

So guiding a user for a simple new hello world tool, and what are the steps to add when and to invoke what would be awesome.

A brief summary of what I did:

1. Generated a project

cargo new --bin demo-mantra-usage

# Cargo.toml
# other stuff omitted..

[dependencies]
mantra-rust-macros = "0.2.14"

2. Added some tracing "annotations" to the code

# src/main.rs

use mantra_rust_macros::req;

fn main() {
    foo();
}

#[req(JIRAX-35604)]
fn foo() {
    println!("foo");
}

3. Create the requirements "database"

# .requirements.json (Note: this comment is not in the file)

{
  "requirements": [
    {
      "id": "JIRAX-35604",
      "title": "Some req added from JSON.",
      "origin": "local",
      "manual": false,
      "deprecated": false,
      "data": {
        "custom_field": "some data"
      }
    },
    {
      "id": "JIRAX-35605",
      "title": "Another req added from JSON.",
      "origin": "local",
      "manual": false,
      "deprecated": false,
      "data": {
        "custom_field": "some data"
      }
    }
  ]
}

4. Create the minimal mantra config file

[[requirements]]
# The path to JSON files containing requirements.
files = [".requirements.json"]

5. Collect with mantra

mantra collect

6. Create report

mantra report --formats=html foo.html

Observations and Remarks

• the report does not show any of the requirement annotated in code
• it's not clear what of the files and steps are maybe optional
• in the config file, when omitting the [package] section, I would somehow expect the report gets this infos from the Cargo.toml manifest of the scanned project (if this is too much of an expectation, it would be worth to note why that is)
• it's unclear to me, what are the things I do wrong or I should better be doing to get the requirement collected and shown up in the report

Update: also in the readme show all ways to do things in one shot, is a bit contra helpful. I would divide this e.g. into 2 pieces

#[req(req_id)]
fn some_fn() {
  println!("This reports via the attribute macro");
}

fn bar() {
  println!("This reports via reqcov! macro call");
  reqcov!(function_like_trace);
}

[mhatzl]

Hi @sassman,

thank you very much for trying out mantra and the tips to improve the docs.
docs have certainly been lacking. sorry for that

about your problem:

I am honestly not quite sure what went wrong for you.
There are a few things that you could try

1. Use the markdown approach to specify the requirements

   In mantra.toml, use root instead of files under the requirements section.
   Check out the config in my pacemaker demo to see how.

2. Try removing the leading . for the .requirements.json file

   Not sure if a leading . could create a problem.

3. Wrap your requirement IDs in double quotes in your Rust code

   Your IDs contain - and are therefore not valid Rust identifiers.
   Either wrap your IDs in double quotes like #[req("JIRAX-35605")],
   or replace the - with _

4. Query the SQLite DB

   After running mantra collect, a mantra.db should have been created.
   You can access the DB and browse it either via CLI or some GUI tool.
   Of interest would be the Requirements table.
   If there are none in the table, it means something went wrong in parsing the JSON file.

To see if mantra itself works for you, I suggest checking out the pacemaker demo. If you are on Linux or MacOS, you need to change the backslashes to forward slashes in the coverage.json file (unfortunately...).

Running mantra collect in the pacemaker demo should then work without needing to run/build anything for an embedded device. This worked for me on a clean clone last week.

In general, I would really like to improve mantra's usability.
So instead of you trying to find out how to currently use mantra, I would really like to know how you would expect to interact with it, because I am likely going to change at least the structure of mantra.toml and JSON schemas.

One problem/tradeoff I will likely keep, is to specify project parameters in mantra.toml,
because I cannot assume that there will be a Cargo.toml, since mantra should also be usable for other languages. I could try to look for one, but in case two or more languages are used in a project, it might become tricky to select the correct project metadata. Maybe that is not that big of a problem and just takes effort to implement and handle such cases by giving some language configs more priority.

[sassman]

Thanks for all your hints, I guess 2 things made it working now:

# mantra.toml
[[traces]]
root = ""
keep-path-absolute = false

Now the traces from the code are actually found.

The other one was

Wrap your requirement IDs in double quotes in your Rust code.

So regarding your other points:

One problem/tradeoff I will likely keep, is to specify project parameters in mantra.toml, because I cannot assume that there will be a Cargo.toml, since mantra should also be usable for other languages

I see your point. And it's totally fine how it is right now. This is nothing I would put energy into.

In general, I would really like to improve mantra's usability.
I think a mdbook with just a handful chapters, walking a user step by step thru a minimal setup in a tutorial style would be something I could imagine to be super helpful.

The pacemaker demo is also very good, but it's also very complex, I guess this makes it super hard to distil what's the minimal setup to derive from the demo that might work for one.

Also in there are files, like the lsif.json file where it's unclear where do they come from or is mantra generating them or maintaining them etc.?

So I guess simple and less is more.

[mhatzl]

good to hear that you got it working and thanks for the feedback.

the lsif file is optional and used to resolve identifier names in Rust, because Rust's module system is too complex to resolve the fully qualified identifier just by checking one file. This was one of the last changes to mantra, and I am not that happy with how it is handled at the moment.

I am currently trying to make mantra work with libtest and LLVM code coverage.
Once this is done, mantra should be easier to use for non-embedded projects,
and then I will focus more on documentation.