review and consolidate the metrics documentation

[mhow2]

We need to refine the existing documentation at https://scava-docs.readthedocs.io/en/latest/users/Scava-Metrics/ . A good starting point would be to review each metric's description with a least add the type (string) unit (hours, urls..) and possibly the range and increment steps it is expected to get as values.

I'm quoting what mentioned in a previous correspondence about that:

I already told Yannis about that : as you now, Boris and I
initiated the "Scava Metrics" document [1]. That's a good start but
it still misses some important data for some of the metrics that could
ease up building our quality models:

• what is the type of the value returned : Int ? String ?
  list ? URL ?
• what's the unit ? seconds ? hours ? scalar ?
• what is the expected range values ?
• what is the increment value within the interval ?

For example, if we take the metric "Topics" within the Newsgroup
metrics, it is described : "Using the clustering algorithm, we
determine which are the most frequent topics discussed in the
newsgroup.". The questions are then: what's the type of the returned
value ? How many references can we expect ? Do we expect URL(s) or topic
name(s) ? etc.

Of course, if the documentation provided from the REST API at /metrics endpoint is complete and structured enough we could choose to mention that the best way to get started with it is to use the REST API (and explain how).

[mhow2]

I had a look to the new metrics documentation.

A few comments by looking side by side at /metrics endpoints from a live instance (deployed this morning) and the new documentation. I'm taking the first metric (0) from the endpoint above as a matter of example:

  {
    "id": "bugs.cumulativeNewUsers",
    "name": "New Users (Cumulative)",
    "description": "The number of new users up to and including the current date.",
    "type": "LineChart",
    "datatable": {
      "cols": [
        {
          "name": "Date",
          "field": "$__date"
        },
        {
          "name": "Users",
          "field": "$cumulativeNumberOfNewUsers"
        }
      ]
    },
    "x": "Date",
    "y": "Users",
    "timeSeries": true,
    "metricId": "org.eclipse.scava.metricprovider.historic.bugs.newusers.NewUsersHistoricMetricProvider"
  },

1. What about the id field ? there is no string match on "bugs.cumulativeNewUsers" or even "cumulativeNewUsers" in the new documentation, can you clarify ?

2. metricId: I can find a match in the new documentation for org.eclipse.scava.metricprovider.historic.bugs.newusers. However, NewUsersHistoricMetricProvider is not mentioned in the new doc. Could it be BugsNewUsersHistoricMetric or is it a different thing ?

3. This one is more about terminology question : there is a key called metricId in the response but it seems to return a "Provider" and not to be unique to a metric. If I'm right, a good term could metricProviderId then ? moreover, having both id and metricId in the same response entry looks a bit confusing (at least to me).

4. What are you planning to do with the description fields from the REST API ? I'm not sure it always match the description string from new documentation ?

5. It seems org.eclipse.scava.metricprovider.historic.bugs.newusers depends on org.eclipse.scava.metricprovider.trans.bugs.activeusers but the latter is calculated "in the last 15 days". So I assume the former is calculated on the same basis, right ? maybe the description should reflect that and others metrics are concerned.

The goal in the end is to be able to easily go back and forth between the REST API response and the documentation details, in one way or the other.

Flagging @crossminer/ehu and @crossminer/sft

[creat89]

Hello @mhow2, I see what do you mean with you example. The fact is that @nnamokon created the documentation based on the metrics in general. However, what you are showing us comes from the visualization file if I'm correct, which in turn only covers historic metrics. What you observe in the example is a more specific description of how the metric org.eclipse.scava.metricprovider.historic.bugs.newusers is split in submetrics that are used in the visualization.
If I'm not wrong, visualization files haven't been changed yet and were not taken into account during the documentation. I'll discuss with the team tomorrow, as today is a bank holiday here in the UK.

[mhow2]

Hi Adrian, thank you for the answer.
I'm starting to understand both per your answer and @ambpro explanation:

I guess there is a confusion between metricId of metric with visualisation which is basically the visualization id and the metric-provider id metricId: which is the identifier of all the metric-providers inclusing factoid, transient and historic metrics.

So I understand that the raw values, which match to your new documentation can be found (for instance) at :

/raw/projects/p/docdokuplm/m/org.eclipse.scava.metricprovider.historic.bugs.newusers.NewUsersHistoricMetricProvider
This is where I find numberOfNewUsers and cumulativeNumberOfNewUsers that are mentioned in the new doc. I'm still wondering about NewUsersHistoricMetricProviderwhich is not mentioned in the new doc (as told before).

And to findout what are the avalaible (raw?) metric providers for a specific project :

/raw/metrics/p/docdokuplm

For example, here I can find the metricProviderId :
org.eclipse.scava.metricprovider.historic.bugs.newusers.model.BugsNewUsersHistoricMetric

But again I am wondering about not finding a reference to BugsNewUsersHistoricMetric in the new doc.

I'm not sure to understand everything yet. Moreover, I don't understand what values/endpoints I should use to implement our usecase.
Flagging @valeriocos too

[creat89]

@mhow2, the fact that you're not finding BugsNewUsersHistoricMetric might be just an error. I'm seeing that before that it contains the word model. That makes reference to the internal architecture of Crossminer. Look for org.eclipse.scava.metricprovider.historic.bugs.newusers, and the list that it returns should be the correct name. We'll check from our side, it might be just a wrong name after some refactoring.

[mhow2]

I think we would need all partners to confirm that their respective part is up to date so we can close this issue

[blueoly]

I am feeling not so bright right now but I thought that the page that describes the metrics is the following -> https://scava-docs.readthedocs.io/en/latest/developers-guide/metric-provider-developement-guide/

I just came across this issue and I realised I have to put AUEB metrics here -> https://scava-docs.readthedocs.io/en/latest/user-guide/metrics/. @mhow2 Sorry for not having done this already. I was lost in the structure of the documentation.

I will be glad to help with the restructure/cleaning of the documentation after Liverpool meeting.

[creat89]

Hello @blueoly ,

Yes, you need to modify this file to be exact:

https://github.com/crossminer/scava-docs/blob/master/docs/user-guide/metrics/index.md

I have added a menu at the top, you can add as many entries are needed.

[blueoly]

Thanks @creat89 !
I realised my mistake when I saw your comment in Slack where you talked about the index and the anchors and I was trying to find where you are referring to.