Project Health Reporting (take 2)

[maoo]

Follow up of #125

The following health metrics have been built for the CDM project, using the following LFX Insights reports:

CDM (manual) report:

From Velocity (last quarter) - https://insights.lfx.linuxfoundation.org/foundation/finos/velocity?project=isda-cdm&routedFrom=Github&dateFilters=Last%2012%20Months&dateRange=2023-07-26%20to%202024-07-25&compare=PP&granularity=month&hideBots=true

• Download Performance Metrics CSV
• Download Lead Time CSV
• Download Wait Time for 1st Review CSV

From Productivity (last quarter) - https://insights.lfx.linuxfoundation.org/foundation/finos/productivity?project=isda-cdm&routedFrom=Github&dateFilters=Last%20Quarter&dateRange=2024-04-01%20to%202024-06-30&compare=PP&granularity=week&hideBots=true&repository=all

• Download Commits per Active Day CSV
• Download New Contributors CSV

From Reports > Contributors (last quarter) - https://insights.lfx.linuxfoundation.org/foundation/finos/reports/contributors?project=isda-cdm&routedFrom=Github&dateFilters=Last%20Quarter&dateRange=2024-04-01%20to%202024-06-30&compare=PP&granularity=week&hideBots=true&repository=all

• Download Total Contributors CSV
• Download Monthly Active Contributors CSV

From Reports > Activities (last quarter) - https://insights.lfx.linuxfoundation.org/foundation/finos/reports/activities?project=isda-cdm&routedFrom=Github&dateFilters=Last%20Quarter&dateRange=2024-04-01%20to%202024-06-30&compare=PP&granularity=week&hideBots=true&repository=all

• Download Total Activities CSV
• Download Monthly Activities CSV (Warning! This returns an empty CSV)

Still unclear how to use Reports > Retention on https://insights.lfx.linuxfoundation.org/foundation/finos/reports/retention?project=isda-cdm&routedFrom=Github&dateFilters=Last%2012%20Months&dateRange=2023-07-26%20to%202024-07-25&compare=PP&granularity=month&hideBots=true

Attaching spreadsheet with one sheet per CSV export - CDM metrics (May-July 2024).xlsx

As next steps:

• @maoo to work with LFX team to understand how to automate the download of all the CSV files for each project and how to iterate across all FINOS hosted projects
• @finos/toc to review metrics above and provide feedback
• @finos/toc to update the Jupyter Notebook accordingly and build the board report on project health

For all security aspects related with project healthcheck, we're considering to adopt/contribute to the OSPS Baseline, currently led by @eddie-knight

[eddie-knight]

During today's TOC call, @psmulovics commented that this is a great improvement from #125, but has distinct room for improvement.

@psmulovics Can you please elaborate on the gaps / room for improvement?

Both @chris-twiner and @caradelia have accepted the call for feedback, and intend to comment here before the next TOC call.

[psmulovics]

@psmulovics Can you please elaborate on the gaps / room for improvement?

Sure. Due to the data being 'human quantified' instead of 'scientific quantified', we do have entries saying '1.25K' instead of '1250' which makes it hard to display, needs manual intervention, etc. Same way, some of the data types, like dates, get mistakenly converted to other format, like '7/1/2024 12:00:00 AM' becoming '1.71T' in the report. Also, there are missing spots in the report, eg data that is missing. Also, we need to have better understanding for reports on what 'total activities', 'retention', etc actually means.

[maoo]

@psmulovics Can you please elaborate on the gaps / room for improvement?

Sure. Due to the data being 'human quantified' instead of 'scientific quantified', we do have entries saying '1.25K' instead of '1250' which makes it hard to display, needs manual intervention, etc. Same way, some of the data types, like dates, get mistakenly converted to other format, like '7/1/2024 12:00:00 AM' becoming '1.71T' in the report. Also, there are missing spots in the report, eg data that is missing. Also, we need to have better understanding for reports on what 'total activities', 'retention', etc actually means.

Thanks for the valuable feedback @psmulovics and thanks @eddie-knight for gathering feedback.

I agree on 'human' VS 'scientific' quantified data, especially on data exports.

Regarding the 'missing spots', it would be great to have more details on which metrics are missing.

Regarding the understanding for 'total activities', 'retention', etc , I'd ask you to use the LFX Insights UI tips, as shown below. Thank you!

[mcderk]

Thank you for the feedback. I work on LFX as head of product & design. I can add a backlog item to see if we can make our downloads more human friendly. Also, we can investigate the use case around automating the send of the report as a future feature.

Please see this documentation on reports - https://docs.linuxfoundation.org/lfx/insights/v3-beta-version-current/reports that should clarify to provide better understanding.

[eddie-knight]

We have now added explicit health ("maintenance") requirements in #233. While we haven't completely ironed out the measurement mechanisms, there are things like the Baseline GitHub Action and the various parts of LFX Insights which will help with evaluations against the new policy.

I'm closing this issue as complete, and we can tackle any measurement or enforcement complexities as standalone issues when they arise.

@maoo please reach out if you're aware of anything we haven't yet considered!