Skip to content

Use dataset weights when aggregating IQB requirement scores #177

@sonikagutha

Description

@sonikagutha

While going through the IQBCalculator implementation, I noticed that the requirement agreement score (RAS) is currently computed as a simple average over the dataset scores.
However, each dataset in IQB_CONFIG has an associated weight (w). Since the current aggregation does not appear to use these weights, all datasets effectively contribute equally to the final RAS value.
This could potentially lead to misleading IQB scores when multiple datasets are enabled, as providers with higher configured weight would not influence the result proportionally.

suggested approach:
The changes would be made in ( library/src/iqb/calculator.py )
1)Collect dataset scores (ds_scores) along with their corresponding weights (ds_weights).
2)Compute this as weighted average [ ras = sum(weighted_scores) / sum(ds_weights) ]
3)Handle the case where no dataset has a positive weight (either by raising an exception or failing gracefully).

Please let me know If this aligns with the expected behavior, I’d be happy to work on a PR to implement the change. @sermpezis , @bassosimone
Thank you

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions