Skip to content

Testgroup verdicts, expectations

Open
No due date
Last updated Apr 21, 2024
75% complete

For type: pass-fail

  • Apply {input, output}_validator_flags from testdata.yaml to the respective testcase when validating
  • Collect testcase verdicts, infer testdata verdicts for each testgroup
  • Update UI to report testgroup verdicts

For type: scoring:

  • Allow testcase verdicts to have numerical score
  • Apply grading flags from testdata.yaml to respective testcase scores
  • Apply aggregation rules from grading in testdata.yaml
  • Update UI to report aggregate scores for testgroups

Expectations:

  • Read and parse expectations.yaml
  • Compare testcase verdicts with expectations
  • Compare testgroup verdicts with expectations

List view

  • Status: Draft (not ready).
    RagnarGrootKoerkamp/BAPCtoolsnumber 307
    #307 In RagnarGrootKoerkamp/BAPCtools;
    · thorehusfeldt opened on Sep 16, 2023