Skip to content

Fix: MP test rewamp, add baseline errors and make sure that test runs correctly.#200

Merged
ninpnin merged 3 commits into
devfrom
fix_mp_test
Mar 12, 2026
Merged

Fix: MP test rewamp, add baseline errors and make sure that test runs correctly.#200
ninpnin merged 3 commits into
devfrom
fix_mp_test

Conversation

@mandlilaast
Copy link
Copy Markdown
Contributor

Summary:
The previous mp.test had several issues:

  • It did not take the full date into account (when available), which led to false negatives (people appearing in protocols but missing a mandate date prior to the protocol date).
  • It could not detect people appearing in protocols after their death date because the array storing these cases was overwritten during each protocol processing step.
  • It provided insufficient context about the affected individuals, making verification of issues more difficult.

Changes:
This pull request introduces the following improvements:

  • Baseline comparison against the current state to make deviations visible both now and in future runs. Most mp issues appear to be related to minor inaccuracies in death dates or the persons repository not being updated recently.
  • Additional contextual information for reported errors to make manual verification easier.
  • Structured logging via trainerlog using info, warning, and error levels.
  • Confidence-interval based comparison of error counts against baseline CSV files.

Some notes:
The number of errors is expected to fluctuate over time:

  • Updating the persons repository should reduce the number of errors.
  • Adding new protocol records may temporarily increase them.

All checks are executed before the test fails so that the full set of discrepancies can be inspected in a single run.

@ninpnin ninpnin merged commit f9397ee into dev Mar 12, 2026
0 of 8 checks passed
@mandlilaast mandlilaast deleted the fix_mp_test branch March 13, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants