Tide should force retest a suspiciously passing required job on mergeable PRs #541
Description
This is a companion issue to #540, where status-reconciler component went haywire and started falsely retiring job contexts on in-flight pull requests.
If Tide considers a PR to be mergeable (passes required jobs, has required labels etc), it forces a retest of any required Prow job that was executed with base ref different than the current HEAD of the base branch. We need to make sure Tide forces a retest also in the case where the passing job result (github context status) matching the job required for merges is "suspiciously" green, not coming from either a passing Prowjob or from an /override invocation. This way, Tide would protect the system from the impact of bugs like #540 : haywire status-reconciler falsely marked a job a passing, but Tide should know the job is required, and should only respect the passing result if it came from an actual passing prow job.
I have not verified how exactly Tide behaves today, I heard that during #540 incident it was not retesting the falsely retired jobs unfortunately.
/area tide
/kind feature