[Bug]: Validator keeps checking approval state of removed block

[Lederstrumpf]

Bug Summary

Error occurs when approval state fails on a block

Bug Description

1. Approval state fails on a block (in the example log, https://polkadot.js.org/apps/#/explorer/query/27560937, where the proposed failed block's hash is 0xd324…bcdf)
2. Block gets removed from storage when another block gets finalized at the same height (in the example log, 0x03b1…ced5 gets finalized)
3. Despite block getting removed, main_runner keeps reporting that its approval state is failed (ApprovalDistribution Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7)

Steps to Reproduce

Start node as validator, but also with --parachain-single-process (done as mitigation for #2402).

Effects of the Bug

Potential consequence is: Warning ClusterTracker Cluster has too many pending statements, something wrong with our connection to our group peers. Restart might be needed if validator gets 0 backing rewards for more than 3-4 consecutive sessions

Filtered log excerpt:

Mar 19 22:02:37 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:37.790430  worker.13        Info      BlockStorage  Added block #27560937 (0xd324…bcdf) as child of #27560936 (0xbb0b…67a6)
Mar 19 22:02:37 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:37.813737  main_runner      Info      BlockExecutor  Imported block #27560937 (0xd324…bcdf) within 63 ms. (lag 1813 ms.)
Mar 19 22:02:47 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:47.156684  main_runner      Info      ParachainProcessorImpl  Starting validation task.(para id=2105, relay parent=0xd324…bcdf, candidate_hash=0xbe80…23bf)
Mar 19 22:02:47 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:47.415988  main_runner      Info      ParachainProcessorImpl  Async validation complete.(relay parent=0xd324…bcdf, para_id=2105)
Mar 19 22:02:47 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:47.416108  main_runner      Info      ParachainProcessorImpl  Send Seconded to collator.(peer=…CxD1ex, relay parent=0xd324…bcdf)
Mar 19 22:02:48 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:48.418824  main_runner      Error     Synchronizer  Can't complete blocks loading from …F9VMZJ starting from block #27560937 (0xd324…bcdf): Received block is not descendant of previous
Mar 19 22:02:52 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:52.766441  main_runner      Error     Synchronizer  Can't complete blocks loading from …kB4BbP starting from block #27560937 (0xd324…bcdf): Received block is not descendant of previous
Mar 19 22:02:59 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:59.377554  main_runner      Info      BlockStorage  Removed block #27560937 (0xd324…bcdf)
Mar 19 22:02:59 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:59.683435  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:02:59 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:02:59.756680  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:03:08 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:03:08.036489  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:03:21 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:03:21.725348  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:03:21 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:03:21.973925  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:03:37 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:03:37.859584  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x71e5…61e0
Mar 19 22:03:39 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:03:39.283999  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:03:44 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:03:44.513526  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x71e5…61e0
Mar 19 22:03:50 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:03:50.783943  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x71e5…61e0
Mar 19 22:04:07 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:04:07.628117  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:04:08 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:04:08.798636  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:04:34 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:04:34.972646  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:04:36 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:04:36.938111  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:04:53 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:04:53.804910  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:04:54 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:04:54.727059  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x843c…6df7
Mar 19 22:05:04 oneprovider-tyo-692269 kagome[130748]: 25.03.19 22:05:04.740403  main_runner      Warning   ApprovalDistribution  Approval state is failed. Block hash 0xd324…bcdf, session index 46516, validator index 206, relay parent 0x71e5…61e0

Full relevant log excerpt attached

kagobob-block-header-not-found-excerpt.log

Expected Behavior

Block should no longer be checked for approval once removed from storage?

System Information

NixOS 24.11 with kernel 6.13.7
Compiler: gcc 13.2.0
CMake: cmake version 3.25.3

Additional Context

No response