chore: Reduce pod scheduling error log spam with deduplication

[moko-poi]

Description

Implements log deduplication for pod scheduling errors to address excessive logging when pods cannot be scheduled.

Problem

Fixes #1904

When pods cannot be scheduled (e.g., due to missing NodePool, taint intolerance, or resource constraints), the scheduler logs an error on every scheduling attempt. This results in 100k+ log entries per hour for a single unschedulable pod, causing:

• Log storage bloat
• Difficulty in identifying actual issues
• Performance overhead from excessive I/O

Solution

Added cache-based log deduplication using a 5-minute timeout window:

• Logs are deduplicated per pod UID + error message combination
• Different errors for the same pod are logged separately
• Same error is logged again after 5 minutes if still occurring
• Matches the existing event deduplication timeout for consistency

Changes

• Modified pkg/controllers/provisioning/scheduling/scheduler.go:
  • Added logErrorCache to Results struct
  • Implemented shouldLogPodError() method with composite cache key
  • Integrated cache check before logging scheduling errors
• Added pkg/controllers/provisioning/scheduling/log_deduplication_test.go:
  • Comprehensive test coverage for deduplication scenarios
  • Validates different pods, same errors, different errors, and timeout expiration

Testing

• All existing tests pass (336/337 specs)
• New deduplication tests validate correct behavior
• Verified different error types for same pod are logged separately
• Confirmed timeout allows re-logging after 5 minutes

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: moko-poi
Once this PR has been reviewed and has the lgtm label, please assign derekfrank for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @moko-poi. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[coveralls]

Pull Request Test Coverage Report for Build 20206128003

Details

• 17 of 24 (70.83%) changed or added relevant lines in 1 file are covered.
• 11 unchanged lines in 2 files lost coverage.
• Overall coverage decreased (-0.08%) to 80.25%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/controllers/provisioning/scheduling/scheduler.go	17	24	70.83%

Files with Coverage Reduction	New Missed Lines	%
pkg/controllers/disruption/consolidation.go	4	88.0%
pkg/controllers/provisioning/scheduling/preferences.go	7	88.76%

Totals
Change from base Build 19948193551:	-0.08%
Covered Lines:	11869
Relevant Lines:	14790

---

💛 - Coveralls