Allow for recreation of pods once fully terminated in the job controller

[kannon92]

Enhancement Description

• One-line enhancement description (can be used as a release note): For Job controllers we want to optionally include a way for users to include terminating pods in the active pod count.

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-apps/3939-allow-replacement-when-fully-terminated

• Discussion Link: Jobs create replacement Pods as soon as a Pod is marked for deletion kubernetes#115844

• Primary contact (assignee): @kannon92, @dejanzele

• Responsible SIGs: sig-apps

• Enhancement target (which target equals to which milestone):

  • Alpha release target (x.y): 1.28
  • Beta release target (x.y): 1.29
  • Beta terminating fixes release target (x.y): 1.31
  • Stable release target (x.y): 1.33

• Alpha

  • KEP (k/enhancements) update PR(s): KEP-3939: Consider terminating pods in job controller #3940
  • Code (k/k) update PR(s):
    • Job API - add podReplacementPolicy and terminating field to job api kubernetes#119301
    • Job Controller Changes - Job: create replacement pods only after terminated kubernetes#117015
    • PodGC - KEP-3939: pod gc changes for pod replacement policy kep kubernetes#118772
  • Docs (k/website) update PR(s):
    • Job Section - Add information about PodReplacementPolicy in Job API website#41745
    • Alpha Blog Post - Blog post on new Job features in 1.28 (PodReplacementPolicy and BackoffLimitPerIndex) website#41924

• Beta

  • KEP (k/enhancements) update PR(s): KEP-3939: promote to beta in 1.29. #4252
  • Code (k/k) update PR(s):
    • Add a missing integration test for PodReplacementPolicy kubernetes#119912
    • Fix panic when enablement of pod replacement policy is skewed kubernetes#121011
    • Remove terminating count from rmAtLeast kubernetes#121147
    • cleanup: improve assertions for Failed PodReplacementPolicy integration test cases kubernetes#121274
    • Fix tracking of terminating Pods when nothing else changes kubernetes#121342
    • Metrics addition: feat: add job_pods_creation_total metric kubernetes#121481
    • Feature Enable and E2E Tests: Switch feature flag to beta for pod replacement policy and add e2e test kubernetes#121491
  • Docs (k/website) update(s):
    • Graduate JobPodReplacementPolicy to beta website#43529

• Beta (1.31)

  • KEP (k/enhancements) update PR(s): Update the managedBy KEP with fix for terminating pods #4657
  • Code (k/k) update PR(s):
    • Count terminating pods when deleting active pods for failed jobs kubernetes#125175
    • Delay setting terminal Job conditions until all pods are terminal kubernetes#125510
  • Docs (k/website) update(s):
    • Update the docs for JobManagedBy and JobPodReplacementPolicy related to pod termination website#46808

• Stable

  • KEP (k/enhancements) update PR(s): KEP-3939: Job Pod Replacement Policy; promote to GA #5168
  • Code (k/k) update PR(s):
  • Docs (k/website) update(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[kannon92]

/sig apps

[wangyysde]

/cc @wangyysde

[alculquicondor]

/assign
I'm taking over the KEP while @kannon92 is on PTO

[soltysh]

/stage alpha
/milestone v1.28
/label lead-opted-in