Commit 5d1f0da
committed
feat: add aggressive kueue gating and enhanced verbose output
Replace 6 per-resource-type kueue label checks with a single
DataIntegrityCheck that validates three cluster-wide invariants:
1. Every workload in a kueue-managed namespace has the queue-name label
2. Every workload with the queue-name label is in a kueue-managed namespace
3. Within each top-level CR's ownership tree, all resources agree on the
queue-name label value
Key design decisions:
- Top-down traversal from monitored CRs (Notebook, InferenceService,
LLMInferenceService, RayCluster, RayJob, PyTorchJob) through an
ownership graph of intermediate types (Deployment, StatefulSet,
ReplicaSet, DaemonSet, Job, CronJob, Pod)
- Ownership graph built once per namespace and reused across all CRs
- Single KueueConsistency condition (True/False) with aborting impact
- First-violation-wins per CR to avoid redundant diagnostics
- ImpactedObjects populated with TypeMeta for consistency with other checks
- Pre-filter to only process kueue-relevant namespaces (union of
kueue-managed namespaces and namespaces with kueue-labeled workloads)
Introduces ImpactAborting as the highest severity level ("aborting"),
representing conditions where an upgrade MUST NOT proceed. The kueue
data-integrity check uses this impact to gate upgrades when label
inconsistencies are detected.
Fatal severity support:
- New SeverityLevelFatal filter level for --severity flag
- Fatal banner (box-drawn border) rendered above the summary table
when any aborting conditions are detected, listing all fatal findings
- Double exclamation mark (‼) status icon for fatal conditions
- Bold "fatal" severity label (other severities remain non-bold)
- FATAL verdict in output when aborting conditions are present
- FilterBySeverity updated to support fatal-only filtering
Refactors NotebookVerboseFormatter into a centrally defined
EnhancedVerboseFormatter in pkg/lint/check/:
- Handles mixed resource types by deriving per-object CRD FQN from
TypeMeta, with annotation preference for single-kind results
- Exports CRDFullyQualifiedName and DeriveCRDFQNFromTypeMeta helpers
- Supports optional per-object context via AnnotationObjectContext,
rendered as a sub-bullet beneath each impacted object reference
- Notebook checks updated to embed check.EnhancedVerboseFormatter
- Kueue DataIntegrityCheck embeds EnhancedVerboseFormatter for
CRD-qualified verbose output across its 6 monitored resource types,
with per-object context indicating which invariant was violated
Adds resource type definitions for StatefulSet, ReplicaSet, DaemonSet,
Job, and CronJob. Updates ToPartialObjectMetadata helper to preserve
UID and OwnerReferences needed for ownership graph construction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Andy Stoneberg <astonebe@redhat.com>1 parent 2bed7c2 commit 5d1f0da
37 files changed
Lines changed: 2353 additions & 2412 deletions
File tree
- pkg
- lint
- checks/workloads
- kueue
- notebook
- check
- result
- resources
- util/kube
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
19 | 25 | | |
20 | 26 | | |
21 | 27 | | |
| |||
33 | 39 | | |
34 | 40 | | |
35 | 41 | | |
36 | | - | |
| 42 | + | |
37 | 43 | | |
| 44 | + | |
38 | 45 | | |
39 | 46 | | |
40 | 47 | | |
| |||
71 | 78 | | |
72 | 79 | | |
73 | 80 | | |
74 | | - | |
| 81 | + | |
75 | 82 | | |
76 | 83 | | |
77 | 84 | | |
78 | 85 | | |
79 | 86 | | |
80 | | - | |
| 87 | + | |
81 | 88 | | |
82 | | - | |
83 | | - | |
| 89 | + | |
| 90 | + | |
84 | 91 | | |
85 | 92 | | |
86 | 93 | | |
| |||
254 | 261 | | |
255 | 262 | | |
256 | 263 | | |
| 264 | + | |
| 265 | + | |
257 | 266 | | |
258 | | - | |
| 267 | + | |
259 | 268 | | |
260 | | - | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
261 | 272 | | |
262 | 273 | | |
263 | 274 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
307 | 307 | | |
308 | 308 | | |
309 | 309 | | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
310 | 357 | | |
311 | 358 | | |
312 | 359 | | |
| |||
790 | 837 | | |
791 | 838 | | |
792 | 839 | | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
| 875 | + | |
| 876 | + | |
| 877 | + | |
| 878 | + | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
| 883 | + | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
793 | 937 | | |
794 | 938 | | |
795 | 939 | | |
| |||
0 commit comments