Skip to content

Commit eb9f916

Browse files
committed
refactor support policy grouping
Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com>
1 parent 70a2130 commit eb9f916

File tree

10 files changed

+134
-56
lines changed

10 files changed

+134
-56
lines changed

.kiro/steering/docs.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -376,18 +376,22 @@ table_order:
376376

377377
### Support Policy Consolidation
378378

379-
The `framework_groups` configuration consolidates support policy rows by framework. Repositories in the same group are combined into a single row using the framework name (e.g., "PyTorch").
379+
The `framework_groups` configuration consolidates support policy rows by framework. The generation follows a 3-step flow:
380380

381-
**Version Display:**
381+
**Validation (at load time):** `load_repository_images` validates that images sharing the same full version within a single repository have identical GA/EOP dates. Raises `ValueError` if not (this is a data bug).
382382

383-
- Images with the same major.minor version (e.g., `2.6.0` and `2.6.1`) are consolidated into a single row displayed as `2.6` if they have identical GA/EOP dates
384-
- If patch versions have different GA/EOP dates, each is displayed separately with full version (e.g., `2.6.0`, `2.6.1`) and a warning is logged
383+
**Step 1 — Group by full version:** All images in a framework group are grouped by full version (e.g., `2.6.0`), deduplicated per repository (one representative image per repo since intra-repo consistency is guaranteed).
384+
385+
**Step 2 — Cross-repo agreement check:** For each full version, check if all repositories agree on GA/EOP dates:
386+
387+
- If all repositories agree → one entry using the framework group name (e.g., "PyTorch")
388+
- If repositories disagree → warning logged, each repository gets its own row using its individual display name (e.g., "PyTorch Inference")
389+
390+
**Step 3 — Major.minor collapse:** Non-split entries are grouped by major.minor. If all full versions within a major.minor share the same dates, they collapse into a single row displayed as the major.minor (e.g., `2.6`). Collapse is skipped for any major.minor that has split (per-repo) rows.
385391

386392
**Requirements:**
387393

388-
- All repositories in a group that have a given full version (X.Y.Z) must have identical GA/EOP dates
389394
- Missing versions in some repositories are allowed (only present repos are consolidated)
390-
- A `ValueError` is raised if dates differ within a group for the same full version
391395

392396
To add a new framework group, add an entry to `framework_groups` with the framework name as key and list of repositories as value.
393397

docs/src/data/pytorch-inference-arm64/2.6-cpu-ec2.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ python: py312
55
os: ubuntu22.04
66
platform: ec2
77
ga: "2025-01-29"
8-
eop: "2026-01-29"
8+
eop: "2026-06-29"
99
public_registry: true
1010

1111
tags:

docs/src/data/pytorch-inference-arm64/2.6-cpu-sagemaker.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ python: py312
55
os: ubuntu22.04
66
platform: sagemaker
77
ga: "2025-01-29"
8-
eop: "2026-01-29"
8+
eop: "2026-06-29"
99
public_registry: true
1010

1111
tags:

docs/src/data/pytorch-inference-arm64/2.6-gpu-ec2.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ cuda: cu124
66
os: ubuntu22.04
77
platform: ec2
88
ga: "2025-01-29"
9-
eop: "2026-01-29"
9+
eop: "2026-06-29"
1010
public_registry: true
1111

1212
tags:

docs/src/data/pytorch-inference/2.6-cpu-ec2.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ python: py312
55
os: ubuntu22.04
66
platform: ec2
77
ga: "2025-01-29"
8-
eop: "2026-01-29"
8+
eop: "2026-06-29"
99
public_registry: true
1010

1111
tags:

docs/src/data/pytorch-inference/2.6-cpu-sagemaker.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ python: py312
55
os: ubuntu22.04
66
platform: sagemaker
77
ga: "2025-01-29"
8-
eop: "2026-01-29"
8+
eop: "2026-06-29"
99
public_registry: true
1010

1111
tags:

docs/src/data/pytorch-inference/2.6-gpu-ec2.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ cuda: cu124
66
os: ubuntu22.04
77
platform: ec2
88
ga: "2025-01-29"
9-
eop: "2026-01-29"
9+
eop: "2026-06-29"
1010
public_registry: true
1111

1212
tags:

docs/src/data/pytorch-inference/2.6-gpu-sagemaker.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ cuda: cu124
66
os: ubuntu22.04
77
platform: sagemaker
88
ga: "2025-01-29"
9-
eop: "2026-01-29"
9+
eop: "2026-06-29"
1010
public_registry: true
1111

1212
tags:

docs/src/generate.py

Lines changed: 99 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414

1515
import logging
1616
from pathlib import Path
17+
from pprint import pformat
1718

1819
import sorter as sorter_module
1920
from constants import (
@@ -176,6 +177,55 @@ def generate_release_notes(dry_run: bool = False) -> None:
176177
LOGGER.info("Generated release notes")
177178

178179

180+
def _collapse_minor_versions(
181+
entries: list[tuple[ImageConfig, dict[str, str]]],
182+
) -> list[tuple[ImageConfig, dict[str, str]]]:
183+
"""Collapse patch versions (e.g., A.B.C, A.B.D) into major.minor (A.B) when all share identical dates.
184+
185+
Skips any major.minor that has split (per-repo) rows, since mixing collapsed and split rows
186+
under the same major.minor would be confusing.
187+
188+
Args:
189+
entries: List of (image, overrides) tuples. Split rows have "framework_group" in overrides.
190+
191+
Returns:
192+
New list with collapsible groups replaced by a single major.minor entry.
193+
"""
194+
uncollapsible: set[str] = set()
195+
collapsible_groups: dict[str, list[int]] = {}
196+
for idx, (img, overrides) in enumerate(entries):
197+
version_obj = parse_version(img.version)
198+
mm = f"{version_obj.major}.{version_obj.minor}"
199+
if "framework_group" in overrides:
200+
# Find major.minors that have split rows by repository — these cannot be collapsed
201+
# Collapsing split rows will create ambiguity between patch versions
202+
uncollapsible.add(mm)
203+
else:
204+
collapsible_groups.setdefault(mm, []).append(idx)
205+
206+
# Collapse: if all entries in a major.minor group share dates, keep one with major.minor display
207+
for mm, indices in collapsible_groups.items():
208+
if mm in uncollapsible:
209+
continue
210+
group_imgs = [entries[idx][0] for idx in indices]
211+
ref_img = group_imgs[0]
212+
if all(cmp_img.ga == ref_img.ga and cmp_img.eop == ref_img.eop for cmp_img in group_imgs):
213+
entries[indices[0]] = (ref_img, {"version": mm})
214+
for idx in indices[1:]:
215+
entries[idx] = None # mark duplicates for removal
216+
else:
217+
# Log images that have different ga/eop dates
218+
for cmp_img in group_imgs[1:]:
219+
if cmp_img.ga != ref_img.ga or cmp_img.eop != ref_img.eop:
220+
LOGGER.warning(
221+
f"Cannot collapse {mm}: "
222+
f"{ref_img._repository} {ref_img.version} ({ref_img.ga}, {ref_img.eop}) vs "
223+
f"{cmp_img._repository} {cmp_img.version} ({cmp_img.ga}, {cmp_img.eop})"
224+
)
225+
226+
return [e for e in entries if e is not None]
227+
228+
179229
def generate_support_policy(dry_run: bool = False) -> str:
180230
"""Generate support_policy.md from image configs with GA/EOP dates."""
181231
output_path = REFERENCE_DIR / "support_policy.md"
@@ -197,66 +247,73 @@ def generate_support_policy(dry_run: bool = False) -> str:
197247
if not images:
198248
continue
199249

200-
# Group by major.minor, then decide display format based on date consistency
201-
major_minor_groups: dict[str, list[ImageConfig]] = {}
250+
# Step 1: Group by full version, deduplicate per repo
251+
# Result: {"2.6": [pt-training img, pt-inference img, ...], "2.7": [...]}
252+
version_entries: dict[str, list[ImageConfig]] = {}
202253
for img in images:
203-
v = parse_version(img.version)
204-
major_minor = f"{v.major}.{v.minor}"
205-
major_minor_groups.setdefault(major_minor, []).append(img)
206-
207-
version_map: dict[str, ImageConfig] = {}
208-
for major_minor, group in major_minor_groups.items():
209-
# Check if all images in group have same ga/eop
210-
first = group[0]
211-
all_same_dates = all(img.ga == first.ga and img.eop == first.eop for img in group)
212-
213-
if all_same_dates:
214-
# Consolidate to major.minor display
215-
version_map[major_minor] = first
254+
bucket = version_entries.setdefault(img.version, [])
255+
if not any(existing._repository == img._repository for existing in bucket):
256+
bucket.append(img)
257+
LOGGER.debug(
258+
"[%s] Step 1 - versions:\n%s",
259+
framework_group,
260+
pformat({v: [i._repository for i in imgs] for v, imgs in version_entries.items()}),
261+
)
262+
263+
# Step 2: Cross-repo agreement check
264+
entries: list[tuple[ImageConfig, dict[str, str]]] = []
265+
seen_versions: set[str] = set()
266+
for full_ver, repo_imgs in version_entries.items():
267+
seen_versions.add(full_ver)
268+
ref_img = repo_imgs[0]
269+
if all(
270+
cmp_img.ga == ref_img.ga and cmp_img.eop == ref_img.eop for cmp_img in repo_imgs
271+
):
272+
entries.append((ref_img, {"version": full_ver}))
216273
else:
217-
# Keep full versions, warn about inconsistency
218-
versions_info = ", ".join(f"{img.version} ({img.ga}, {img.eop})" for img in group)
219274
LOGGER.warning(
220-
f"Different GA/EOP dates for {framework_group} patch versions: {versions_info}"
275+
f"GA/EOP mismatch in {framework_group} {full_ver} across repositories. "
276+
f"Splitting into individual repository rows."
221277
)
222-
# Keep each patch version as separate row with full version display
223-
for img in group:
224-
existing = version_map.get(img.version)
225-
# Error if same full version (e.g., X.Y.Z) has different dates across images
226-
if existing and (existing.ga != img.ga or existing.eop != img.eop):
227-
raise ValueError(
228-
f"Inconsistent dates for {framework_group} {img.version}: \n"
229-
f"\tExisting: {existing._repository}-{existing.version}-{existing.accelerator}-{existing.platform}\n"
230-
f"\tImage: {img._repository}-{img.version}-{img.accelerator}-{img.platform}\n"
231-
f"\t({existing.ga}, {existing.eop}) vs ({img.ga}, {img.eop})"
232-
)
233-
# Deduplicate same full version with same dates
234-
version_map[img.version] = img
278+
for img in repo_imgs:
279+
entries.append(
280+
(img, {"version": full_ver, "framework_group": img.display_repository})
281+
)
282+
LOGGER.debug(
283+
"[%s] Step 2 - entries:\n%s",
284+
framework_group,
285+
pformat([(img._repository, overrides) for img, overrides in entries]),
286+
)
287+
288+
# Step 3: Collapse patch versions into major.minor where possible
289+
# Result: entries with "version" collapsed (e.g., "2.7.0" -> "2.7"), split rows unchanged
290+
entries = _collapse_minor_versions(entries)
291+
LOGGER.debug(
292+
"[%s] Step 3 - collapsed:\n%s",
293+
framework_group,
294+
pformat([(img._repository, overrides) for img, overrides in entries]),
295+
)
235296

236297
# Merge legacy entries for this framework
237298
for legacy_img in legacy_data.get(framework_group, []):
238-
if legacy_img.version not in version_map:
239-
version_map[legacy_img.version] = legacy_img
299+
if legacy_img.version not in seen_versions:
300+
entries.append((legacy_img, {"version": legacy_img.version}))
240301

241302
# Sort by version descending within this framework group
242-
# Key is the display version (major.minor if consolidated, full version otherwise)
243-
sorted_keys = sorted(
244-
version_map.keys(), key=lambda k: parse_version(version_map[k].version), reverse=True
245-
)
246-
for key in sorted_keys:
247-
img = version_map[key]
248-
(supported if img.is_supported else unsupported).append((img, key))
303+
entries.sort(key=lambda e: parse_version(e[0].version), reverse=True)
304+
for img, overrides in entries:
305+
(supported if img.is_supported else unsupported).append((img, overrides))
249306

250307
# Build tables
251308
table_config = load_table_config("extra/support_policy")
252309
columns = table_config.get("columns", [])
253310
headers = [col["header"] for col in columns]
254311

255312
supported_table = render_table(
256-
headers, [build_image_row(img, columns, {"version": ver}) for img, ver in supported]
313+
headers, [build_image_row(img, columns, overrides) for img, overrides in supported]
257314
)
258315
unsupported_table = render_table(
259-
headers, [build_image_row(img, columns, {"version": ver}) for img, ver in unsupported]
316+
headers, [build_image_row(img, columns, overrides) for img, overrides in unsupported]
260317
)
261318

262319
# Render template

docs/src/image_config.py

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,24 @@ def load_repository_images(repository: str) -> list[ImageConfig]:
185185
repo_dir = DATA_DIR / repository
186186
if not repo_dir.exists():
187187
return []
188-
return [ImageConfig.from_yaml(f, repository) for f in sorted(repo_dir.glob("*.yml"))]
188+
images = [ImageConfig.from_yaml(f, repository) for f in sorted(repo_dir.glob("*.yml"))]
189+
190+
# Validate: images in the same repository sharing same full version must have identical GA/EOP dates
191+
date_by_version: dict[str, ImageConfig] = {}
192+
for img in images:
193+
if not img.has_support_dates:
194+
continue
195+
if img.version in date_by_version:
196+
ref = date_by_version[img.version]
197+
if ref.ga != img.ga or ref.eop != img.eop:
198+
raise ValueError(
199+
f"Inconsistent dates within {repository} for version {img.version}: "
200+
f"({ref.ga}, {ref.eop}) vs ({img.ga}, {img.eop})"
201+
)
202+
else:
203+
date_by_version[img.version] = img
204+
205+
return images
189206

190207

191208
def load_images_by_framework_group(

0 commit comments

Comments
 (0)