Merge branch 'main' into sglang

Author: sirutBuasai

.gitignore

@@ -8,5 +8,6 @@ __pycache__
 
 # Generated docs
 docs/tutorials
+# docs/releasenotes/*/
 docs/reference/available_images.md
 docs/reference/support_policy.md

.kiro/steering/docs.md

@@ -10,20 +10,29 @@ Within this documentations page, website navigation through `.nav.yml` file shou
 
 ## Documentation Generation System
 
-The documentation uses an automatic generation system for `available_images.md` and `support_policy.md`.
+The documentation uses an automatic generation system for `available_images.md`, `support_policy.md`, and release notes.
 
 ### Directory Structure
 
 ```
 docs/src/
-├── templates/reference/           # Jinja2 templates
-│   ├── available_images.template.md
-│   └── support_policy.template.md
+├── templates/
+│   ├── reference/                 # Reference page templates
+│   │   ├── available_images.template.md
+│   │   └── support_policy.template.md
+│   └── releasenotes/              # Release notes templates
+│       ├── release_note.template.md
+│       └── index.template.md
 ├── tables/                        # Table column configs (one per repository)
+│   ├── extra/                     # Special table configs for generated pages
+│   │   ├── support_policy.yml     # Columns for support_policy.md
+│   │   └── release_notes.yml      # Columns for release notes index pages
 │   ├── pytorch-training.yml
 │   ├── pytorch-inference.yml
 │   └── ...
 ├── data/                          # Image configs (one file per image)
+│   ├── template/
+│   │   └── image-template.yml     # Template with all possible fields documented
 │   ├── pytorch-training/
 │   │   ├── 2.9-gpu-ec2.yml
 │   │   ├── 2.9-cpu-sagemaker.yml
@@ -47,11 +56,11 @@ docs/src/
 
 ### File Responsibilities
 
-- `constants.py` - Path constants, global variables, and `GLOBAL_CONFIG`
-- `sorter.py` - Sorting tiebreaker functions: `platform_sorter`, `accelerator_sorter`
-- `utils.py` - Utility functions: `load_yaml()`, `load_table_config()`, `load_jinja2()`, `render_table()`, `write_output()`, `parse_version()`, `clone_git_repository()`, `build_public_registry_note()`, `get_framework_order()`
-- `image_config.py` - `ImageConfig` class, image loaders (`load_repository_images`, `load_legacy_images`), `sort_by_version`, `get_latest_image`, `build_image_row`, `check_public_registry`
-- `generate.py` - `generate_support_policy()`, `generate_available_images()`, `generate_all()`
+- `constants.py` - Path constants, global variables, `GLOBAL_CONFIG`, and `RELEASE_NOTES_REQUIRED_FIELDS`
+- `sorter.py` - Sorting tiebreaker functions: `platform_sorter`, `accelerator_sorter`, `repository_sorter`
+- `utils.py` - Utility functions: `load_yaml()`, `load_table_config()`, `load_jinja2()`, `render_table()`, `write_output()`, `parse_version()`, `clone_git_repository()`, `build_ecr_uri()`, `build_public_ecr_uri()`, `get_framework_order()`
+- `image_config.py` - `ImageConfig` class, image loaders (`load_repository_images`, `load_legacy_images`, `load_images_by_framework_group`), `sort_by_version`, `get_latest_image_uri`, `build_image_row`, `check_public_registry`
+- `generate.py` - `generate_support_policy()`, `generate_available_images()`, `generate_release_notes()`, `generate_all()`
 - `macros.py` - MkDocs macros plugin integration
 - `hooks.py` - MkDocs hooks entry point
 
@@ -104,15 +113,21 @@ img.get("cuda", "-")  # "cu130" or "-" if not present
 # Properties (no parentheses)
 img.is_supported  # True if eop >= today
 img.has_support_dates  # True if ga and eop fields exist
+img.has_release_notes  # True if all RELEASE_NOTES_REQUIRED_FIELDS are present
 img.display_repository  # "PyTorch Training" (uses GLOBAL_CONFIG)
 img.display_framework_group  # "PyTorch" (uses GLOBAL_CONFIG)
+img.display_tag  # First tag from tags list (used in available_images.md)
+img.release_note_filename  # "<repo>-<version>-<accelerator>-<platform>.md"
 
 # Display properties for table rendering
 img.display_framework_version  # "PyTorch 2.9"
-img.display_example_url  # formatted ECR URL
+img.display_example_url  # formatted ECR URL (uses display_tag)
 img.display_platform  # "EC2, ECS, EKS" (mapped from "ec2")
 img.display_accelerator  # "GPU" (mapped from "gpu")
 
+# Methods
+img.get_image_uris()  # List of image URIs (private ECR + public ECR if public_registry=true)
+
 # Generic display accessor (uses display_<field> property if exists, else raw value)
 img.get_display("platform")  # "EC2, ECS, EKS"
 img.get_display("python")  # "py312" (no display_ property, returns raw)
@@ -176,7 +191,9 @@ table = render_table(headers, rows)
    accelerator: gpu             # gpu, cpu, or neuronx
    python: py312                # Python version
    platform: ec2                # ec2 or sagemaker
-   tag: "2.9.0-gpu-py312-cu130-ubuntu22.04-ec2"  # Full image tag
+   # Image tags (first tag is used in available_images.md)
+   tags:
+     - "2.9.0-gpu-py312-cu130-ubuntu22.04-ec2"
    ```
 
 1. Optional fields:
@@ -188,11 +205,82 @@ table = render_table(headers, rows)
    public_registry: true        # If in ECR Public Gallery
    ga: "2025-10-15"             # GA date (for support policy)
    eop: "2026-10-15"            # EOP date (for support policy)
-   ecr_account: "007439368137"  # Override default ECR account
+   example_ecr_account: "007439368137"  # Override default ECR account
    ```
 
 1. GA/EOP dates are only needed for repositories that appear in support policy (base, pytorch-*, tensorflow-* excluding neuronx).
 
+### Adding Release Notes to an Image
+
+To generate release notes for an image, add the following fields to the image config YAML:
+
+```yaml
+# Required for release notes generation (both must be present)
+announcement:
+  - "Introduced containers for PyTorch 2.9 for Training"
+  - "Added Python 3.12 support"
+
+packages:
+  python: "3.12"
+  pytorch: "2.9.0"
+  cuda: "13.0"
+  cudnn: "9.13.0.50"
+  nccl: "2.27.7-1"
+
+# Optional sections (rendered dynamically at end of release notes)
+optional:
+  known_issues:
+    - "Description of known issue"
+```
+
+**Required fields for release notes** are defined in `constants.py`:
+
+```python
+RELEASE_NOTES_REQUIRED_FIELDS = ["announcement", "packages"]
+```
+
+The `packages` field uses keys that map to display names in `global.yml` under `display_names`.
+
+### Release Notes Generation
+
+Release notes are automatically generated for images that have all required fields (`announcement` and `packages`).
+
+**Output structure:**
+
+```
+docs/releasenotes/
+├── index.md                    # Main release notes index
+├── <framework_group>/          # e.g., pytorch/, tensorflow/, base/
+│   ├── index.md                # Framework-specific index with table of links
+│   └── <repo>-<ver>-<acc>-<plat>.md  # Individual release notes
+└── archive/                    # Historical manual release notes
+    ├── index.md
+    ├── pytorch/
+    ├── tensorflow/
+    └── ...
+```
+
+**Framework index page structure:**
+
+The framework-specific `index.md` uses table rendering (like `available_images.md` and `support_policy.md`) with columns defined in `tables/extra/release_notes.yml`:
+
+- Repository (displayed as Framework), Accelerator, Platform, Link
+
+The index separates release notes into two sections:
+
+1. **Supported images** - Images where `eop >= today` or no EOP date is set
+1. **Deprecated images** - Images where `eop < today`, displayed in a warning admonition
+
+This separation uses the `ImageConfig.is_supported` property.
+
+**Generated release note sections:**
+
+1. **Announcement** - Bullet list from `announcement` field
+1. **Core Packages** - Table from `packages` field (keys mapped via `display_names`)
+1. **Security Advisory** - Hardcoded section with link to AWS Security Bulletin
+1. **Reference** - Docker image URIs (private ECR + public ECR if `public_registry: true`) and links to available_images.md and support_policy.md
+1. **Known Issues** (optional) - Bullet list from `known_issues` field
+
 ### Adding a New Repository
 
 1. Create directory: `docs/src/data/<repository-name>/`
@@ -205,9 +293,6 @@ table = render_table(headers, rows)
        header: "Framework"
      - field: python
        header: "Python"
-     - field: framework
-       data: framework_group  # optional: use different data source for display
-       header: "Framework"
      # ... add columns in desired order
    ```
 
@@ -227,18 +312,27 @@ amazon: "Amazon"
 dlc: "Deep Learning Containers"
 
 # Composed terminology (resolved at load time)
-dlc_long: "${aws} ${dlc}"        # -> "AWS Deep Learning Containers"
-sagemaker: "${amazon} SageMaker" # -> "Amazon SageMaker"
+dlc_long: "${aws} ${dlc}"            # -> "AWS Deep Learning Containers"
+sagemaker: "${amazon} SageMaker AI"  # -> "Amazon SageMaker AI"
 
 # Platform display mappings
 platforms:
   ec2: "EC2, ECS, EKS"
   sagemaker: "SageMaker"
 
-# Repository display names (required for all repositories)
+# Consolidated display names (repositories, frameworks, packages, optional sections)
 display_names:
+  # Framework groups
+  pytorch: "PyTorch"
+  tensorflow: "TensorFlow"
+  # Repositories
   pytorch-training: "PyTorch Training"
-  # ...
+  # Packages (for release notes Core Packages table)
+  python: "Python"
+  cuda: "CUDA"
+  cudnn: "cuDNN"
+  # Optional sections
+  known_issues: "Known Issues"
 
 # Framework groups for support policy consolidation (lowercase keys)
 framework_groups:
@@ -292,6 +386,11 @@ cd /path/to/deep-learning-containers
 source .venv/bin/activate
 cd docs/src && python main.py --verbose
 
+# Generate specific outputs
+python main.py --available-images-only
+python main.py --support-policy-only
+python main.py --release-notes-only
+
 # With MkDocs (automatic via hooks)
 mkdocs serve
 mkdocs build
@@ -336,4 +435,7 @@ pytorch:
 ## Update knowledge base
 
 If there are any new changes to the documentations generation and organization, make sure to update you knowledge base in the steering/docs.md file and any runbook or update to processes should also be updated in DEVELOPMENT.md files.
-This is done so that developers get the most up-to-date information on the current codebase.
+This is done so that developers get the most up-to-date information on the current codebase. Be sure to not let this steering document get too large since it will overflow the context window.
+If the document gets longer than 500 lines, make sure to delete unnecessary sections and condense verbose sections where necessary. Also, do not delete comments unless the it is unnecessary.
+
+When making changes to image config field reading or processing, update `docs/src/data/template/image-template.yml` to reflect those changes.

.pre-commit-config.yaml

@@ -104,7 +104,7 @@ repos:
         language: python
         additional_dependencies: [flowmark]
         files: ^docs/
-        exclude: (\.template\.md$|/\.venv/)
+        exclude: (\.template\.md$|(README|DEVELOPMENT)\.md$)
         types: [markdown]
 
   - repo: https://github.com/hukkin/mdformat