Skip to content

Commit 999fd95

Browse files
authored
feat: add automatic release notes generation feature (#5602)
* update data Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * udpdate data Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * add gitignore Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * refactor release note gen Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * update developers.md Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * update developers.md Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * update developers.md Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * update developers.md Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * remove external URL Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * format release notes template Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * revert pt 2.7 Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * add cuda test Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * revert index.md Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * move support policy table Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * symlink support policy table Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * symlink extra tables Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * fix test Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * add release notes test Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * revert release notes Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * revert release notes Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * revert data Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> * revert nav Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com> --------- Signed-off-by: sirutBuasai <sirutbuasai27@outlook.com>
1 parent e69f112 commit 999fd95

28 files changed

+1152
-542
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,5 +8,6 @@ __pycache__
88

99
# Generated docs
1010
docs/tutorials
11+
# docs/releasenotes/*/
1112
docs/reference/available_images.md
1213
docs/reference/support_policy.md

.kiro/steering/docs.md

Lines changed: 122 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -10,20 +10,29 @@ Within this documentations page, website navigation through `.nav.yml` file shou
1010

1111
## Documentation Generation System
1212

13-
The documentation uses an automatic generation system for `available_images.md` and `support_policy.md`.
13+
The documentation uses an automatic generation system for `available_images.md`, `support_policy.md`, and release notes.
1414

1515
### Directory Structure
1616

1717
```
1818
docs/src/
19-
├── templates/reference/ # Jinja2 templates
20-
│ ├── available_images.template.md
21-
│ └── support_policy.template.md
19+
├── templates/
20+
│ ├── reference/ # Reference page templates
21+
│ │ ├── available_images.template.md
22+
│ │ └── support_policy.template.md
23+
│ └── releasenotes/ # Release notes templates
24+
│ ├── release_note.template.md
25+
│ └── index.template.md
2226
├── tables/ # Table column configs (one per repository)
27+
│ ├── extra/ # Special table configs for generated pages
28+
│ │ ├── support_policy.yml # Columns for support_policy.md
29+
│ │ └── release_notes.yml # Columns for release notes index pages
2330
│ ├── pytorch-training.yml
2431
│ ├── pytorch-inference.yml
2532
│ └── ...
2633
├── data/ # Image configs (one file per image)
34+
│ ├── template/
35+
│ │ └── image-template.yml # Template with all possible fields documented
2736
│ ├── pytorch-training/
2837
│ │ ├── 2.9-gpu-ec2.yml
2938
│ │ ├── 2.9-cpu-sagemaker.yml
@@ -47,11 +56,11 @@ docs/src/
4756

4857
### File Responsibilities
4958

50-
- `constants.py` - Path constants, global variables, and `GLOBAL_CONFIG`
51-
- `sorter.py` - Sorting tiebreaker functions: `platform_sorter`, `accelerator_sorter`
52-
- `utils.py` - Utility functions: `load_yaml()`, `load_table_config()`, `load_jinja2()`, `render_table()`, `write_output()`, `parse_version()`, `clone_git_repository()`, `build_public_registry_note()`, `get_framework_order()`
53-
- `image_config.py` - `ImageConfig` class, image loaders (`load_repository_images`, `load_legacy_images`), `sort_by_version`, `get_latest_image`, `build_image_row`, `check_public_registry`
54-
- `generate.py` - `generate_support_policy()`, `generate_available_images()`, `generate_all()`
59+
- `constants.py` - Path constants, global variables, `GLOBAL_CONFIG`, and `RELEASE_NOTES_REQUIRED_FIELDS`
60+
- `sorter.py` - Sorting tiebreaker functions: `platform_sorter`, `accelerator_sorter`, `repository_sorter`
61+
- `utils.py` - Utility functions: `load_yaml()`, `load_table_config()`, `load_jinja2()`, `render_table()`, `write_output()`, `parse_version()`, `clone_git_repository()`, `build_ecr_uri()`, `build_public_ecr_uri()`, `get_framework_order()`
62+
- `image_config.py` - `ImageConfig` class, image loaders (`load_repository_images`, `load_legacy_images`, `load_images_by_framework_group`), `sort_by_version`, `get_latest_image_uri`, `build_image_row`, `check_public_registry`
63+
- `generate.py` - `generate_support_policy()`, `generate_available_images()`, `generate_release_notes()`, `generate_all()`
5564
- `macros.py` - MkDocs macros plugin integration
5665
- `hooks.py` - MkDocs hooks entry point
5766

@@ -104,15 +113,21 @@ img.get("cuda", "-") # "cu130" or "-" if not present
104113
# Properties (no parentheses)
105114
img.is_supported # True if eop >= today
106115
img.has_support_dates # True if ga and eop fields exist
116+
img.has_release_notes # True if all RELEASE_NOTES_REQUIRED_FIELDS are present
107117
img.display_repository # "PyTorch Training" (uses GLOBAL_CONFIG)
108118
img.display_framework_group # "PyTorch" (uses GLOBAL_CONFIG)
119+
img.display_tag # First tag from tags list (used in available_images.md)
120+
img.release_note_filename # "<repo>-<version>-<accelerator>-<platform>.md"
109121

110122
# Display properties for table rendering
111123
img.display_framework_version # "PyTorch 2.9"
112-
img.display_example_url # formatted ECR URL
124+
img.display_example_url # formatted ECR URL (uses display_tag)
113125
img.display_platform # "EC2, ECS, EKS" (mapped from "ec2")
114126
img.display_accelerator # "GPU" (mapped from "gpu")
115127

128+
# Methods
129+
img.get_image_uris() # List of image URIs (private ECR + public ECR if public_registry=true)
130+
116131
# Generic display accessor (uses display_<field> property if exists, else raw value)
117132
img.get_display("platform") # "EC2, ECS, EKS"
118133
img.get_display("python") # "py312" (no display_ property, returns raw)
@@ -176,7 +191,9 @@ table = render_table(headers, rows)
176191
accelerator: gpu # gpu, cpu, or neuronx
177192
python: py312 # Python version
178193
platform: ec2 # ec2 or sagemaker
179-
tag: "2.9.0-gpu-py312-cu130-ubuntu22.04-ec2" # Full image tag
194+
# Image tags (first tag is used in available_images.md)
195+
tags:
196+
- "2.9.0-gpu-py312-cu130-ubuntu22.04-ec2"
180197
```
181198
182199
1. Optional fields:
@@ -188,11 +205,82 @@ table = render_table(headers, rows)
188205
public_registry: true # If in ECR Public Gallery
189206
ga: "2025-10-15" # GA date (for support policy)
190207
eop: "2026-10-15" # EOP date (for support policy)
191-
ecr_account: "007439368137" # Override default ECR account
208+
example_ecr_account: "007439368137" # Override default ECR account
192209
```
193210
194211
1. GA/EOP dates are only needed for repositories that appear in support policy (base, pytorch-*, tensorflow-* excluding neuronx).
195212
213+
### Adding Release Notes to an Image
214+
215+
To generate release notes for an image, add the following fields to the image config YAML:
216+
217+
```yaml
218+
# Required for release notes generation (both must be present)
219+
announcement:
220+
- "Introduced containers for PyTorch 2.9 for Training"
221+
- "Added Python 3.12 support"
222+
223+
packages:
224+
python: "3.12"
225+
pytorch: "2.9.0"
226+
cuda: "13.0"
227+
cudnn: "9.13.0.50"
228+
nccl: "2.27.7-1"
229+
230+
# Optional sections (rendered dynamically at end of release notes)
231+
optional:
232+
known_issues:
233+
- "Description of known issue"
234+
```
235+
236+
**Required fields for release notes** are defined in `constants.py`:
237+
238+
```python
239+
RELEASE_NOTES_REQUIRED_FIELDS = ["announcement", "packages"]
240+
```
241+
242+
The `packages` field uses keys that map to display names in `global.yml` under `display_names`.
243+
244+
### Release Notes Generation
245+
246+
Release notes are automatically generated for images that have all required fields (`announcement` and `packages`).
247+
248+
**Output structure:**
249+
250+
```
251+
docs/releasenotes/
252+
├── index.md # Main release notes index
253+
├── <framework_group>/ # e.g., pytorch/, tensorflow/, base/
254+
│ ├── index.md # Framework-specific index with table of links
255+
│ └── <repo>-<ver>-<acc>-<plat>.md # Individual release notes
256+
└── archive/ # Historical manual release notes
257+
├── index.md
258+
├── pytorch/
259+
├── tensorflow/
260+
└── ...
261+
```
262+
263+
**Framework index page structure:**
264+
265+
The framework-specific `index.md` uses table rendering (like `available_images.md` and `support_policy.md`) with columns defined in `tables/extra/release_notes.yml`:
266+
267+
- Repository (displayed as Framework), Accelerator, Platform, Link
268+
269+
The index separates release notes into two sections:
270+
271+
1. **Supported images** - Images where `eop >= today` or no EOP date is set
272+
1. **Deprecated images** - Images where `eop < today`, displayed in a warning admonition
273+
274+
This separation uses the `ImageConfig.is_supported` property.
275+
276+
**Generated release note sections:**
277+
278+
1. **Announcement** - Bullet list from `announcement` field
279+
1. **Core Packages** - Table from `packages` field (keys mapped via `display_names`)
280+
1. **Security Advisory** - Hardcoded section with link to AWS Security Bulletin
281+
1. **Reference** - Docker image URIs (private ECR + public ECR if `public_registry: true`) and links to available_images.md and support_policy.md
282+
1. **Known Issues** (optional) - Bullet list from `known_issues` field
283+
196284
### Adding a New Repository
197285
198286
1. Create directory: `docs/src/data/<repository-name>/`
@@ -205,9 +293,6 @@ table = render_table(headers, rows)
205293
header: "Framework"
206294
- field: python
207295
header: "Python"
208-
- field: framework
209-
data: framework_group # optional: use different data source for display
210-
header: "Framework"
211296
# ... add columns in desired order
212297
```
213298

@@ -227,18 +312,27 @@ amazon: "Amazon"
227312
dlc: "Deep Learning Containers"
228313

229314
# Composed terminology (resolved at load time)
230-
dlc_long: "${aws} ${dlc}" # -> "AWS Deep Learning Containers"
231-
sagemaker: "${amazon} SageMaker" # -> "Amazon SageMaker"
315+
dlc_long: "${aws} ${dlc}" # -> "AWS Deep Learning Containers"
316+
sagemaker: "${amazon} SageMaker AI" # -> "Amazon SageMaker AI"
232317

233318
# Platform display mappings
234319
platforms:
235320
ec2: "EC2, ECS, EKS"
236321
sagemaker: "SageMaker"
237322

238-
# Repository display names (required for all repositories)
323+
# Consolidated display names (repositories, frameworks, packages, optional sections)
239324
display_names:
325+
# Framework groups
326+
pytorch: "PyTorch"
327+
tensorflow: "TensorFlow"
328+
# Repositories
240329
pytorch-training: "PyTorch Training"
241-
# ...
330+
# Packages (for release notes Core Packages table)
331+
python: "Python"
332+
cuda: "CUDA"
333+
cudnn: "cuDNN"
334+
# Optional sections
335+
known_issues: "Known Issues"
242336

243337
# Framework groups for support policy consolidation (lowercase keys)
244338
framework_groups:
@@ -292,6 +386,11 @@ cd /path/to/deep-learning-containers
292386
source .venv/bin/activate
293387
cd docs/src && python main.py --verbose
294388

389+
# Generate specific outputs
390+
python main.py --available-images-only
391+
python main.py --support-policy-only
392+
python main.py --release-notes-only
393+
295394
# With MkDocs (automatic via hooks)
296395
mkdocs serve
297396
mkdocs build
@@ -336,4 +435,7 @@ pytorch:
336435
## Update knowledge base
337436

338437
If there are any new changes to the documentations generation and organization, make sure to update you knowledge base in the steering/docs.md file and any runbook or update to processes should also be updated in DEVELOPMENT.md files.
339-
This is done so that developers get the most up-to-date information on the current codebase.
438+
This is done so that developers get the most up-to-date information on the current codebase. Be sure to not let this steering document get too large since it will overflow the context window.
439+
If the document gets longer than 500 lines, make sure to delete unnecessary sections and condense verbose sections where necessary. Also, do not delete comments unless the it is unnecessary.
440+
441+
When making changes to image config field reading or processing, update `docs/src/data/template/image-template.yml` to reflect those changes.

.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ repos:
104104
language: python
105105
additional_dependencies: [flowmark]
106106
files: ^docs/
107-
exclude: (\.template\.md$|/\.venv/)
107+
exclude: (\.template\.md$|(README|DEVELOPMENT)\.md$)
108108
types: [markdown]
109109

110110
- repo: https://github.com/hukkin/mdformat

0 commit comments

Comments
 (0)