Skip to content

Release-notes agent: improve coverage so foundational features aren't missed (Preview 5 retrospective) #10435

@danroth27

Description

@danroth27

Summary

The .NET 11 Preview 5 release-notes generation missed two foundational Blazor features that shipped under area-blazor + feature-validation, plus one borderline OpenAPI fix. The misses surfaced only when validating the draft by building working samples against the Preview 5 build. Filing this to capture concrete improvements to the release-notes skills so a future preview's draft catches them automatically.

What was missed

  • dotnet/aspnetcore#66526 — Async form validation (AddValidationTask, IsValidationPending, IsValidationFaulted). No milestone tag.
  • dotnet/aspnetcore#66646 — Validation resource-based localization (AddValidationLocalization<T>()). Labeled area-infrastructure, easy to skip in a Blazor-component slice.
  • dotnet/aspnetcore#65650 — MVC + OpenAPI multiple Produces per status code preserved. No milestone tag.

All three were present in changes.json with full metadata. The input data was fine; the editorial pipeline filtered them out.

Root causes

  1. generate-features editorial-scoring stage was skipped. Stub features.json was written and sub-agents got slim {id, title, url, labels} JSON, bypassing the scoring rubric the pipeline expects.
  2. review-release-notes multi-model QA pass was skipped. That pass is designed precisely to catch under-covered clusters.
  3. Label-shaped blind spots in sub-agent slices. #66646 is area-infrastructure, not area-blazor. Sub-agents handed one component slice drift toward "matches my area label" PRs and miss adjacent ones.
  4. PR titles don't always include the marquee API name. #66526's title says "async form validation" — none of AddValidationTask / IsValidationPending / IsValidationFaulted appear. Title-keyword ranking under-rates these.
  5. No milestone cross-check. A simple gh api /search/issues milestone:11.0-preview<N> sweep against the draft would have surfaced #66526 and #66646 immediately. Not part of the current workflow.

Proposed skill changes

  1. release-notes/SKILL.md — add a mandatory milestone cross-check step: pull the full PR list for the milestone and reconcile against the draft's covered PR set before the QA pass.
  2. generate-features/SKILL.md — add explicit cluster-enumeration guidance: when a cluster (e.g., "Blazor validation") is identified, enumerate every PR in changes.json whose title, body, or labels intersect that cluster before scoring.
  3. feature-scoring.md — document the label-myopia anti-pattern: cross-cutting features often live under area-infrastructure, area-mvc, etc., not the obvious component area.
  4. release-notes/SKILL.md — require sub-agents receive the full feature JSON (description, scoring rationale, body excerpt), not a slim {id, title, url, labels} projection.

Validation lesson (orthogonal to the skill changes)

Building working samples for each drafted feature against the actual preview build flushes missing APIs out almost immediately: a sample that needs AddValidationLocalization<T>() or EditContext.AddValidationTask fails to compile when those features aren't documented yet. Consider building this in as a draft validation step.

What was already fixed in Preview 5

The three misses were patched directly into the Preview 5 draft (PR #10422, commits 8e76e667 + 9d52bc6d). This issue tracks the workflow-level fixes so Preview 6 doesn't repeat the pattern.

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-area-labelNo area label was automatically applied

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions