Skip to content

Release: the npm changelog backport can silently attribute unshipped changes to a published version #81506

Description

@sirreal

The npm release backport uses git cherry-pick to replay a changelog-rewriting commit onto trunk. When trunk has moved during the publish, git resolves the overlap textually. Sometimes that produces a conflict, which is fine, because it stops. Sometimes it merges cleanly and produces a changelog that is wrong, which is not fine, because nothing stops.

The August 12 release did both at once, and then did it again during the manual recovery. I would like to draw attention to the second half, because the first half is what everyone looks at and it is the less interesting problem.

What happened

The release run published 121 packages, pushed wp/latest, and pushed all package tags. It then failed cherry-picking e18aa00 ("Update changelog files") onto trunk, with a conflict in packages/env/CHANGELOG.md. #81447 had landed on trunk a few hours earlier and added an entry under ## Unreleased, which is exactly where the release commit wants to insert the ## 11.13.0 (2026-08-12) heading.

That conflict is the visible failure, and it is the only reason anyone looked closer.

Two other changelogs had diverged in the same window, and git merged both without complaint. The #81361 entry landed under ## 39.0.0 (2026-08-12) in components; the #81449 entry landed under ## 18.0.0 (2026-08-12) in dataviews. Neither commit is an ancestor of the publish commit. Neither change is in the published tarballs. Both were filed under a version heading claiming they shipped in a release that does not contain them.

Had env been one of the packages that shipped changes in this release, nothing would have conflicted at all, the push would have gone through, and we would have published a false changelog with no error anywhere. That is not a hypothetical — see the mechanism below, which makes it the expected outcome rather than the unlucky one.

Then it happened again

The recovery, #81485 (landed in d36bccf), was a hand-driven cherry-pick of the same two commits. It hit the same problem, and worse, because four more changelog-touching PRs had landed on trunk in the meantime. Nine entries across four packages ended up inside published version sections:

File Version heading Entries
packages/block-editor/CHANGELOG.md 16.2.0 #80747
packages/block-library/CHANGELOG.md 10.4.0 #81292
packages/components/CHANGELOG.md 39.0.0 #81361, #81449, #81450, #81451
packages/dataviews/CHANGELOG.md 18.0.0 #81449, #81450, #81451

None of those six PRs are ancestors of 989764d. ## Unreleased was left empty in all four files. #81502 moves the entries back.

packages/env/CHANGELOG.md came out correct both times, because it is the file that conflicted, so it was the only one anybody had to think about.

The whole thing fits in one diff. Every package tag from this release points at the publish commit, so any of them anchors it against the backport:

git fetch --tags
git diff 989764d42ff d36bccf8ca2 -- 'packages/*/CHANGELOG.md'
 packages/block-editor/CHANGELOG.md  | 1 +
 packages/block-library/CHANGELOG.md | 1 +
 packages/components/CHANGELOG.md    | 4 ++++
 packages/dataviews/CHANGELOG.md     | 3 +++
 packages/env/CHANGELOG.md           | 4 ++++
 5 files changed, 13 insertions(+)

Thirteen added lines, nothing removed. Read the hunk context rather than the additions: env is the only file whose hunk starts at ## Unreleased. The other four splice new entries into ### Bug Fixes and ### Internal blocks sitting at lines 13 through 55, deep inside a version section that was tagged and published minutes earlier.

Full diff
diff --git a/packages/block-library/CHANGELOG.md b/packages/block-library/CHANGELOG.md
@@ -22,6 +22,7 @@
 -   Footnotes: Treat unreadable `footnotes` post meta as no footnotes instead of throwing, so the block shows its placeholder rather than the block crash warning ([#81201](https://github.com/WordPress/gutenberg/pull/81201)).
 -   Playlist: Improve handling of declarative waveform player configuration ([#81342](https://github.com/WordPress/gutenberg/pull/81342)).
 -   Cover: Pass `'full'` instead of `null` as the featured image size for parallax and repeated backgrounds, so a null array offset is no longer reached on PHP 8.5 ([#81444](https://github.com/WordPress/gutenberg/pull/81444)).
+-   Icon: Apply only padding to the inner SVG in the editor, so margin is no longer applied twice compared to the front end ([#81292](https://github.com/WordPress/gutenberg/pull/81292)).

 ### Internal

diff --git a/packages/dataviews/CHANGELOG.md b/packages/dataviews/CHANGELOG.md
@@ -13,8 +13,11 @@
 -   DataViews: Replace the inlined `kebabCase` utility with the new `@wordpress/kebab-case` package. [#81294](https://github.com/WordPress/gutenberg/pull/81294)
 -   DataForm: Internalize `ValidatedSelectControl` and its `ControlWithError` foundation instead of unlocking them from the `@wordpress/components` private APIs. The foundation is a temporary copy slated to be replaced by the upcoming `@wordpress/ui` implementation. [#81391](https://github.com/WordPress/gutenberg/pull/81391)
 -   DataForm: Internalize `ValidatedCheckboxControl` instead of unlocking it from the `@wordpress/components` private APIs. [#81435](https://github.com/WordPress/gutenberg/pull/81435)
+-   DataForm: Internalize `ValidatedComboboxControl` instead of unlocking it from the `@wordpress/components` private APIs. [#81449](https://github.com/WordPress/gutenberg/pull/81449)
 -   DataForm: Internalize `ValidatedNumberControl` instead of unlocking it from the `@wordpress/components` private APIs. [#81433](https://github.com/WordPress/gutenberg/pull/81433)
 -   DataForm: Internalize `ValidatedRadioControl` instead of unlocking it from the `@wordpress/components` private APIs. [#81434](https://github.com/WordPress/gutenberg/pull/81434)
+-   DataForm: Internalize `ValidatedFormTokenField` instead of unlocking it from the `@wordpress/components` private APIs. [#81451](https://github.com/WordPress/gutenberg/pull/81451)
+-   DataForm: Internalize `ValidatedToggleGroupControl` instead of unlocking it from the `@wordpress/components` private APIs. [#81450](https://github.com/WordPress/gutenberg/pull/81450)

 ### New Features

diff --git a/packages/env/CHANGELOG.md b/packages/env/CHANGELOG.md
@@ -2,6 +2,10 @@

 ## Unreleased

+### Bug Fixes
+
+-   Update git sources to the latest commit when `--update` is passed. Previously, a source pointing at a branch (such as `"core": "WordPress/WordPress"`) stayed at the commit it was first cloned at, no matter how many times it was updated.
+
 ## 11.13.0 (2026-08-12)

block-editor and components are the same shape and are trimmed here for length.

To be clear about the count: nine of those thirteen lines are the defect. The four in env are correct — that is the file that conflicted, and it is the control case.

That is the shape of the problem in one sentence: the files git refuses to merge are fine, and the files it merges quietly are wrong. Careful people do not catch this, because there is nothing to catch it with.

Why it happens

updatePackages decides which entries belong to which version, on wp/latest, before publication. That decision is frozen into a text diff. Ten minutes later, after npm publication finishes, backportCommitsToBranch replays that diff onto trunk:

https://github.com/WordPress/gutenberg/blob/trunk/tools/release/commands/packages.js#L1132

for ( const commitHash of commits ) {
    await repo.raw( 'cherry-pick', commitHash );
}

await repo.push( 'origin', branchName );

A three-way merge of a markdown list has no idea what "this entry shipped" means. It knows line positions. If a trunk-side insertion is far enough from the heading insertion, the hunks do not overlap and git does the reasonable textual thing, which is the wrong semantic thing.

The race window is the entire npm publication, roughly ten minutes on a branch that people are actively merging into.

There is no check after the cherry-pick. It pushes whatever came out.

The conflict fires on exactly the wrong packages

It is tempting to read the two failures as bad luck — five changelogs diverged during the second attempt and only one of them conflicted. It is not luck, and the pattern is worse than random.

The release commit's edit is a single insertion of ## X.Y.Z (date) immediately below ## Unreleased. A trunk-side entry is an insertion into a ### <subsection> underneath that heading. Whether git conflicts comes down to whether those two insertions land within its three-line context window:

  • If ## Unreleased was non-empty at release time, the subsection the new entry belongs to is already many lines below the heading. The hunks do not overlap, git merges cleanly, and the entry silently ends up under the published version.
  • If ## Unreleased was empty, the trunk-side entry is inserted at the same line as the version heading. Hunks overlap, conflict, someone has to look at it.

An empty ## Unreleased at release time means the package shipped nothing in that release. So the alarm only ever fires for packages with no changes in the release, and packages that actually shipped something are corrupted quietly, every time.

Measured on both attempts, with dbb73c9be7e as the cherry-pick base:

Package Lines under ## Unreleased at base Outcome
env 0 conflict, resolved correctly both times
block-library 15 silent, wrong
block-editor 24 silent, wrong
dataviews 26 silent, wrong
components 36 silent, wrong

Three diverged on the first attempt (one conflict, two silent), five on the second (one conflict, four silent). In both cases the single conflict was env, the only package in the set with an empty ## Unreleased. This is deterministic, not probabilistic, and it means the failure rate is effectively 100% for any package that both shipped in the release and picked up a changelog entry during the publish window.

Suggested fixes

Ranked by how much they help, not by how easy they are.

1. Recompute instead of replay. The set of entries that shipped is exactly the ## Unreleased content at e18aa00bb4f^, and we know it at backport time. Rather than cherry-picking a diff, finalize could apply the transform directly to trunk: insert ## X.Y.Z (date) below anything trunk has added since that base. This is deterministic and does not care how long publication took or what landed meanwhile. Everything else here narrows the window; only this closes it.

2. Assert the post-condition before pushing. For every package, after the backport, the released portion of the changelog must be byte-identical to what was published:

diff <( git show "$PUBLISH_COMMIT:$f" | awk '/^## [0-9]/{p=1} p' ) \
     <( git show "HEAD:$f"            | awk '/^## [0-9]/{p=1} p' )

Anything else means an entry moved into or out of a shipped version. This takes about a second across all 125 changelogs and would have caught both failures.

Two notes on writing it. It has to run inside the release tooling, not only in CI, because the backport is a direct push to trunk and never becomes a PR. And it must fail loudly when it inspects zero files — I got this wrong on the first pass here with a git ls-tree pathspec that quietly matched nothing and reported success, which is a fair illustration of how this class of check goes bad.

We already have the underlying idea in the tree: .github/workflows/check-package-changelogs.yml (lines 115-132) asserts that a PR's entry lives in the ## Unreleased section. It simply never runs on the one commit that reorganises every changelog in the repository.

3. Do not let one branch failure cancel the other.

https://github.com/WordPress/gutenberg/blob/trunk/tools/release/commands/packages.js#L1088

await backportCommitsToBranchFn( 'trunk', commits, config );
if ( config.releaseType === 'latest' && pluginReleaseBranch ) {
    await backportCommitsToBranchFn( pluginReleaseBranch, commits, config );
}

Trunk threw, so release/23.8 never got its backport, even though those same two commits apply to it with no conflict at all. Two independent operations, one shared failure. Catching per branch and reporting both outcomes would have halved the cleanup.

4. Give recovery a command. cli.js has four publish commands and nothing for "npm is done, please finish the git side." Every occurrence is hand-driven git against a runbook, and #81485 is what that produces even when the person driving knows exactly what to watch for. A finalize subcommand taking the two commit SHAs would turn a careful afternoon into a command, and would carry the check from (2) with it.

On the work already in flight

#79906 (open, waiting on #80202) splits prepare/publish/finalize into separate jobs so finalize can be re-run on a fresh runner. That is a good change and I am not arguing against it. It does not help here: re-running finalize replays the same cherry-pick and hits the same conflict, or worse, the same silent merge.

More generally, the July hardening series — #79904, #79905, #80187, #80334 — went after npm registry interaction: version verification, dist-tags, gitHead, propagation retries. That side has been holding up fine. Every failure we have actually had is on the git side, and none of those PRs touched it. #80187 says as much in its own description.

Frequency

Automated Update changelog files commits stop appearing on trunk after 2026-07-14. Both releases since then needed manual recovery: July 29 (#81303, #81304) and August 12 (#81480, #81481, #81485, #81502). There is an earlier cluster around June 30 to July 2 (#79702, #79816, #79821) that looks related, though I have not been through those in detail.

Two consecutive cycles requiring hand-repair, and one of those repairs itself needing a repair, suggests this is the normal outcome now rather than bad luck.

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions