Skip to content

hocuspocus: Add provider/server wire-protocol integration test with CI version-skew guard - #23755

Merged
akabiru merged 3 commits into
devfrom
test/hocuspocus-provider-server-integration
Jun 16, 2026
Merged

hocuspocus: Add provider/server wire-protocol integration test with CI version-skew guard#23755
akabiru merged 3 commits into
devfrom
test/hocuspocus-provider-server-integration

Conversation

@akabiru

@akabiru akabiru commented Jun 15, 2026

Copy link
Copy Markdown
Member

https://community.openproject.org/wp/STC-834 · Follows #23754 (Hocuspocus v4.2.0 lockstep bump)

The Hocuspocus client and server live in separate dependabot ecosystems and ship as independently deployed artifacts, so they can drift apart in version on future bumps — with nothing testing that the real @hocuspocus/provider ↔ server wire protocol still holds across that drift.

This adds a failsafe on two fronts: a vitest integration test that boots the server in-process and connects a real provider, asserting the connect/auth/load/sync handshake across adjacent provider majors so the supported skew stays proven; and a soft-warning CI check that flags whenever the client and server majors diverge, since only a single major of drift is supported (two or more is not).

The check never blocks the build, and the test runs in the existing npm test with no workflow change.

@akabiru
akabiru force-pushed the test/hocuspocus-provider-server-integration branch from ae91175 to 5abea44 Compare June 15, 2026 12:46
@akabiru akabiru changed the title Test/hocuspocus: Add provider/server wire-protocol integration test hocuspocus: Add provider/server wire-protocol integration test with CI version-skew guard Jun 15, 2026
@akabiru
akabiru changed the base branch from dev to chore/hocuspocus-server-v4 June 15, 2026 12:48
@akabiru akabiru self-assigned this Jun 15, 2026
@github-actions

Copy link
Copy Markdown

Warning

Flaky specs

  • rspec ./modules/my_page/spec/features/my/work_package_table_spec.rb[1:1:1]

@akabiru
akabiru force-pushed the chore/hocuspocus-server-v4 branch from 489b07a to bb450fd Compare June 15, 2026 16:33
@akabiru
akabiru force-pushed the test/hocuspocus-provider-server-integration branch from 5abea44 to 25dec9b Compare June 15, 2026 16:56
@akabiru
akabiru force-pushed the chore/hocuspocus-server-v4 branch from 8ed13d1 to a12237e Compare June 15, 2026 17:16
@akabiru
akabiru force-pushed the test/hocuspocus-provider-server-integration branch 3 times, most recently from 43fca17 to 5d4e6d8 Compare June 15, 2026 17:49
@akabiru akabiru added this to the 17.6.x milestone Jun 15, 2026
@akabiru
akabiru marked this pull request as ready for review June 15, 2026 18:18
@akabiru
akabiru requested review from a team and Copilot June 15, 2026 18:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a safety net against future client/server Hocuspocus version drift by (1) continuously validating the real provider↔server wire-protocol handshake across an allowed one-major skew window, and (2) surfacing a non-blocking PR warning when the client/server majors diverge.

Changes:

  • Add a Vitest integration test that boots the Hocuspocus server in-process and verifies connect/auth/load/sync using both the current and previous provider major.
  • Add a GitHub Actions workflow + script that compares @hocuspocus/provider (frontend) vs @hocuspocus/server (extension) major versions and posts/removes a sticky PR warning comment on skew.
  • Add @hocuspocus/provider and a pinned “previous-major” npm alias (@hocuspocus/provider-prev) to the extension devDependencies.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
script/ci/hocuspocus_version_skew.sh Reads dependency ranges via jq, extracts majors, and exports outputs used by the PR-commenting workflow.
extensions/op-blocknote-hocuspocus/test/integration/provider-server-sync.test.ts New provider↔server handshake integration test with a two-major provider matrix and installed-version guard.
extensions/op-blocknote-hocuspocus/package.json Adds current provider plus a previous-major alias for the integration matrix.
extensions/op-blocknote-hocuspocus/package-lock.json Lockfile updates to include the added provider packages and their dependency trees.
.github/workflows/hocuspocus-version-skew.yml New workflow that runs on relevant package.json changes and posts/deletes a sticky warning comment when majors skew.
Files not reviewed (1)
  • extensions/op-blocknote-hocuspocus/package-lock.json: Generated file

Comment thread extensions/op-blocknote-hocuspocus/test/integration/provider-server-sync.test.ts Outdated
@akabiru
akabiru force-pushed the test/hocuspocus-provider-server-integration branch from 5d4e6d8 to ff65e01 Compare June 15, 2026 18:24

@brunopagno brunopagno left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job flexing the specs & CI skills

I vote for shipping it 🚢 🚢 🚢

"@eslint/js": "^9.35.0",
"@eslint/json": "^1.2.0",
"@hocuspocus/provider": "^4.2.0",
"@hocuspocus/provider-prev": "npm:@hocuspocus/provider@^3",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this line is the only thing I don't like, as it's a bit obscure to whoever is observing.

We could call it @hocuspocus/provider-test-only to make it a bit more specific. Curse JSON for not allowing comments

@akabiru akabiru Jun 16, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could call it @hocuspocus/provider-test-only to make it a bit more specific. Curse JSON for not allowing comments

Good idea- happy to rename. For context, this is only added to the dev/test bundle of extentions/op-blocknote-hocuspocus to run compatibility tests in the event the provider version is bumped before the server catches up. Given @myabc 's feedback - perhaps all this can be simplified via dependabot dependency grouping?

@myabc myabc Jun 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akabiru I've done some digging into this and it looks like Dependabot can't keep server and client in lock-step in this case.

  • What it can do: group updates for a dependency of the same name across multiple manifests (introduced relatively recently – see blog article). Unfortunately this doesn't help us since @hocuspocus/provider lives in /frontend and @hocuspocus/server in /extensions/op-blocknote-hocuspocus, i.e. different names in different dirs.
  • What it can't do: one PR for the whole @hocuspocus/* group across both directories.

That said, I've created a PR to consolidate Dependabot npm entries and add a grouping for @hocuspocus/*. It's mostly a DRY-ness improvement; it also couples the server packages (@hocuspocus/server + extension-logger) within the extension's PR. Client and server still get separate PRs, each whenever its own update lands.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Experiment here: #23783

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Alex, as per our chat- will look into whether re-using/extending the existing script #23755 (review) is feasible (before merge)

@akabiru
akabiru requested a review from myabc June 16, 2026 14:06
@akabiru
akabiru force-pushed the chore/hocuspocus-server-v4 branch from a12237e to 3d9bfd7 Compare June 16, 2026 16:25

@myabc myabc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cbliard added a script a while back to help keep Primer version in lock-step. At the time I remember suggesting something rather over-engineered (reading package-lock.json and Gemfile.lock by their respective Bundler APIs). It may be that this simple shell script also does the trick.

script/check_same_primer_view_components_version_everywhere.

Comment on lines +41 to +42
PROVIDER_RANGE=$(jq -r '.dependencies["@hocuspocus/provider"] // empty' frontend/package.json)
SERVER_RANGE=$(jq -r '.dependencies["@hocuspocus/server"] // empty' extensions/op-blocknote-hocuspocus/package.json)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably moot, but should we parse package.json or package-lock.json? script/check_same_primer_view_components_version_everywheredoes the latter.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@myabc
myabc self-requested a review June 16, 2026 17:02
Base automatically changed from chore/hocuspocus-server-v4 to dev June 16, 2026 17:26
akabiru added 2 commits June 16, 2026 20:33
The provider (frontend) and server (extension) live in separate dependabot
ecosystems and bump independently, so they can silently drift across a major.
A soft-warning sticky PR comment surfaces a major mismatch for review without
blocking, since Hocuspocus supports a one-major skew in both directions.
The existing extension specs exercise the server hooks with a mocked API but
never a real client, so nothing proves the @hocuspocus/provider actually speaks
the wire protocol to the server. This boots the server in-process and connects a
real provider, asserting the connect/authenticate/load/sync handshake for both
the current and previous provider majors — making the one-major skew the
version-skew guard tolerates a standing assertion.
@akabiru
akabiru force-pushed the test/hocuspocus-provider-server-integration branch from ff65e01 to 761efb2 Compare June 16, 2026 17:33
The npm alias was @hocuspocus/provider-prev; JSON can't carry a comment saying it exists solely to pin the previous major for the skew test. Renamed to @hocuspocus/provider-prev-test-only so the intent is on the key.
@akabiru

akabiru commented Jun 16, 2026

Copy link
Copy Markdown
Member Author

@cbliard added a script a while back to help keep Primer version in lock-step. At the time I remember suggesting something rather over-engineered (reading package-lock.json and Gemfile.lock by their respective Bundler APIs). It may be that this simple shell script also does the trick.

script/check_same_primer_view_components_version_everywhere.

I had a look and the main difference is that for hocuspocus we compare only majors which would be identical in package.json and package-lock.json; the primer script is comparing exact versions across files which is not directly needed here.

@github-actions

Copy link
Copy Markdown

Warning

Flaky specs

  • rspec ./spec/features/projects/create_spec.rb[1:12:3:2:1]
  • rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:2]
🤖 Ask Copilot to investigate

Copy the prompt below into a new comment on this PR to delegate the investigation to GitHub Copilot. It will look into the flakiness and open a separate pull request with you as reviewer.

@copilot The following spec(s) were detected as flaky on PR #23755:

- `rspec ./spec/features/projects/create_spec.rb[1:12:3:2:1]`
- `rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:2]`

Follow the playbook in docs/development/testing/handling-flaky-tests/README.md to investigate the root cause and apply a fix. Open a separate pull request with the change and request a review from @akabiru.

@akabiru

akabiru commented Jun 16, 2026

Copy link
Copy Markdown
Member Author

@copilot The following spec(s) were detected as flaky on PR #23755:

  • rspec ./spec/features/projects/create_spec.rb[1:12:3:2:1]
  • rspec ./spec/features/work_packages/table/switch_types_spec.rb[1:1:2]

Follow the playbook in docs/development/testing/handling-flaky-tests/README.md to investigate the root cause and apply a fix. Open a separate pull request with the change and request a review from @akabiru.

Copilot AI commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

@akabiru I've opened a new pull request, #23784, to work on those changes. Once the pull request is ready, I'll request review from you.

@akabiru
akabiru merged commit cd46d9a into dev Jun 16, 2026
17 checks passed
@akabiru
akabiru deleted the test/hocuspocus-provider-server-integration branch June 16, 2026 18:36
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Development

Successfully merging this pull request may close these issues.

5 participants