Skip to content

test(e2e): add deterministic offline update-detection smoke#209

Merged
ryota-murakami merged 2 commits into
mainfrom
feat/e2e-update-detection-smoke
Jun 13, 2026
Merged

test(e2e): add deterministic offline update-detection smoke#209
ryota-murakami merged 2 commits into
mainfrom
feat/e2e-update-detection-smoke

Conversation

@ryota-murakami

@ryota-murakami ryota-murakami commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Why

v0.23.0 release closeout could confirm the public latest-mac.yml + asset URLs, but live old-version update detection timed out (net::ERR_TIMED_OUT) from a temp-launched app. This adds a deterministic, offline smoke so update detection is provable without depending on GitHub/Electron net.

What

A new Electron e2e spec — e2e/spec/update-detection.e2e.ts — proves an app at currentVersion 0.0.1 observes a feed advertising 99.0.0 as available, end-to-end through the real electron-updater → IPC → Redux chain.

Determinism / offline / safety (by construction)

  • Localhost feed only: an ephemeral-port (listen(0)) http server serves a controlled latest-mac.yml (v99.0.0). provider: 'generic', never the GitHub provider.
  • Detection-only: autoDownload = false → the dummy artifact is never fetched; downloadUpdate/quitAndInstall are never called.
  • Isolated: runs under a tempdir HOME; never touches the real HOME, userData, or /Applications.
  • Production-dead seam: initAutoUpdaterForE2E is reached only when E2E_UPDATE_FEED_URL is set (injected by the spec, never in production). The production gate app.isPackaged && E2E_DISABLE_UPDATE !== '1' is preserved byte-for-byte in the else-branch.

Production changes (minimal seam + tidy refactor)

  • src/main/updater.ts: extracted the 6 autoUpdater.on(...) handlers into a private registerUpdaterEventHandlers() (verbatim, no behavior change) shared by both entry points; 3000UPDATE_CHECK_DELAY_MS; added the test-only initAutoUpdaterForE2E({ feedUrl, currentVersion? }).
  • src/main/index.ts: env-gated E2E_UPDATE_FEED_URL branch; original packaged gate preserved.
  • src/main/updater.test.ts: 5 new behavior-named unit tests for the seam.

Verification (run locally on this branch's content)

  • pnpm exec vitest run src/main/updater.test.ts8/8 passed
  • npx tsc -p e2e/tsconfig.json --noEmit → clean
  • pnpm typecheck → clean
  • eslint on all 5 files → clean
  • pnpm build:e2e && playwright test e2e/spec/update-detection.e2e.ts1 passed (offline, ~0.9s)

Known design note

The spec re-triggers update.check() every 500ms inside the wait poll — this is the load-bearing delivery path (the main-process immediate emit can race the renderer's IPC subscription, which doesn't buffer). It routes through the existing update:check IPC with autoDownload=false, so it stays safe. Flagged for reviewer awareness rather than hidden.

Satisfies the standing TODO

"deterministic auto-update detection smoke that does not depend on Electron net reaching GitHub" — simulated availability from older→newer against a controlled local endpoint, proving old observes newer, leaving real HOME/app data//Applications untouched.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • テスト用のローカル更新フィードを使ったアップデート検知のテストモードを追加。テスト環境での更新検出動作を即時確認可能にしました。
  • Tests
    • 更新検知のエンドツーエンドテストを追加し、検出・通知の安定性を検証するシナリオを導入しました。

Adds an Electron e2e spec proving an older app version observes a newer
release as available, without depending on GitHub/Electron net reaching
out -- closing the gap where v0.23.0 release closeout could not verify
live update detection (net::ERR_TIMED_OUT from a temp-launched app).

Determinism / offline / safety (by construction):
- A localhost http server (ephemeral port via listen(0)) serves a
  controlled latest-mac.yml advertising version 99.0.0.
- A test-only seam, initAutoUpdaterForE2E, points the real
  electron-updater at that feed (provider: generic), forces dev update
  config (the e2e build is unpacked), and lowers currentVersion to 0.0.1
  so the feed compares as newer. autoDownload stays false, so the dummy
  artifact is never fetched and no install ever runs.
- The seam is reached only when E2E_UPDATE_FEED_URL is set, which the
  spec injects and production never sets. The production updater gate
  (app.isPackaged && E2E_DISABLE_UPDATE !== '1') is preserved verbatim.
- Runs under an isolated HOME; never touches the real HOME, userData, or
  /Applications.

The spec asserts the renderer Redux update slice reaches
status === 'available' with version === '99.0.0', exercising the full
real chain: feed -> electron-updater semver compare -> UPDATE_AVAILABLE
IPC -> setAvailable.

Also extracts the updater event-handler registration into a shared
private helper (no behavior change) and moves the 3000ms boot-check
delay to a UPDATE_CHECK_DELAY_MS constant.
@vercel

vercel Bot commented Jun 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
skills-desktop Ready Ready Preview, Comment Jun 13, 2026 2:43pm

Request Review

@coderabbitai

coderabbitai Bot commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2d988b52-392e-409c-851f-de291b2c323d

📥 Commits

Reviewing files that changed from the base of the PR and between ec47b02 and e3fcf59.

📒 Files selected for processing (2)
  • e2e/spec/update-detection.e2e.ts
  • src/main/updater.ts

Walkthrough

Electron の自動アップデータ周りをリファクタし、E2E用の initAutoUpdaterForE2E と環境分岐を追加。ローカル HTTP フィードで更新検知をテストする E2E スペック、関連定数、およびユニットテストを実装しています。

Changes

E2E update detection testing

Layer / File(s) Summary
E2E test constants
e2e/constants.ts
UPDATE_DETECTION_ADVERTISED_VERSION='99.0.0', UPDATE_DETECTION_CURRENT_VERSION='0.0.1', UPDATE_DETECTION_FEED_HOST='127.0.0.1' を追加。
Updater: event registration and delayed check
src/main/updater.ts
registerUpdaterEventHandlers() を導入し、通常起動時は UPDATE_CHECK_DELAY_MS (3000ms) 後に checkForUpdates() を実行するように変更。
E2E-specific updater initialization
src/main/updater.ts
E2EUpdaterOptionsinitAutoUpdaterForE2E(options) を追加:ループバック URL 検証、forceDevUpdateConfig=trueautoDownload=falsesetFeedURL({provider:'generic', url})、即時 checkForUpdates()
Main process E2E wiring
src/main/index.ts
E2E_UPDATE_FEED_URL が設定されていれば initAutoUpdaterForE2E(feedUrl, currentVersion) を呼ぶ分岐を追加(未設定時は従来の initAutoUpdater())。
Unit tests for E2E init
src/main/updater.test.ts
mockAutoUpdater を拡張し、initAutoUpdaterForE2EforceDevUpdateConfig を有効化、autoDownload を無効化、currentVersion を設定、setFeedURL を generic で呼び、checkForUpdates を一度呼ぶことを検証するテストを追加。
Local E2E update feed server
e2e/spec/update-detection.e2e.ts
エフェメラルポートの HTTP サーバを起動し、/latest-mac.yml のみ YAML を返す。その他パスは 404。
E2E test with Electron launch and verification
e2e/spec/update-detection.e2e.ts
隔離ホームで Electron を起動し、Redux update.statusavailable になるまで 500ms 間隔でポーリング。未到達時は window.electron.update.check() を再実行。タイムアウト時に最終 Redux 状態をエラーに埋め込み、finally で後処理を実行。

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed タイトルは E2E テストの追加を明確に説明し、「deterministic offline update-detection smoke」と具体的に変更内容を特定している。
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/e2e-update-detection-smoke

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov-commenter

codecov-commenter commented Jun 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 52.38095% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.75%. Comparing base (d77992a) to head (e3fcf59).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/main/updater.ts 64.70% 4 Missing and 2 partials ⚠️
src/main/index.ts 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #209      +/-   ##
==========================================
+ Coverage   66.68%   66.75%   +0.07%     
==========================================
  Files         198      198              
  Lines        6127     6146      +19     
  Branches     1384     1388       +4     
==========================================
+ Hits         4086     4103      +17     
  Misses       1623     1623              
- Partials      418      420       +2     
Flag Coverage Δ
unittests 66.75% <52.38%> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/main/updater.test.ts (1)

20-30: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

main-process test で fs/promises をモックしてください。

src/main/updater.ts./services/settings を import しているので、このテストは import graph の変更次第で実ファイル I/O に漏れます。既存のモック群に vi.mock('fs/promises') を追加して、main-process test を hermetic にしてください。

修正例
 vi.mock('electron-updater', () => ({
   autoUpdater: mockAutoUpdater,
 }))
 
+vi.mock('fs/promises')
+
 // updater.ts transitively imports `electron` via typedSend (BrowserWindow)
 // and the settings service (app). Neither is touched by the unit under test,
 // so a minimal surface keeps the import graph from throwing at load time.
 vi.mock('electron', () => ({

As per coding guidelines, **/*.test.ts: "Main process tests: mock fs/promises with vi.mock('fs/promises')."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/main/updater.test.ts` around lines 20 - 30, Add a vi.mock('fs/promises')
call in src/main/updater.test.ts alongside the existing mocks (the
vi.mock('electron-updater', () => ({ autoUpdater: mockAutoUpdater })) and
vi.mock('electron', ...) lines) so the test hermetically stubs filesystem
promises used by updater.ts's import of ./services/settings; place it before any
imports that trigger the updater import graph.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@e2e/spec/update-detection.e2e.ts`:
- Around line 113-190: The test currently launches the app with
_electron.launch() outside the main try, so if launch fails server and
isolatedHome leak, and a failing electronApp.close() can abort the remaining
cleanup; move the _electron.launch(...) call inside the existing try block so
launch failures are cleaned up, then make the finally block perform independent,
resilient cleanup: wrap each cleanup action (await electronApp?.close(), await
stopUpdateFeed(server), destroyIsolatedHome(isolatedHome)) in its own try/catch
or null-check so one failure doesn't prevent the others; reference the symbols
_electron.launch, electronApp, stopUpdateFeed, destroyIsolatedHome, server, and
isolatedHome when applying these changes.

In `@src/main/updater.ts`:
- Around line 146-172: The initAutoUpdaterForE2E function must validate
options.feedUrl is a loopback address before calling autoUpdater.setFeedURL;
reject any non-loopback URL (e.g., throw or return early with a clear error/log)
to prevent real network requests. Implement a check in initAutoUpdaterForE2E
that parses options.feedUrl, accepts only hosts that resolve to localhost
variants (localhost, 127.0.0.1, ::1) and denies everything else, and only call
autoUpdater.setFeedURL({ provider: 'generic', url: options.feedUrl }) when the
check passes; otherwise abort the setup immediately. Ensure the validation lives
in initAutoUpdaterForE2E and references options.feedUrl and
autoUpdater.setFeedURL so reviewers can find the change.

---

Outside diff comments:
In `@src/main/updater.test.ts`:
- Around line 20-30: Add a vi.mock('fs/promises') call in
src/main/updater.test.ts alongside the existing mocks (the
vi.mock('electron-updater', () => ({ autoUpdater: mockAutoUpdater })) and
vi.mock('electron', ...) lines) so the test hermetically stubs filesystem
promises used by updater.ts's import of ./services/settings; place it before any
imports that trigger the updater import graph.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ca1c7a72-6716-4ce5-9774-b5ef3e51cbde

📥 Commits

Reviewing files that changed from the base of the PR and between d77992a and ec47b02.

📒 Files selected for processing (5)
  • e2e/constants.ts
  • e2e/spec/update-detection.e2e.ts
  • src/main/index.ts
  • src/main/updater.test.ts
  • src/main/updater.ts

Comment thread e2e/spec/update-detection.e2e.ts Outdated
Comment thread src/main/updater.ts
- updater.ts: restrict the E2E update feed URL to a loopback http origin
  (127.0.0.1 / localhost / ::1, bracket-aware for the WHATWG `[::1]` form)
  before setFeedURL, so the test-only seam can never be pointed at a real
  network host (which would also bypass the production app.isPackaged gate).
- update-detection.e2e.ts: move _electron.launch inside the try and run
  cleanup via Promise.allSettled, so a launch failure or a failing close()
  no longer leaks the localhost feed server or the isolated HOME.
@ryota-murakami

Copy link
Copy Markdown
Contributor Author

✅ CodeRabbit findings addressed (e3fcf59)

1. E2E feed URL should be guarded to loopback — added assertLoopbackFeedUrl() in src/main/updater.ts, called first in initAutoUpdaterForE2E. E2E_UPDATE_FEED_URL flows straight into setFeedURL, so the seam now refuses any non-http:/non-loopback origin before wiring it in (defense-in-depth on top of the app.isPackaged gate).

📝 Improved on the suggested hostname === '::1' check: WHATWG URL.hostname keeps IPv6 hosts bracketed ("[::1]"), so a literal === '::1' would reject a valid IPv6 loopback feed. The guard strips brackets (.replace(/^\[|\]$/g, '')) before comparing — accepts 127.0.0.1, localhost, and ::1.

2. Resilient teardown in the spece2e/spec/update-detection.e2e.ts now declares electronApp before the try and runs cleanup via Promise.allSettled([...]) in finally, so a launch failure leaves nothing to close and one failing teardown step no longer skips the others.

Verified locally: pnpm validate green + targeted e2e green (the guard accepts the real 127.0.0.1 feed and the new teardown completes).

@ryota-murakami ryota-murakami merged commit c4ba65c into main Jun 13, 2026
11 checks passed
@ryota-murakami ryota-murakami deleted the feat/e2e-update-detection-smoke branch June 13, 2026 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants