Skip to content

Commit f9ec97a

Browse files
fix(first-run-tour): do not spend the tutorial flag on a deferred url-intent boot (#15355)
## What Two fixes from chasing the reported ~1-in-10 flake in `gettingStartedTour.spec.ts › arriving on a template link › tours the template the link loaded` (`coach-spotlight` never appears). **1. The flake is a harness defect, not an app race.** `comfyPage.setupSettings()` seeds `initialSettings` through the devtools `/devtools/set_settings` route. The fixture wrapped that call in `try { … } catch (e) { console.error(e) }`, so when the route is unavailable the seed is silently skipped and every test runs against whatever the shared per-worker user (`playwright-test-<parallelIndex>`) last persisted server-side. For this test that means `Comfy.TutorialCompleted` stays `true` from an earlier test, `resolveStartupOutcome()` returns `restored` instead of `url-intent`, `handleUrlWorkflow` correctly declines, and the failure surfaces as `coach-spotlight` not found — several layers away from the cause. Whether a given worker's user file happens to hold `true` is what makes it look like a low-rate race. Adding `waitForNodes()` does not help because the nodes were never the problem. Instrumented proof (temporary build, since `console.info`/`log` are dropped from cloud builds by the `pure` list in `vite.config.mts`): ``` [TOURDBG] initializeWorkflow {"search":"","tabState":null,"tutorialCompleted":true} [TOURDBG] resolveStartupOutcome{"search":"?template=image_z_image_turbo","tutorialCompleted":true,…} [TOURDBG] handleStartupOutcome {"outcome":"restored","tutorialCompleted":true} [TOURDBG] handleUrlWorkflow {"outcome":"restored","templateId":"image_z_image_turbo","candidate":false} ``` Every decision input the tour reads (`onboardingTourEnabled`, `isDesktopWidth`, `isSubscriptionEnabled`, `isNewUser`) was settled and correct. The seed was the only thing missing. The fixture now lets the failure through, so a broken backend reports `Failed to setup settings: 405: Method Not Allowed` on the first test instead of flaking somewhere else later. **2. A real user-facing bug the same investigation exposed.** `handleStartupOutcome` marked `Comfy.TutorialCompleted` unconditionally for a `url-intent` startup, *before* `handleUrlWorkflow` decided whether the tour would run. That flag is write-once and server-side. So a new cloud user opening a template or share link while the decision defers — window below the `md` breakpoint, `onboarding_tour_enabled` off, `subscription_required` not in remote config yet — has their one first-run tour spent without ever seeing it, on that boot and every later boot, on every device. The same write was also too early even for an eligible boot: `handleUrlWorkflow` can still decline when the link loaded nothing (the case `arriving on a link that loads nothing` covers), spending the flag on a tour that never ran. So the write now belongs to whatever earns it. `handleStartupOutcome` marks a `url-intent` boot only for a `complete` decision; `handleUrlWorkflow` marks it once `beginTour()` reports a tour actually started. A deferred boot, a dead link, or a start the engine refused all leave the account eligible — the next boot has no URL to honour, so it offers Getting Started, which is the onboarding the failed boot owed. Candidacy is still read once per boot: `isNewUser()` is decided by `initializeIfNewUser()` and cached, not re-read per call, which is what makes this ordering safe. ## Measurements | harness | before | after | | --- | --- | --- | | devtools route down (as reported) | 4/10, then 0/6 as the leaked `true` accumulates | fails immediately with the real reason | | devtools route healthy | 50/50 | 20/20, and 18/18 for the whole spec file | `--repeat-each=50 --workers=14` on a healthy backend does not reproduce the flake before or after, which is the control for "this was never a timing race". No timeouts were raised. The `405` came from `custom_nodes/ComfyUI_devtools/__init__.py` aborting on `from comfy_api.v0_0_2 import IO` against an older ComfyUI checkout — the node import runs before the route registrations, so all of its endpoints go missing. Worth pinning devtools to the backend it is tested against separately; this PR only makes the frontend notice. ## Test plan - `pnpm vitest run src/renderer/extensions/firstRunTour/gettingStarted/firstRunEntry.test.ts` — 37 passed; 5 of them fail without the source change (red/green verified). - `pnpm vitest run src/components/graph/GraphCanvas.firstRunEntry.test.ts` — passed. - `pnpm typecheck`, `pnpm typecheck:browser`, `pnpm lint`, `pnpm format:check` — clean. - `browser_tests/tests/gettingStartedTour.spec.ts` full file, `--repeat-each=3` — 18/18. - `nodeSearchBox.spec.ts` run with and without the fixture change to confirm its 23 local failures are pre-existing environment breakage, not this diff. ## Note for reviewers `browser_tests/tests/gettingStartedTour.spec.ts` is also edited on `glary/layout-ref-retain-release` (#15092), which adds a test to this same describe. This PR deliberately leaves that spec untouched — the fixes are in app code and in the shared fixture — so there is no conflict.
1 parent 21ca5a3 commit f9ec97a

3 files changed

Lines changed: 143 additions & 33 deletions

File tree

browser_tests/fixtures/ComfyPage.ts

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -645,11 +645,7 @@ export const comfyPageFixture = base.extend<{
645645
...(isVueNodes && { 'Comfy.VueNodes.Enabled': true }),
646646
...initialSettings
647647
}
648-
try {
649-
await comfyPage.setupSettings(startupSettings)
650-
} catch (e) {
651-
console.error(e)
652-
}
648+
await comfyPage.setupSettings(startupSettings)
653649
if (testInfo.tags.includes('@cloud')) {
654650
const context = page.context()
655651
await context.route('**/api/auth/session', (route) =>

src/renderer/extensions/firstRunTour/gettingStarted/firstRunEntry.test.ts

Lines changed: 130 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import { beforeEach, describe, expect, it, vi } from 'vitest'
33
import type * as VueUseCoreModule from '@vueuse/core'
44

55
import type { StartupOutcome } from '@/platform/workflow/persistence/base/draftTypes'
6+
import type { SharedWorkflowUrlLoadStatus } from '@/platform/workflow/sharing/composables/useSharedWorkflowUrlLoader'
67

78
type VueUseCore = typeof VueUseCoreModule
89

@@ -77,6 +78,8 @@ async function freshEntry() {
7778
return useFirstRunEntry()
7879
}
7980

81+
type FirstRunEntry = Awaited<ReturnType<typeof freshEntry>>
82+
8083
describe('useFirstRunEntry', () => {
8184
beforeEach(() => {
8285
mocks.isCloud = true
@@ -88,8 +91,23 @@ describe('useFirstRunEntry', () => {
8891
mocks.setSetting.mockImplementation((key: string, value: unknown) => {
8992
mocks.settings[key] = value
9093
})
94+
// beginTour reports whether a tour actually started; default to the
95+
// ordinary case so only tests about a refused start have to say so.
96+
mocks.beginTour.mockResolvedValue(true)
9197
})
9298

99+
const permanentDisqualifiers = [
100+
['not on cloud', () => void (mocks.isCloud = false)],
101+
['a returning user', () => void (mocks.isNewUser = false)]
102+
] as const
103+
104+
const transientDisqualifiers = [
105+
['below the md breakpoint', () => void (mocks.isDesktopWidth = false)],
106+
['subscription disabled', () => void (mocks.subscriptionEnabled = false)],
107+
['new-user state undetermined', () => void (mocks.isNewUser = null)],
108+
['the tour flag off', () => void (mocks.tourFlag = false)]
109+
] as const
110+
93111
describe('what a fresh user sees', () => {
94112
it('shows Getting Started to a candidate', async () => {
95113
const entry = await freshEntry()
@@ -100,18 +118,6 @@ describe('useFirstRunEntry', () => {
100118
expect(mocks.execute).not.toHaveBeenCalled()
101119
})
102120

103-
const permanentDisqualifiers: [string, () => void][] = [
104-
['not on cloud', () => void (mocks.isCloud = false)],
105-
['a returning user', () => void (mocks.isNewUser = false)]
106-
]
107-
108-
const transientDisqualifiers: [string, () => void][] = [
109-
['below the md breakpoint', () => void (mocks.isDesktopWidth = false)],
110-
['subscription disabled', () => void (mocks.subscriptionEnabled = false)],
111-
['new-user state undetermined', () => void (mocks.isNewUser = null)],
112-
['the tour flag off', () => void (mocks.tourFlag = false)]
113-
]
114-
115121
it.for([...permanentDisqualifiers, ...transientDisqualifiers])(
116122
'opens the template browser instead, with %s',
117123
async ([, disqualify]) => {
@@ -174,7 +180,7 @@ describe('useFirstRunEntry', () => {
174180
})
175181
})
176182

177-
it('marks a url-intent startup completed without taking over the screen', async () => {
183+
it('marks a url-intent startup completed once its tour starts, without taking over the screen', async () => {
178184
const entry = await freshEntry()
179185

180186
await entry.handleStartupOutcome('url-intent')
@@ -184,12 +190,86 @@ describe('useFirstRunEntry', () => {
184190
'A share or template link is the user’s choice; onboarding must not cover it'
185191
).toBe(false)
186192
expect(mocks.execute).not.toHaveBeenCalled()
193+
194+
await entry.handleUrlWorkflow('url-intent', 'image_z_image_turbo')
195+
187196
expect(
188197
mocks.setSetting,
189198
'Without this the template browser reopens on every launch, as it did before this flow existed'
190199
).toHaveBeenCalledWith('Comfy.TutorialCompleted', true)
191200
})
192201

202+
/** Both shapes of URL a first-run boot can arrive on. */
203+
const urlArrivals = [
204+
['a template link', 'image_z_image_turbo', undefined],
205+
['a share link', undefined, 'loaded']
206+
] as const satisfies readonly [
207+
string,
208+
string | undefined,
209+
SharedWorkflowUrlLoadStatus | undefined
210+
][]
211+
212+
const deferredUrlBoots = transientDisqualifiers.flatMap(([why, disqualify]) =>
213+
urlArrivals.map(
214+
([arrival, templateId, sharedStatus]) =>
215+
[
216+
`${arrival} with ${why}`,
217+
disqualify,
218+
templateId,
219+
sharedStatus
220+
] as const
221+
)
222+
)
223+
224+
it.for(deferredUrlBoots)(
225+
'runs a url-intent boot to the end for %s, offering no tour and keeping eligibility',
226+
async ([, disqualify, templateId, sharedStatus]) => {
227+
disqualify()
228+
const entry = await freshEntry()
229+
230+
await entry.handleStartupOutcome('url-intent')
231+
await entry.handleUrlWorkflow('url-intent', templateId, sharedStatus)
232+
233+
expect(
234+
mocks.beginTour,
235+
'an ineligible boot has no tour to give'
236+
).not.toHaveBeenCalled()
237+
expect(
238+
entry.gettingStartedVisible.value,
239+
'the link is the user’s choice; onboarding must not cover it'
240+
).toBe(false)
241+
expect(
242+
mocks.setSetting,
243+
'no tour ran, so the write-once flag that pays for one must stay unspent for the boot that can lift this'
244+
).not.toHaveBeenCalled()
245+
}
246+
)
247+
248+
it('tours a template link on the boot after one that only deferred', async () => {
249+
mocks.isDesktopWidth = false
250+
const phone = await freshEntry()
251+
await phone.handleStartupOutcome('url-intent')
252+
await phone.handleUrlWorkflow('url-intent', 'image_z_image_turbo')
253+
254+
expect(mocks.beginTour).not.toHaveBeenCalled()
255+
256+
mocks.isDesktopWidth = true
257+
// What `checkIsNewUser()` reads on the next launch.
258+
mocks.isNewUser = !mocks.settings['Comfy.TutorialCompleted']
259+
const laptop = await freshEntry()
260+
await laptop.handleStartupOutcome('url-intent')
261+
await laptop.handleUrlWorkflow('url-intent', 'image_z_image_turbo')
262+
263+
expect(
264+
mocks.beginTour,
265+
'opening a template link on a phone and returning on a laptop is ordinary behaviour'
266+
).toHaveBeenCalledWith('image_z_image_turbo')
267+
expect(
268+
mocks.settings['Comfy.TutorialCompleted'],
269+
'the flag is spent on the boot that finally delivered the tour, not before'
270+
).toBe(true)
271+
})
272+
193273
it('never onboards over restored work, even for an apparent new user', async () => {
194274
const entry = await freshEntry()
195275

@@ -205,18 +285,34 @@ describe('useFirstRunEntry', () => {
205285
).not.toHaveBeenCalled()
206286
})
207287

208-
it.for(['fresh', 'url-intent'] as const)(
288+
/**
289+
* A `url-intent` boot only settles once `handleUrlWorkflow` has seen what the
290+
* link loaded, so the invariant is asserted over the pair of handlers that
291+
* GraphCanvas awaits in turn, not over the first one alone.
292+
*/
293+
const startups: [StartupOutcome, (entry: FirstRunEntry) => Promise<void>][] =
294+
[
295+
['fresh', async () => {}],
296+
[
297+
'url-intent',
298+
(entry) => entry.handleUrlWorkflow('url-intent', 'image_z_image_turbo')
299+
]
300+
]
301+
302+
it.for(startups)(
209303
'settles the first-run decision on a %s startup rather than leaving it pending',
210-
async (outcome: StartupOutcome) => {
304+
async ([outcome, finishBoot]) => {
211305
const entry = await freshEntry()
212306

213307
await entry.handleStartupOutcome(outcome)
308+
await finishBoot(entry)
214309

215310
expect(
216311
entry.gettingStartedVisible.value ||
312+
mocks.beginTour.mock.calls.length > 0 ||
217313
mocks.setSetting.mock.calls.length > 0 ||
218314
mocks.execute.mock.calls.length > 0,
219-
'a startup that opened a blank canvas must offer onboarding, record that it is done, or open the template browser; anything else strands the user with an empty screen'
315+
'a startup that opened a blank canvas must offer onboarding, tour what the link loaded, record that it is done, or open the template browser; anything else strands the user with an empty screen'
220316
).toBe(true)
221317
}
222318
)
@@ -259,7 +355,7 @@ describe('useFirstRunEntry', () => {
259355
).toHaveBeenCalledWith(undefined)
260356
})
261357

262-
it('leaves the completion flag to the startup handler that already settled it', async () => {
358+
it('leaves the completion flag alone when the engine refused to start', async () => {
263359
const entry = await freshEntry()
264360
await entry.handleStartupOutcome('url-intent')
265361
mocks.setSetting.mockClear()
@@ -269,7 +365,23 @@ describe('useFirstRunEntry', () => {
269365

270366
expect(
271367
mocks.setSetting,
272-
'writing it again here would mark onboarding done for a user whose tour never started, and postpone() exists to offer that user the tour again'
368+
'writing it here would mark onboarding done for a user whose tour never started, and postpone() exists to offer that user the tour again'
369+
).not.toHaveBeenCalled()
370+
})
371+
372+
it('keeps the account eligible when the link loaded nothing to tour', async () => {
373+
const entry = await freshEntry()
374+
375+
await entry.handleStartupOutcome('url-intent')
376+
await entry.handleUrlWorkflow('url-intent', undefined, 'failed')
377+
378+
expect(
379+
mocks.beginTour,
380+
'there is no workflow on the canvas to tour'
381+
).not.toHaveBeenCalled()
382+
expect(
383+
mocks.setSetting,
384+
'a dead link must not spend the one tour the account gets; the next boot has no URL to honour and offers Getting Started instead'
273385
).not.toHaveBeenCalled()
274386
})
275387

src/renderer/extensions/firstRunTour/gettingStarted/firstRunEntry.ts

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -52,23 +52,19 @@ export const useFirstRunEntry = createSharedComposable(() => {
5252
return decideFirstRun() === 'getting-started'
5353
}
5454

55-
/**
56-
* Candidacy is read once, here: a later breakpoint, flag or subscription
57-
* change must not unmount the screen out from under the user.
58-
* Only a boot that opened a blank canvas can be onboarded over. `isNewUser()`
59-
* cannot carry that alone — it reads `Comfy.TutorialCompleted`, which is
60-
* exactly what a user predating the setting is missing.
61-
*/
55+
// `url-intent` defers to handleUrlWorkflow: we don't know yet whether
56+
// anything arrived to tour, and TutorialCompleted is write-once.
6257
async function handleStartupOutcome(outcome: StartupOutcome) {
6358
if (outcome === 'restored') return
6459
if (settingStore.get('Comfy.TutorialCompleted')) return
6560

61+
const decision = decideFirstRun()
62+
6663
if (outcome === 'url-intent') {
67-
await markTutorialCompleted()
64+
if (decision === 'complete') await markTutorialCompleted()
6865
return
6966
}
7067

71-
const decision = decideFirstRun()
7268
if (decision === 'getting-started') {
7369
gettingStartedVisible.value = true
7470
return
@@ -82,6 +78,11 @@ export const useFirstRunEntry = createSharedComposable(() => {
8278
* A share or template link loads its workflow instead of the Getting Started
8379
* screen, so the tour is offered over whatever arrived. The engine declines
8480
* to repeat a tour the user has already seen.
81+
*
82+
* A tour that actually started is what `Comfy.TutorialCompleted` pays for, so
83+
* only that writes it. A link that loaded nothing, or a start the engine
84+
* refused, leaves the account eligible: the next boot has no URL to honour and
85+
* offers Getting Started, which is the onboarding this one failed to deliver.
8586
*/
8687
async function handleUrlWorkflow(
8788
outcome: StartupOutcome | undefined,
@@ -92,9 +93,10 @@ export const useFirstRunEntry = createSharedComposable(() => {
9293
const shareLoaded =
9394
sharedStatus === 'loaded' || sharedStatus === 'loaded-without-assets'
9495
if (templateId === undefined && !shareLoaded) return
95-
await useFirstRunTourController().beginTour(
96+
const started = await useFirstRunTourController().beginTour(
9697
shareLoaded ? undefined : templateId
9798
)
99+
if (started) await markTutorialCompleted()
98100
}
99101

100102
// Applied locally before the request, so a failed write is next launch's problem.

0 commit comments

Comments
 (0)