Skip to content

Commit 0897166

Browse files
yomybabyyomybaby
committed
fix(FR-2875,FR-2876): drop @smoke-any tag, switch tsconfig to NodeNext, address mixed-role and unstable specs
1 parent c61d22d commit 0897166

8 files changed

Lines changed: 130 additions & 25752 deletions

File tree

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,3 +179,6 @@ e2e/.agent-output/
179179

180180

181181
e2e/recordings/
182+
183+
# ignore stale branch-suffixed pnpm lockfiles (see PR #7340)
184+
pnpm-lock.*-*.yaml

‎e2e/E2E-TEST-NAMING-GUIDELINES.md‎

Lines changed: 23 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -412,10 +412,14 @@ that may be air-gapped.
412412
| Tag | Meaning |
413413
|-----|---------|
414414
| `@smoke` | Base smoke set. Included in every run. |
415-
| `@smoke-any` | Role-agnostic (works whether the supplied account is admin or user). |
416415
| `@smoke-admin` | Requires admin role. Skipped when running as a regular user. |
417416
| `@smoke-user` | Requires user-level access only. Admin accounts can run these too. |
418-
| `@smoke-extended` | Opt-in via `--profile extended`; excluded from the default run. |
417+
418+
> `@smoke-any` and `@smoke-extended` are intentionally **not** part of the
419+
> MVP taxonomy. The first turned out unworkable in practice (every e2e
420+
> helper hard-codes a role via `loginAsAdmin` / `loginAsUser`, so no
421+
> describe is genuinely role-agnostic at the helper level), and the second
422+
> requires a `--profile` flag the CLI does not implement yet.
419423

420424
These are **additive metadata** — existing tags (`@critical`, `@regression`,
421425
`@functional`, etc.) are preserved. Existing CI jobs that grep by other tags
@@ -442,6 +446,11 @@ following:
442446
5. **No outbound dependencies.** Smoke runs on air-gapped hosts. Specs must
443447
not call `fetch('https://...')` for any host other than the WebUI endpoint
444448
itself.
449+
6. **Describe-level tags apply to every nested test.** A describe that
450+
mixes `loginAsAdmin` and `loginAsUser` is **not** taggable as smoke at
451+
the describe level — split the describe so each one uses a single
452+
account, then tag only the role-uniform half. This keeps rule 1
453+
("single account") consistent with the role tags.
445454

446455
### How to apply
447456

@@ -451,22 +460,16 @@ Add the tag to the existing `tag: [...]` array on the outermost
451460
```typescript
452461
// Before
453462
test.describe(
454-
'Session Lifecycle Management',
455-
{ tag: ['@critical', '@session', '@functional'] },
463+
'Login',
464+
{ tag: ['@auth', '@functional'] },
456465
() => { /* ... */ },
457466
);
458467

459-
// After — added @smoke and @smoke-user
468+
// After — added @smoke and @smoke-admin (beforeEach uses loginAsAdmin)
460469
test.describe(
461-
'Session Lifecycle Management',
470+
'Login',
462471
{
463-
tag: [
464-
'@critical',
465-
'@session',
466-
'@functional',
467-
'@smoke',
468-
'@smoke-user',
469-
],
472+
tag: ['@auth', '@functional', '@smoke', '@smoke-admin'],
470473
},
471474
() => { /* ... */ },
472475
);
@@ -478,12 +481,16 @@ A spec without a `tag:` option needs one added — never strip existing tags.
478481

479482
The initial smoke set covers the highest-signal flows:
480483

481-
- Login / authentication → `@smoke-any`
482-
- Dashboard render → `@smoke-any`
483-
- Session lifecycle (create → running → terminate) → `@smoke-user`
484+
- Login / authentication → `@smoke-admin` (`loginAsAdmin`)
485+
- Dashboard render (admin describe only; user-role checks live in a sibling describe without `@smoke*`) → `@smoke-admin`
484486
- VFolder basic CRUD → `@smoke-user`
485487
- Agent list (admin signal) → `@smoke-admin`
486488

489+
Session lifecycle is intentionally **not** in the MVP smoke set: its
490+
scenarios use 240s timeouts and `test.fixme(no agents)` skips, which
491+
violates the bounded-runtime rule. A trimmed-down session smoke
492+
scenario is tracked as a follow-up.
493+
487494
Pick **quality over quantity**: do not tag every spec in a folder. The smoke
488495
suite's value is its short runtime and high signal-to-noise ratio.
489496

‎e2e/auth/login.spec.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ test.beforeEach(async ({ page, request }) => {
3232

3333
test.describe(
3434
'Before Login',
35-
{ tag: ['@smoke', '@smoke-any', '@auth', '@functional'] },
35+
{ tag: ['@smoke', '@smoke-admin', '@auth', '@functional'] },
3636
() => {
3737
test('should display the login form', async ({ page }) => {
3838
await expect(page.getByLabel('Email or Username')).toBeVisible();
@@ -44,7 +44,7 @@ test.describe(
4444

4545
test.describe(
4646
'Login',
47-
{ tag: ['@smoke', '@smoke-any', '@auth', '@functional'] },
47+
{ tag: ['@smoke', '@smoke-admin', '@auth', '@functional'] },
4848
() => {
4949
test.beforeEach(async ({ page, request }) => {
5050
await loginAsAdmin(page, request);

‎e2e/dashboard/dashboard.spec.ts‎

Lines changed: 91 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ const WIDGET_TIMEOUT = 30_000;
1212
test.describe(
1313
'Dashboard Page',
1414
{
15-
tag: ['@regression', '@dashboard', '@functional', '@smoke', '@smoke-any'],
15+
tag: ['@regression', '@dashboard', '@functional', '@smoke', '@smoke-admin'],
1616
},
1717
() => {
1818
// -----------------------------------------------------------------------
@@ -60,79 +60,6 @@ test.describe(
6060
.first(),
6161
).toBeVisible({ timeout: WIDGET_TIMEOUT });
6262
});
63-
64-
test('Regular user sees "My Sessions" title instead of "Active Sessions" on the session count widget', async ({
65-
page,
66-
request,
67-
}) => {
68-
// 1. Login as regular user and navigate to /summary
69-
await loginAsUser(page, request);
70-
await navigateTo(page, 'summary');
71-
72-
// 2. Verify the widget title reads "My Sessions" (not "Active Sessions")
73-
await expect(
74-
page
75-
.locator('.bai_grid_item')
76-
.filter({ hasText: 'My Sessions' })
77-
.first(),
78-
).toBeVisible({ timeout: WIDGET_TIMEOUT });
79-
await expect(
80-
page
81-
.locator('.bai_grid_item')
82-
.filter({ hasText: 'Active Sessions' })
83-
.first(),
84-
).not.toBeVisible();
85-
});
86-
87-
test('Regular user sees dashboard without admin-only widgets', async ({
88-
page,
89-
request,
90-
}) => {
91-
// 1. Login as regular user and navigate to /summary
92-
await loginAsUser(page, request);
93-
await navigateTo(page, 'summary');
94-
95-
// 2. Verify the "My Sessions" widget is visible
96-
await expect(
97-
page
98-
.locator('.bai_grid_item')
99-
.filter({ hasText: 'My Sessions' })
100-
.first(),
101-
).toBeVisible({ timeout: WIDGET_TIMEOUT });
102-
103-
// 3. Verify the "My Resources" widget is visible
104-
await expect(
105-
page
106-
.locator('.bai_grid_item')
107-
.filter({ hasText: 'My Total Resources Limit' })
108-
.first(),
109-
).toBeVisible({ timeout: WIDGET_TIMEOUT });
110-
111-
// 4. Verify the "My Resources in Resource Group" widget is visible
112-
// The widget title is "My Resources in" followed by a resource group selector
113-
await expect(
114-
page
115-
.locator('.bai_grid_item')
116-
.filter({ hasText: 'My Resources in' })
117-
.first(),
118-
).toBeVisible({ timeout: WIDGET_TIMEOUT });
119-
120-
// 5. Verify the "Agent Stats" widget is NOT present (admin only)
121-
await expect(
122-
page
123-
.locator('.bai_grid_item')
124-
.filter({ hasText: 'Agent Statistics' })
125-
.first(),
126-
).not.toBeVisible();
127-
128-
// 6. Verify the "Recently Created Sessions" table is visible
129-
await expect(
130-
page
131-
.locator('.bai_grid_item')
132-
.filter({ hasText: 'Recently Created Sessions' })
133-
.first(),
134-
).toBeVisible({ timeout: WIDGET_TIMEOUT });
135-
});
13663
});
13764

13865
// -----------------------------------------------------------------------
@@ -404,6 +331,13 @@ test.describe(
404331
// -----------------------------------------------------------------------
405332
// 12. Board Layout (Resize and Move - smoke tests)
406333
// -----------------------------------------------------------------------
334+
// -----------------------------------------------------------------------
335+
// User-role dashboard tests
336+
//
337+
// Kept outside the @smoke / @smoke-admin describe because the smoke set
338+
// must use a single account; these tests log in as a regular user.
339+
// -----------------------------------------------------------------------
340+
407341
test.describe('Board Layout', () => {
408342
test('Admin can see resizable and movable widgets on the Dashboard', async ({
409343
page,
@@ -435,3 +369,86 @@ test.describe(
435369
});
436370
},
437371
);
372+
373+
// Regular-user dashboard checks live in a separate describe so they are
374+
// excluded from `@smoke` / `@smoke-admin` runs (smoke runs use a single
375+
// logged-in account, here admin).
376+
test.describe(
377+
'Dashboard Page (user role)',
378+
{
379+
tag: ['@regression', '@dashboard', '@functional'],
380+
},
381+
() => {
382+
test('Regular user sees "My Sessions" title instead of "Active Sessions" on the session count widget', async ({
383+
page,
384+
request,
385+
}) => {
386+
// 1. Login as regular user and navigate to /summary
387+
await loginAsUser(page, request);
388+
await navigateTo(page, 'summary');
389+
390+
// 2. Verify the widget title reads "My Sessions" (not "Active Sessions")
391+
await expect(
392+
page
393+
.locator('.bai_grid_item')
394+
.filter({ hasText: 'My Sessions' })
395+
.first(),
396+
).toBeVisible({ timeout: WIDGET_TIMEOUT });
397+
await expect(
398+
page
399+
.locator('.bai_grid_item')
400+
.filter({ hasText: 'Active Sessions' })
401+
.first(),
402+
).not.toBeVisible();
403+
});
404+
405+
test('Regular user sees dashboard without admin-only widgets', async ({
406+
page,
407+
request,
408+
}) => {
409+
// 1. Login as regular user and navigate to /summary
410+
await loginAsUser(page, request);
411+
await navigateTo(page, 'summary');
412+
413+
// 2. Verify the "My Sessions" widget is visible
414+
await expect(
415+
page
416+
.locator('.bai_grid_item')
417+
.filter({ hasText: 'My Sessions' })
418+
.first(),
419+
).toBeVisible({ timeout: WIDGET_TIMEOUT });
420+
421+
// 3. Verify the "My Resources" widget is visible
422+
await expect(
423+
page
424+
.locator('.bai_grid_item')
425+
.filter({ hasText: 'My Total Resources Limit' })
426+
.first(),
427+
).toBeVisible({ timeout: WIDGET_TIMEOUT });
428+
429+
// 4. Verify the "My Resources in Resource Group" widget is visible
430+
await expect(
431+
page
432+
.locator('.bai_grid_item')
433+
.filter({ hasText: 'My Resources in' })
434+
.first(),
435+
).toBeVisible({ timeout: WIDGET_TIMEOUT });
436+
437+
// 5. Verify the "Agent Stats" widget is NOT present (admin only)
438+
await expect(
439+
page
440+
.locator('.bai_grid_item')
441+
.filter({ hasText: 'Agent Statistics' })
442+
.first(),
443+
).not.toBeVisible();
444+
445+
// 6. Verify the "Recently Created Sessions" table is visible
446+
await expect(
447+
page
448+
.locator('.bai_grid_item')
449+
.filter({ hasText: 'Recently Created Sessions' })
450+
.first(),
451+
).toBeVisible({ timeout: WIDGET_TIMEOUT });
452+
});
453+
},
454+
);

‎e2e/session/session-lifecycle.spec.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,11 @@ import { test, expect } from '@playwright/test';
2626
test.describe(
2727
'Session Lifecycle Management',
2828
{
29-
tag: ['@critical', '@session', '@functional', '@smoke', '@smoke-user'],
29+
// NOTE: not tagged @smoke for MVP — multiple scenarios use 240s timeouts
30+
// and `test.fixme(no agents)` skips, which makes the spec too heavy /
31+
// environment-dependent for a 5–10 minute smoke run. Tracked as a
32+
// follow-up to introduce a lightweight session smoke scenario.
33+
tag: ['@critical', '@session', '@functional'],
3034
},
3135
() => {
3236
// Run tests sequentially to avoid resource exhaustion

‎packages/backend.ai-webui-smoke-cli/src/catalog.ts‎

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -23,21 +23,15 @@ export const SMOKE_CATALOG: ReadonlyArray<SmokeCategory> = [
2323
{
2424
category: 'auth',
2525
description: 'Login form rendering and successful sign-in flow.',
26-
tags: ['@smoke', '@smoke-any'],
26+
tags: ['@smoke', '@smoke-admin'],
2727
specs: ['e2e/auth/login.spec.ts'],
2828
},
2929
{
3030
category: 'dashboard',
31-
description: 'Dashboard widgets render after login.',
32-
tags: ['@smoke', '@smoke-any'],
33-
specs: ['e2e/dashboard/dashboard.spec.ts'],
34-
},
35-
{
36-
category: 'session',
3731
description:
38-
'Session lifecycle: create batch session, wait for RUNNING, terminate.',
39-
tags: ['@smoke', '@smoke-user'],
40-
specs: ['e2e/session/session-lifecycle.spec.ts'],
32+
'Dashboard widgets render after admin login. Regular-user dashboard checks live in a separate (non-smoke) describe.',
33+
tags: ['@smoke', '@smoke-admin'],
34+
specs: ['e2e/dashboard/dashboard.spec.ts'],
4135
},
4236
{
4337
category: 'vfolder',

‎packages/backend.ai-webui-smoke-cli/tsconfig.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
{
22
"compilerOptions": {
33
"target": "ES2022",
4-
"module": "ESNext",
5-
"moduleResolution": "bundler",
4+
"module": "NodeNext",
5+
"moduleResolution": "NodeNext",
66
"lib": ["ES2022"],
77
"outDir": "./dist",
88
"rootDir": "./src",

0 commit comments

Comments
 (0)