This repository was archived by the owner on Aug 1, 2025. It is now read-only.
e2e-flakiness-detector #494
This run and associated checks have been archived and are scheduled for deletion.
Learn more about checks retention
Annotations
16 errors, 3 warnings, and 2 notices
|
auth.test.ts:25:3 › requires a valid auth token and allows logouts:
vscode\test\fixtures\mock-server.ts#L661
9) auth.test.ts:25:3 › requires a valid auth token and allows logouts ────────────────────────────
Error: listen EACCES: permission denied 0.0.0.0:49300
at ..\fixtures\mock-server.ts:661
659 | });
660 |
> 661 | const server = app.listen(SERVER_PORT);
| ^
662 |
663 | // Calling close() on the server only stops accepting new connections
664 | // and does not terminate existing connections. This can result in
at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24)
at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28)
at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
|
|
attribution.test.ts:37:1 › attribution search disabled in chat:
vscode\test\fixtures\mock-server.ts#L661
8) attribution.test.ts:37:1 › attribution search disabled in chat ────────────────────────────────
Error: listen EACCES: permission denied 0.0.0.0:49300
at ..\fixtures\mock-server.ts:661
659 | });
660 |
> 661 | const server = app.listen(SERVER_PORT);
| ^
662 |
663 | // Calling close() on the server only stops accepting new connections
664 | // and does not terminate existing connections. This can result in
at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24)
at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28)
at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
|
|
attribution.test.ts:12:1 › attribution search enabled in chat:
vscode\test\fixtures\mock-server.ts#L661
7) attribution.test.ts:12:1 › attribution search enabled in chat ─────────────────────────────────
Error: listen EACCES: permission denied 0.0.0.0:49300
at ..\fixtures\mock-server.ts:661
659 | });
660 |
> 661 | const server = app.listen(SERVER_PORT);
| ^
662 |
663 | // Calling close() on the server only stops accepting new connections
664 | // and does not terminate existing connections. This can result in
at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24)
at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28)
at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
|
|
attribution.test.ts:37:1 › attribution search disabled in chat:
vscode\test\fixtures\mock-server.ts#L661
6) attribution.test.ts:37:1 › attribution search disabled in chat ────────────────────────────────
Error: listen EACCES: permission denied 0.0.0.0:49300
at ..\fixtures\mock-server.ts:661
659 | });
660 |
> 661 | const server = app.listen(SERVER_PORT);
| ^
662 |
663 | // Calling close() on the server only stops accepting new connections
664 | // and does not terminate existing connections. This can result in
at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24)
at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28)
at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
|
|
attribution.test.ts:12:1 › attribution search enabled in chat:
vscode\test\fixtures\mock-server.ts#L661
5) attribution.test.ts:12:1 › attribution search enabled in chat ─────────────────────────────────
Error: listen EACCES: permission denied 0.0.0.0:49300
at ..\fixtures\mock-server.ts:661
659 | });
660 |
> 661 | const server = app.listen(SERVER_PORT);
| ^
662 |
663 | // Calling close() on the server only stops accepting new connections
664 | // and does not terminate existing connections. This can result in
at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24)
at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28)
at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
|
|
attribution.test.ts:37:1 › attribution search disabled in chat:
vscode\test\fixtures\mock-server.ts#L661
4) attribution.test.ts:37:1 › attribution search disabled in chat ────────────────────────────────
Error: listen EACCES: permission denied 0.0.0.0:49300
at ..\fixtures\mock-server.ts:661
659 | });
660 |
> 661 | const server = app.listen(SERVER_PORT);
| ^
662 |
663 | // Calling close() on the server only stops accepting new connections
664 | // and does not terminate existing connections. This can result in
at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24)
at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28)
at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
|
|
attribution.test.ts:12:1 › attribution search enabled in chat:
vscode\test\fixtures\mock-server.ts#L661
3) attribution.test.ts:12:1 › attribution search enabled in chat ─────────────────────────────────
Error: listen EACCES: permission denied 0.0.0.0:49300
at ..\fixtures\mock-server.ts:661
659 | });
660 |
> 661 | const server = app.listen(SERVER_PORT);
| ^
662 |
663 | // Calling close() on the server only stops accepting new connections
664 | // and does not terminate existing connections. This can result in
at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24)
at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28)
at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
|
|
attribution.test.ts:37:1 › attribution search disabled in chat:
vscode\test\fixtures\mock-server.ts#L661
2) attribution.test.ts:37:1 › attribution search disabled in chat ────────────────────────────────
Error: listen EACCES: permission denied 0.0.0.0:49300
at ..\fixtures\mock-server.ts:661
659 | });
660 |
> 661 | const server = app.listen(SERVER_PORT);
| ^
662 |
663 | // Calling close() on the server only stops accepting new connections
664 | // and does not terminate existing connections. This can result in
at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24)
at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28)
at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
|
|
attribution.test.ts:12:1 › attribution search enabled in chat:
vscode\test\fixtures\mock-server.ts#L661
1) attribution.test.ts:12:1 › attribution search enabled in chat ─────────────────────────────────
Error: listen EACCES: permission denied 0.0.0.0:49300
at ..\fixtures\mock-server.ts:661
659 | });
660 |
> 661 | const server = app.listen(SERVER_PORT);
| ^
662 |
663 | // Calling close() on the server only stops accepting new connections
664 | // and does not terminate existing connections. This can result in
at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24)
at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28)
at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
|
|
test-e2e (windows)
Error: ENOENT: no such file or directory, scandir 'D:\a\cody\cody\playwright'
|
|
test-e2e (ubuntu)
Process completed with exit code 1.
|
|
initial-context.test.ts:46:18 › initial context - file:
vscode/test/e2e/initial-context.test.ts#L68
4) initial-context.test.ts:46:18 › initial context - file ────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- Expected - 0
+ Received + 1
Array [
"main.c",
+ "main.c:1-3",
"myrepo",
]
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- locator resolved to 3 elements
- locator resolved to 3 elements
- locator resolved to 3 elements
- locator resolved to 3 elements
- locator resolved to 3 elements
- locator resolved to 3 elements
- locator resolved to 3 elements
- locator resolved to 3 elements
66 |
67 | await clickEditorTab(page, 'main.c')
> 68 | await expect(chatInputMentions(lastChatInput)).toHaveText(['main.c', 'myrepo'])
| ^
69 |
70 | // After typing into the input, it no longer updates the initial context.
71 | await lastChatInput.press('x')
at /home/runner/work/cody/cody/vscode/test/e2e/initial-context.test.ts:68:52
|
|
command-edit.test.ts:121:1 › edit (fixup) input - range selection:
vscode/test/e2e/command-edit.test.ts#L150
3) command-edit.test.ts:121:1 › edit (fixup) input - range selection ─────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for getByLabel('symbol-method fizzbuzz')
- locator resolved to <div role="option" data-index="0" aria-level="1" aria-setsize="3" id="list_id_6_0" aria-posinset="1" draggable="false" data-parity="even" data-last-element="false" class="monaco-list-row focused" aria-label="symbol-method fizzbuzz, symbols">…</div>
- attempting click action
- waiting for element to be visible, enabled and stable
- element is not stable
- retrying click action, attempt #1
- waiting for element to be visible, enabled and stable
- element is not visible
- retrying click action, attempt #2
- waiting 20ms
148 | await symbolitem.click()
149 | const selectedSymbolItem = page.getByLabel('symbol-method fizzbuzz')
> 150 | await selectedSymbolItem.click()
| ^
151 |
152 | // Check that the range input updated correctly to reflect the selected symbol
153 | const inputBox = page.getByPlaceholder(/^Enter edit instructions \(type @ to include code/)
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:150:30
|
|
command-edit.test.ts:121:1 › edit (fixup) input - range selection:
vscode/test/e2e/command-edit.test.ts#L139
3) command-edit.test.ts:121:1 › edit (fixup) input - range selection ─────────────────────────────
Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements:
1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block')
2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' })
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByText('Nearest Code Block')
137 | // Check the correct range item is auto-selected
138 | const rangeItem = page.getByText('Nearest Code Block')
> 139 | expect(rangeItem).toBeVisible()
| ^
140 |
141 | // Open the range input and check it has the correct item selected
142 | await rangeItem.click()
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:139:23
|
|
command-edit.test.ts:33:3 › edit (fixup) task:
vscode/test/e2e/command-edit.test.ts#L53
2) command-edit.test.ts:33:3 › edit (fixup) task ─────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
Expected: visible
Received: hidden
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
51 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
52 | await nap()
> 53 | expect(modelItem).toBeVisible()
| ^
54 |
55 | // Open the model input and check it has the correct item selected
56 | await modelItem.click()
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:53:23
|
|
command-edit.test.ts:33:3 › edit (fixup) task:
vscode/test/e2e/command-edit.test.ts#L53
1) command-edit.test.ts:33:3 › edit (fixup) task ─────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
Expected: visible
Received: hidden
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
51 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
52 | await nap()
> 53 | expect(modelItem).toBeVisible()
| ^
54 |
55 | // Open the model input and check it has the correct item selected
56 | await modelItem.click()
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:53:23
|
|
test-e2e (windows)
The process 'C:\Program Files\Git\bin\git.exe' failed with exit code 128
|
|
test-e2e (windows)
No files were found with the provided path: playwright/. No artifacts will be uploaded.
|
|
test-e2e (ubuntu)
The process '/usr/bin/git' failed with exit code 128
|
|
🎭 Playwright Run Summary
164 failed
attribution.test.ts:12:1 › attribution search enabled in chat ──────────────────────────────────
attribution.test.ts:37:1 › attribution search disabled in chat ─────────────────────────────────
attribution.test.ts:12:1 › attribution search enabled in chat ──────────────────────────────────
attribution.test.ts:37:1 › attribution search disabled in chat ─────────────────────────────────
attribution.test.ts:12:1 › attribution search enabled in chat ──────────────────────────────────
attribution.test.ts:37:1 › attribution search disabled in chat ─────────────────────────────────
attribution.test.ts:12:1 › attribution search enabled in chat ──────────────────────────────────
attribution.test.ts:37:1 › attribution search disabled in chat ─────────────────────────────────
auth.test.ts:25:3 › requires a valid auth token and allows logouts ─────────────────────────────
auth.test.ts:72:7 › test enterprise customers cannot log into dotcomUrl ────────────────────────
auth.test.ts:101:7 › logs out the user when userShouldUseEnterprise is set to true ─────────────
auth.test.ts:25:3 › requires a valid auth token and allows logouts ─────────────────────────────
auth.test.ts:72:7 › test enterprise customers cannot log into dotcomUrl ────────────────────────
auth.test.ts:101:7 › logs out the user when userShouldUseEnterprise is set to true ─────────────
auth.test.ts:25:3 › requires a valid auth token and allows logouts ─────────────────────────────
auth.test.ts:72:7 › test enterprise customers cannot log into dotcomUrl ────────────────────────
auth.test.ts:101:7 › logs out the user when userShouldUseEnterprise is set to true ─────────────
auth.test.ts:25:3 › requires a valid auth token and allows logouts ─────────────────────────────
auth.test.ts:72:7 › test enterprise customers cannot log into dotcomUrl ────────────────────────
auth.test.ts:101:7 › logs out the user when userShouldUseEnterprise is set to true ─────────────
chat-atFile.test.ts:49:7 › @-mention file in chat ──────────────────────────────────────────────
chat-atFile.test.ts:232:3 › editing a chat message with @-mention ──────────────────────────────
chat-atFile.test.ts:278:3 › @-mention file range ───────────────────────────────────────────────
chat-atFile.test.ts:316:3 › @-mention symbol in chat ───────────────────────────────────────────
chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ─────────────────────────────────────────
chat-atFile.test.ts:49:7 › @-mention file in chat ──────────────────────────────────────────────
chat-atFile.test.ts:232:3 › editing a chat message with @-mention ──────────────────────────────
chat-atFile.test.ts:278:3 › @-mention file range ───────────────────────────────────────────────
chat-atFile.test.ts:316:3 › @-mention symbol in chat ───────────────────────────────────────────
chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ─────────────────────────────────────────
chat-atFile.test.ts:49:7 › @-mention file in chat ──────────────────────────────────────────────
chat-atFile.test.ts:232:3 › editing a chat message with @-mention ──────────────────────────────
chat-atFile.test.ts:278:3 › @-mention file range ───────────────────────────────────────────────
chat-atFile.test.ts:316:3 › @-mention symbol in chat ───────────────────────────────────────────
chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ─────────────────────────────────────────
chat-atFile.test.ts:49:7 › @-mention file in chat ──────────────────────────────────────────────
chat-atFile.test.ts:232:3 › editing a chat message with @-mention ──────────────────────────────
chat-atFile.test.ts:278:3 › @-mention file range ───────────────────────────────────────────────
chat-atFile.test.ts:316:3 › @-mention symbol in chat ───────────────────────────────────────────
chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ─────────────────────────────────────────
chat-context.test.ts:11:5 › chat follow
|
|
🎭 Playwright Run Summary
4 failed
command-edit.test.ts:33:3 › edit (fixup) task ──────────────────────────────────────────────────
command-edit.test.ts:33:3 › edit (fixup) task ──────────────────────────────────────────────────
command-edit.test.ts:121:1 › edit (fixup) input - range selection ──────────────────────────────
initial-context.test.ts:46:18 › initial context - file ─────────────────────────────────────────
150 skipped
406 passed (1.8h)
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
playwright-recordings ubuntu
Expired
|
1.2 MB |
sha256:64608921014af5218c2cc5f2a4387f278613c57bb7f4b059d7454edddb0b6040
|
|