Skip to content
This repository was archived by the owner on Aug 1, 2025. It is now read-only.

e2e-flakiness-detector #494

e2e-flakiness-detector

e2e-flakiness-detector #494

Triggered via schedule July 16, 2025 14:05
Status Failure
Total duration 1h 49m 36s
Artifacts 1
This run and associated checks have been archived and are scheduled for deletion. Learn more about checks retention
Matrix: test-e2e
Fit to window
Zoom out
Zoom in

Annotations

16 errors, 3 warnings, and 2 notices
auth.test.ts:25:3 › requires a valid auth token and allows logouts: vscode\test\fixtures\mock-server.ts#L661
9) auth.test.ts:25:3 › requires a valid auth token and allows logouts ──────────────────────────── Error: listen EACCES: permission denied 0.0.0.0:49300 at ..\fixtures\mock-server.ts:661 659 | }); 660 | > 661 | const server = app.listen(SERVER_PORT); | ^ 662 | 663 | // Calling close() on the server only stops accepting new connections 664 | // and does not terminate existing connections. This can result in at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24) at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28) at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
attribution.test.ts:37:1 › attribution search disabled in chat: vscode\test\fixtures\mock-server.ts#L661
8) attribution.test.ts:37:1 › attribution search disabled in chat ──────────────────────────────── Error: listen EACCES: permission denied 0.0.0.0:49300 at ..\fixtures\mock-server.ts:661 659 | }); 660 | > 661 | const server = app.listen(SERVER_PORT); | ^ 662 | 663 | // Calling close() on the server only stops accepting new connections 664 | // and does not terminate existing connections. This can result in at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24) at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28) at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
attribution.test.ts:12:1 › attribution search enabled in chat: vscode\test\fixtures\mock-server.ts#L661
7) attribution.test.ts:12:1 › attribution search enabled in chat ───────────────────────────────── Error: listen EACCES: permission denied 0.0.0.0:49300 at ..\fixtures\mock-server.ts:661 659 | }); 660 | > 661 | const server = app.listen(SERVER_PORT); | ^ 662 | 663 | // Calling close() on the server only stops accepting new connections 664 | // and does not terminate existing connections. This can result in at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24) at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28) at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
attribution.test.ts:37:1 › attribution search disabled in chat: vscode\test\fixtures\mock-server.ts#L661
6) attribution.test.ts:37:1 › attribution search disabled in chat ──────────────────────────────── Error: listen EACCES: permission denied 0.0.0.0:49300 at ..\fixtures\mock-server.ts:661 659 | }); 660 | > 661 | const server = app.listen(SERVER_PORT); | ^ 662 | 663 | // Calling close() on the server only stops accepting new connections 664 | // and does not terminate existing connections. This can result in at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24) at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28) at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
attribution.test.ts:12:1 › attribution search enabled in chat: vscode\test\fixtures\mock-server.ts#L661
5) attribution.test.ts:12:1 › attribution search enabled in chat ───────────────────────────────── Error: listen EACCES: permission denied 0.0.0.0:49300 at ..\fixtures\mock-server.ts:661 659 | }); 660 | > 661 | const server = app.listen(SERVER_PORT); | ^ 662 | 663 | // Calling close() on the server only stops accepting new connections 664 | // and does not terminate existing connections. This can result in at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24) at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28) at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
attribution.test.ts:37:1 › attribution search disabled in chat: vscode\test\fixtures\mock-server.ts#L661
4) attribution.test.ts:37:1 › attribution search disabled in chat ──────────────────────────────── Error: listen EACCES: permission denied 0.0.0.0:49300 at ..\fixtures\mock-server.ts:661 659 | }); 660 | > 661 | const server = app.listen(SERVER_PORT); | ^ 662 | 663 | // Calling close() on the server only stops accepting new connections 664 | // and does not terminate existing connections. This can result in at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24) at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28) at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
attribution.test.ts:12:1 › attribution search enabled in chat: vscode\test\fixtures\mock-server.ts#L661
3) attribution.test.ts:12:1 › attribution search enabled in chat ───────────────────────────────── Error: listen EACCES: permission denied 0.0.0.0:49300 at ..\fixtures\mock-server.ts:661 659 | }); 660 | > 661 | const server = app.listen(SERVER_PORT); | ^ 662 | 663 | // Calling close() on the server only stops accepting new connections 664 | // and does not terminate existing connections. This can result in at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24) at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28) at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
attribution.test.ts:37:1 › attribution search disabled in chat: vscode\test\fixtures\mock-server.ts#L661
2) attribution.test.ts:37:1 › attribution search disabled in chat ──────────────────────────────── Error: listen EACCES: permission denied 0.0.0.0:49300 at ..\fixtures\mock-server.ts:661 659 | }); 660 | > 661 | const server = app.listen(SERVER_PORT); | ^ 662 | 663 | // Calling close() on the server only stops accepting new connections 664 | // and does not terminate existing connections. This can result in at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24) at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28) at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
attribution.test.ts:12:1 › attribution search enabled in chat: vscode\test\fixtures\mock-server.ts#L661
1) attribution.test.ts:12:1 › attribution search enabled in chat ───────────────────────────────── Error: listen EACCES: permission denied 0.0.0.0:49300 at ..\fixtures\mock-server.ts:661 659 | }); 660 | > 661 | const server = app.listen(SERVER_PORT); | ^ 662 | 663 | // Calling close() on the server only stops accepting new connections 664 | // and does not terminate existing connections. This can result in at Function.listen (D:\a\cody\cody\node_modules\.pnpm\express@4.18.2\node_modules\express\lib\application.js:635:24) at Function.run (D:\a\cody\cody\vscode\test\fixtures\mock-server.ts:661:28) at Object.exports.test._test.test.extend.extend.extend.extend.extend.extend.extend.extend.extend.server.auto (D:\a\cody\cody\vscode\test\e2e\helpers.ts:165:28)
test-e2e (windows)
Error: ENOENT: no such file or directory, scandir 'D:\a\cody\cody\playwright'
test-e2e (ubuntu)
Process completed with exit code 1.
initial-context.test.ts:46:18 › initial context - file: vscode/test/e2e/initial-context.test.ts#L68
4) initial-context.test.ts:46:18 › initial context - file ──────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 0 + Received + 1 Array [ "main.c", + "main.c:1-3", "myrepo", ] Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 3 elements - locator resolved to 3 elements - locator resolved to 3 elements - locator resolved to 3 elements - locator resolved to 3 elements - locator resolved to 3 elements - locator resolved to 3 elements - locator resolved to 3 elements 66 | 67 | await clickEditorTab(page, 'main.c') > 68 | await expect(chatInputMentions(lastChatInput)).toHaveText(['main.c', 'myrepo']) | ^ 69 | 70 | // After typing into the input, it no longer updates the initial context. 71 | await lastChatInput.press('x') at /home/runner/work/cody/cody/vscode/test/e2e/initial-context.test.ts:68:52
command-edit.test.ts:121:1 › edit (fixup) input - range selection: vscode/test/e2e/command-edit.test.ts#L150
3) command-edit.test.ts:121:1 › edit (fixup) input - range selection ───────────────────────────── Error: locator.click: Target page, context or browser has been closed Call log: - waiting for getByLabel('symbol-method fizzbuzz') - locator resolved to <div role="option" data-index="0" aria-level="1" aria-setsize="3" id="list_id_6_0" aria-posinset="1" draggable="false" data-parity="even" data-last-element="false" class="monaco-list-row focused" aria-label="symbol-method fizzbuzz, symbols">…</div> - attempting click action - waiting for element to be visible, enabled and stable - element is not stable - retrying click action, attempt #1 - waiting for element to be visible, enabled and stable - element is not visible - retrying click action, attempt #2 - waiting 20ms 148 | await symbolitem.click() 149 | const selectedSymbolItem = page.getByLabel('symbol-method fizzbuzz') > 150 | await selectedSymbolItem.click() | ^ 151 | 152 | // Check that the range input updated correctly to reflect the selected symbol 153 | const inputBox = page.getByPlaceholder(/^Enter edit instructions \(type @ to include code/) at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:150:30
command-edit.test.ts:121:1 › edit (fixup) input - range selection: vscode/test/e2e/command-edit.test.ts#L139
3) command-edit.test.ts:121:1 › edit (fixup) input - range selection ───────────────────────────── Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements: 1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block') 2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' }) Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 137 | // Check the correct range item is auto-selected 138 | const rangeItem = page.getByText('Nearest Code Block') > 139 | expect(rangeItem).toBeVisible() | ^ 140 | 141 | // Open the range input and check it has the correct item selected 142 | await rangeItem.click() at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:139:23
command-edit.test.ts:33:3 › edit (fixup) task: vscode/test/e2e/command-edit.test.ts#L53
2) command-edit.test.ts:33:3 › edit (fixup) task ───────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a') - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" 51 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a') 52 | await nap() > 53 | expect(modelItem).toBeVisible() | ^ 54 | 55 | // Open the model input and check it has the correct item selected 56 | await modelItem.click() at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:53:23
command-edit.test.ts:33:3 › edit (fixup) task: vscode/test/e2e/command-edit.test.ts#L53
1) command-edit.test.ts:33:3 › edit (fixup) task ───────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a') - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" - locator resolved to <a class="label-name">…</a> - unexpected value "hidden" 51 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a') 52 | await nap() > 53 | expect(modelItem).toBeVisible() | ^ 54 | 55 | // Open the model input and check it has the correct item selected 56 | await modelItem.click() at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:53:23
test-e2e (windows)
The process 'C:\Program Files\Git\bin\git.exe' failed with exit code 128
test-e2e (windows)
No files were found with the provided path: playwright/. No artifacts will be uploaded.
test-e2e (ubuntu)
The process '/usr/bin/git' failed with exit code 128
🎭 Playwright Run Summary
164 failed attribution.test.ts:12:1 › attribution search enabled in chat ────────────────────────────────── attribution.test.ts:37:1 › attribution search disabled in chat ───────────────────────────────── attribution.test.ts:12:1 › attribution search enabled in chat ────────────────────────────────── attribution.test.ts:37:1 › attribution search disabled in chat ───────────────────────────────── attribution.test.ts:12:1 › attribution search enabled in chat ────────────────────────────────── attribution.test.ts:37:1 › attribution search disabled in chat ───────────────────────────────── attribution.test.ts:12:1 › attribution search enabled in chat ────────────────────────────────── attribution.test.ts:37:1 › attribution search disabled in chat ───────────────────────────────── auth.test.ts:25:3 › requires a valid auth token and allows logouts ───────────────────────────── auth.test.ts:72:7 › test enterprise customers cannot log into dotcomUrl ──────────────────────── auth.test.ts:101:7 › logs out the user when userShouldUseEnterprise is set to true ───────────── auth.test.ts:25:3 › requires a valid auth token and allows logouts ───────────────────────────── auth.test.ts:72:7 › test enterprise customers cannot log into dotcomUrl ──────────────────────── auth.test.ts:101:7 › logs out the user when userShouldUseEnterprise is set to true ───────────── auth.test.ts:25:3 › requires a valid auth token and allows logouts ───────────────────────────── auth.test.ts:72:7 › test enterprise customers cannot log into dotcomUrl ──────────────────────── auth.test.ts:101:7 › logs out the user when userShouldUseEnterprise is set to true ───────────── auth.test.ts:25:3 › requires a valid auth token and allows logouts ───────────────────────────── auth.test.ts:72:7 › test enterprise customers cannot log into dotcomUrl ──────────────────────── auth.test.ts:101:7 › logs out the user when userShouldUseEnterprise is set to true ───────────── chat-atFile.test.ts:49:7 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:232:3 › editing a chat message with @-mention ────────────────────────────── chat-atFile.test.ts:278:3 › @-mention file range ─────────────────────────────────────────────── chat-atFile.test.ts:316:3 › @-mention symbol in chat ─────────────────────────────────────────── chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-atFile.test.ts:49:7 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:232:3 › editing a chat message with @-mention ────────────────────────────── chat-atFile.test.ts:278:3 › @-mention file range ─────────────────────────────────────────────── chat-atFile.test.ts:316:3 › @-mention symbol in chat ─────────────────────────────────────────── chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-atFile.test.ts:49:7 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:232:3 › editing a chat message with @-mention ────────────────────────────── chat-atFile.test.ts:278:3 › @-mention file range ─────────────────────────────────────────────── chat-atFile.test.ts:316:3 › @-mention symbol in chat ─────────────────────────────────────────── chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-atFile.test.ts:49:7 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:232:3 › editing a chat message with @-mention ────────────────────────────── chat-atFile.test.ts:278:3 › @-mention file range ─────────────────────────────────────────────── chat-atFile.test.ts:316:3 › @-mention symbol in chat ─────────────────────────────────────────── chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-context.test.ts:11:5 › chat follow
🎭 Playwright Run Summary
4 failed command-edit.test.ts:33:3 › edit (fixup) task ────────────────────────────────────────────────── command-edit.test.ts:33:3 › edit (fixup) task ────────────────────────────────────────────────── command-edit.test.ts:121:1 › edit (fixup) input - range selection ────────────────────────────── initial-context.test.ts:46:18 › initial context - file ───────────────────────────────────────── 150 skipped 406 passed (1.8h)

Artifacts

Produced during runtime
Name Size Digest
playwright-recordings ubuntu Expired
1.2 MB
sha256:64608921014af5218c2cc5f2a4387f278613c57bb7f4b059d7454edddb0b6040