Skip to content

e2e-flakiness-detector #222

e2e-flakiness-detector

e2e-flakiness-detector #222

Triggered via schedule October 17, 2024 14:04
Status Failure
Total duration 1h 29m 40s
Artifacts 2
Matrix: test-e2e
Fit to window
Zoom out
Zoom in

Annotations

20 errors and 2 notices
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode\test\e2e\helpers.ts#L377
1) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce53wV0S after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce53wV0S\Crashpad\reports\3e4e570f-ff26-4aa6-9ae8-fa9868bc9ce7.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L369
2) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 2 + Received + 0 Array [ - "buzz.ts", - "buzz.ts:2-5", "buzz.ts:7-10", ] Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element 367 | await selectLineRangeInEditorTab(page, 7, 10) 368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat') > 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10']) | ^ 370 | }) 371 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:369:52
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L1
2) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\b596e5fb3f82d72d05c32222aa9d920e.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_0_retry_0_failure.webm'
chat-context.test.ts:11:5 › chat followup context: vscode\test\e2e\helpers.ts#L377
3) chat-context.test.ts:11:5 › chat followup context ───────────────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceV6VFTA after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceV6VFTA\Crashpad\reports\1a5495ed-b23b-4569-be48-686bf254802e.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-input.test.ts:194:3 › chat model selector: vscode\test\e2e\helpers.ts#L377
4) chat-input.test.ts:194:3 › chat model selector ──────────────────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceQ0P73X after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceQ0P73X\Crashpad\reports\4548a136-b858-4bc9-9c8d-ed7eb4292cb8.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-input.test.ts:194:3 › chat model selector: vscode\test\e2e\helpers.ts#L377
5) chat-input.test.ts:194:3 › chat model selector ──────────────────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscekIix8G after 5 attempts: Error: EPERM: operation not permitted, lstat '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscekIix8G\User\globalStorage\sourcegraph.cody-ai\symf\symf-v0.0.16-x86_64-windows' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection: vscode\test\e2e\helpers.ts#L377
6) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ───────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceXQYDtV after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceXQYDtV\Crashpad\reports\da9ecef2-851c-46ce-b593-d90a0b2099a9.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users: vscode\test\e2e\helpers.ts#L377
7) chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ───────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscexTgk97 after 5 attempts: Error: EPERM: operation not permitted, lstat '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscexTgk97\User\globalStorage\sourcegraph.cody-ai\symf\symf-v0.0.16-x86_64-windows' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users: vscode\test\e2e\helpers.ts#L377
8) chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ───────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscebjMnRs after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscebjMnRs\Crashpad\reports\4e28fe4b-b1aa-4875-9bef-6f61499f1ac2.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users: vscode\test\e2e\helpers.ts#L377
9) chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ───────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceut1CBt after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceut1CBt\Crashpad\reports\d74fa03e-0bc2-4fc6-873f-7e8b3323e0ca.dmp' at helpers.ts:377 375 | } catch (error) { 376 | if (attempts === 1) { > 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 378 | } 379 | 380 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode/test/e2e/chat-atFile.test.ts#L89
1) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) 87 | await chatInput.fill('') 88 | await chatInput.pressSequentially('@mj', { delay: 350 }) > 89 | await expect(chatPanelFrame.getByRole('option', { name: 'Main.java' })).toBeVisible() | ^ 90 | await chatInput.fill('clear') 91 | 92 | // Searching and clicking at /home/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:89:77
chat-atFile.test.ts:35:3 › @-mention file in chat: vscode/test/e2e/chat-atFile.test.ts#L89
2) chat-atFile.test.ts:35:3 › @-mention file in chat ───────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' }) 87 | await chatInput.fill('') 88 | await chatInput.pressSequentially('@mj', { delay: 350 }) > 89 | await expect(chatPanelFrame.getByRole('option', { name: 'Main.java' })).toBeVisible() | ^ 90 | await chatInput.fill('clear') 91 | 92 | // Searching and clicking at /home/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:89:77
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection: vscode/test/e2e/chat-keyboard-shortcuts.test.ts#L1
3) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ───────────────────────── Test timeout of 30000ms exceeded.
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection: vscode/test/e2e/common.ts#L179
3) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ───────────────────────── Error: locator.fill: Target page, context or browser has been closed Call log: - waiting for getByPlaceholder('Type the name of a command to run.') - locator resolved to <input wrap="off" type="text" class="input" role="combobox" autocorrect="off" spellcheck="false" aria-label="input" custom-hover="true" autocapitalize="off" aria-haspopup="menu" aria-expanded="true" aria-autocomplete="list" aria-controls="quickInput_list" aria-activedescendant="list_id_4_0" aria-describedby="quickInput_message" placeholder="Type the name of a command to run."/> - fill("buzz.ts") - attempting fill action - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #1 - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #2 - waiting 20ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #3 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #4 - waiting 100ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #5 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #6 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #7 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #8 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #9 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #10 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #11 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #12 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #13 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #14 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #15 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #16 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #17 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #18 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #19 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible - retrying fill action, attempt #20 - waiting 500ms - waiting for element to be visible, enabled and editable - element is not visible
command-core.test.ts:106:3 › Explain Command from Prompts Tab: vscode/test/e2e/command-core.test.ts#L1
4) command-core.test.ts:106:3 › Explain Command from Prompts Tab ───────────────────────────────── Test timeout of 30000ms exceeded.
command-core.test.ts:106:3 › Explain Command from Prompts Tab: vscode/test/e2e/command-core.test.ts#L127
4) command-core.test.ts:106:3 › Explain Command from Prompts Tab ───────────────────────────────── Error: locator.click: Target page, context or browser has been closed Call log: - waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('option', { name: 'Explain Code' }) 125 | const sidebarTab = sidebarChat.getByTestId('tab-prompts') 126 | await sidebarTab.click() > 127 | await sidebarChat.getByRole('option', { name: 'Explain Code' }).click() | ^ 128 | 129 | // Click on a command from the sidebar should not start a new Editor window when sidebar is empty. 130 | await expect(sidebarChat.getByText('hello from the assistant')).toBeVisible() at /home/runner/work/cody/cody/vscode/test/e2e/command-core.test.ts:127:69
command-core.test.ts:106:3 › Explain Command from Prompts Tab: vscode/test/e2e/command-core.test.ts#L1
5) command-core.test.ts:106:3 › Explain Command from Prompts Tab ───────────────────────────────── Test timeout of 30000ms exceeded.
command-core.test.ts:106:3 › Explain Command from Prompts Tab: vscode/test/e2e/command-core.test.ts#L127
5) command-core.test.ts:106:3 › Explain Command from Prompts Tab ───────────────────────────────── Error: locator.click: Target page, context or browser has been closed Call log: - waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('option', { name: 'Explain Code' }) 125 | const sidebarTab = sidebarChat.getByTestId('tab-prompts') 126 | await sidebarTab.click() > 127 | await sidebarChat.getByRole('option', { name: 'Explain Code' }).click() | ^ 128 | 129 | // Click on a command from the sidebar should not start a new Editor window when sidebar is empty. 130 | await expect(sidebarChat.getByText('hello from the assistant')).toBeVisible() at /home/runner/work/cody/cody/vscode/test/e2e/command-core.test.ts:127:69
command-edit.test.ts:95:1 › edit (fixup) input - range selection: vscode/test/e2e/command-edit.test.ts#L113
6) command-edit.test.ts:95:1 › edit (fixup) input - range selection ────────────────────────────── Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements: 1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block') 2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' }) Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 111 | // Check the correct range item is auto-selected 112 | const rangeItem = page.getByText('Nearest Code Block') > 113 | expect(rangeItem).toBeVisible() | ^ 114 | 115 | // Open the range input and check it has the correct item selected 116 | await rangeItem.click() at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:113:23
command-edit.test.ts:95:1 › edit (fixup) input - range selection: vscode/test/e2e/command-edit.test.ts#L113
7) command-edit.test.ts:95:1 › edit (fixup) input - range selection ────────────────────────────── Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements: 1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block') 2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' }) Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByText('Nearest Code Block') 111 | // Check the correct range item is auto-selected 112 | const rangeItem = page.getByText('Nearest Code Block') > 113 | expect(rangeItem).toBeVisible() | ^ 114 | 115 | // Open the range input and check it has the correct item selected 116 | await rangeItem.click() at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:113:23
🎭 Playwright Run Summary
19 failed chat-atFile.test.ts:35:3 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-context.test.ts:11:5 › chat followup context ────────────────────────────────────────────── chat-input.test.ts:194:3 › chat model selector ───────────────────────────────────────────────── chat-input.test.ts:194:3 › chat model selector ───────────────────────────────────────────────── chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ────────────────────────── chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ────────────────── chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ────────────────── chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ────────────────── command-commit.test.ts:24:3 › use terminal output as context ─────────────────────────────────── command-edit.test.ts:95:1 › edit (fixup) input - range selection ─────────────────────────────── command-edit.test.ts:95:1 › edit (fixup) input - range selection ─────────────────────────────── enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error initial-context.test.ts:83:82 › chat try-again actions ───────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── uninstall.test.ts:7:5 › uninstall extension ──────────────────────────────────────────────────── 12 skipped 157 passed (40.1m)
🎭 Playwright Run Summary
7 failed chat-atFile.test.ts:35:3 › @-mention file in chat ────────────────────────────────────────────── chat-atFile.test.ts:35:3 › @-mention file in chat ────────────────────────────────────────────── chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ────────────────────────── command-core.test.ts:106:3 › Explain Command from Prompts Tab ────────────────────────────────── command-core.test.ts:106:3 › Explain Command from Prompts Tab ────────────────────────────────── command-edit.test.ts:95:1 › edit (fixup) input - range selection ─────────────────────────────── command-edit.test.ts:95:1 › edit (fixup) input - range selection ─────────────────────────────── 30 skipped 433 passed (1.5h)

Artifacts

Produced during runtime
Name Size
playwright-recordings ubuntu
5.22 MB
playwright-recordings windows
860 KB