e2e-flakiness-detector #222
Annotations
20 errors and 2 notices
chat-atFile.test.ts:35:3 › @-mention file in chat:
vscode\test\e2e\helpers.ts#L377
1) chat-atFile.test.ts:35:3 › @-mention file in chat ─────────────────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce53wV0S after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce53wV0S\Crashpad\reports\3e4e570f-ff26-4aa6-9ae8-fa9868bc9ce7.dmp'
at helpers.ts:377
375 | } catch (error) {
376 | if (attempts === 1) {
> 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
378 | }
379 |
380 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L369
2) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- Expected - 2
+ Received + 0
Array [
- "buzz.ts",
- "buzz.ts:2-5",
"buzz.ts:7-10",
]
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
367 | await selectLineRangeInEditorTab(page, 7, 10)
368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat')
> 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10'])
| ^
370 | })
371 |
at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:369:52
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L1
2) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\b596e5fb3f82d72d05c32222aa9d920e.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_0_retry_0_failure.webm'
|
chat-context.test.ts:11:5 › chat followup context:
vscode\test\e2e\helpers.ts#L377
3) chat-context.test.ts:11:5 › chat followup context ─────────────────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceV6VFTA after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceV6VFTA\Crashpad\reports\1a5495ed-b23b-4569-be48-686bf254802e.dmp'
at helpers.ts:377
375 | } catch (error) {
376 | if (attempts === 1) {
> 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
378 | }
379 |
380 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
|
chat-input.test.ts:194:3 › chat model selector:
vscode\test\e2e\helpers.ts#L377
4) chat-input.test.ts:194:3 › chat model selector ────────────────────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceQ0P73X after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceQ0P73X\Crashpad\reports\4548a136-b858-4bc9-9c8d-ed7eb4292cb8.dmp'
at helpers.ts:377
375 | } catch (error) {
376 | if (attempts === 1) {
> 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
378 | }
379 |
380 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
|
chat-input.test.ts:194:3 › chat model selector:
vscode\test\e2e\helpers.ts#L377
5) chat-input.test.ts:194:3 › chat model selector ────────────────────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscekIix8G after 5 attempts: Error: EPERM: operation not permitted, lstat '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscekIix8G\User\globalStorage\sourcegraph.cody-ai\symf\symf-v0.0.16-x86_64-windows'
at helpers.ts:377
375 | } catch (error) {
376 | if (attempts === 1) {
> 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
378 | }
379 |
380 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
|
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection:
vscode\test\e2e\helpers.ts#L377
6) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ─────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceXQYDtV after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceXQYDtV\Crashpad\reports\da9ecef2-851c-46ce-b593-d90a0b2099a9.dmp'
at helpers.ts:377
375 | } catch (error) {
376 | if (attempts === 1) {
> 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
378 | }
379 |
380 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
|
chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users:
vscode\test\e2e\helpers.ts#L377
7) chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ─────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscexTgk97 after 5 attempts: Error: EPERM: operation not permitted, lstat '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscexTgk97\User\globalStorage\sourcegraph.cody-ai\symf\symf-v0.0.16-x86_64-windows'
at helpers.ts:377
375 | } catch (error) {
376 | if (attempts === 1) {
> 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
378 | }
379 |
380 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
|
chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users:
vscode\test\e2e\helpers.ts#L377
8) chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ─────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscebjMnRs after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscebjMnRs\Crashpad\reports\4e28fe4b-b1aa-4875-9bef-6f61499f1ac2.dmp'
at helpers.ts:377
375 | } catch (error) {
376 | if (attempts === 1) {
> 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
378 | }
379 |
380 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
|
chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users:
vscode\test\e2e\helpers.ts#L377
9) chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ─────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceut1CBt after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceut1CBt\Crashpad\reports\d74fa03e-0bc2-4fc6-873f-7e8b3323e0ca.dmp'
at helpers.ts:377
375 | } catch (error) {
376 | if (attempts === 1) {
> 377 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
378 | }
379 |
380 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:377:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:258:13)
|
chat-atFile.test.ts:35:3 › @-mention file in chat:
vscode/test/e2e/chat-atFile.test.ts#L89
1) chat-atFile.test.ts:35:3 › @-mention file in chat ─────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' })
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' })
87 | await chatInput.fill('')
88 | await chatInput.pressSequentially('@mj', { delay: 350 })
> 89 | await expect(chatPanelFrame.getByRole('option', { name: 'Main.java' })).toBeVisible()
| ^
90 | await chatInput.fill('clear')
91 |
92 | // Searching and clicking
at /home/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:89:77
|
chat-atFile.test.ts:35:3 › @-mention file in chat:
vscode/test/e2e/chat-atFile.test.ts#L89
2) chat-atFile.test.ts:35:3 › @-mention file in chat ─────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' })
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('option', { name: 'Main.java' })
87 | await chatInput.fill('')
88 | await chatInput.pressSequentially('@mj', { delay: 350 })
> 89 | await expect(chatPanelFrame.getByRole('option', { name: 'Main.java' })).toBeVisible()
| ^
90 | await chatInput.fill('clear')
91 |
92 | // Searching and clicking
at /home/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:89:77
|
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection:
vscode/test/e2e/chat-keyboard-shortcuts.test.ts#L1
3) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ─────────────────────────
Test timeout of 30000ms exceeded.
|
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection:
vscode/test/e2e/common.ts#L179
3) chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ─────────────────────────
Error: locator.fill: Target page, context or browser has been closed
Call log:
- waiting for getByPlaceholder('Type the name of a command to run.')
- locator resolved to <input wrap="off" type="text" class="input" role="combobox" autocorrect="off" spellcheck="false" aria-label="input" custom-hover="true" autocapitalize="off" aria-haspopup="menu" aria-expanded="true" aria-autocomplete="list" aria-controls="quickInput_list" aria-activedescendant="list_id_4_0" aria-describedby="quickInput_message" placeholder="Type the name of a command to run."/>
- fill("buzz.ts")
- attempting fill action
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #1
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #2
- waiting 20ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #3
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #4
- waiting 100ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #5
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #6
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #7
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #8
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #9
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #10
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #11
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #12
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #13
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #14
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #15
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #16
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #17
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #18
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #19
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
- retrying fill action, attempt #20
- waiting 500ms
- waiting for element to be visible, enabled and editable
- element is not visible
|
command-core.test.ts:106:3 › Explain Command from Prompts Tab:
vscode/test/e2e/command-core.test.ts#L1
4) command-core.test.ts:106:3 › Explain Command from Prompts Tab ─────────────────────────────────
Test timeout of 30000ms exceeded.
|
command-core.test.ts:106:3 › Explain Command from Prompts Tab:
vscode/test/e2e/command-core.test.ts#L127
4) command-core.test.ts:106:3 › Explain Command from Prompts Tab ─────────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('option', { name: 'Explain Code' })
125 | const sidebarTab = sidebarChat.getByTestId('tab-prompts')
126 | await sidebarTab.click()
> 127 | await sidebarChat.getByRole('option', { name: 'Explain Code' }).click()
| ^
128 |
129 | // Click on a command from the sidebar should not start a new Editor window when sidebar is empty.
130 | await expect(sidebarChat.getByText('hello from the assistant')).toBeVisible()
at /home/runner/work/cody/cody/vscode/test/e2e/command-core.test.ts:127:69
|
command-core.test.ts:106:3 › Explain Command from Prompts Tab:
vscode/test/e2e/command-core.test.ts#L1
5) command-core.test.ts:106:3 › Explain Command from Prompts Tab ─────────────────────────────────
Test timeout of 30000ms exceeded.
|
command-core.test.ts:106:3 › Explain Command from Prompts Tab:
vscode/test/e2e/command-core.test.ts#L127
5) command-core.test.ts:106:3 › Explain Command from Prompts Tab ─────────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('option', { name: 'Explain Code' })
125 | const sidebarTab = sidebarChat.getByTestId('tab-prompts')
126 | await sidebarTab.click()
> 127 | await sidebarChat.getByRole('option', { name: 'Explain Code' }).click()
| ^
128 |
129 | // Click on a command from the sidebar should not start a new Editor window when sidebar is empty.
130 | await expect(sidebarChat.getByText('hello from the assistant')).toBeVisible()
at /home/runner/work/cody/cody/vscode/test/e2e/command-core.test.ts:127:69
|
command-edit.test.ts:95:1 › edit (fixup) input - range selection:
vscode/test/e2e/command-edit.test.ts#L113
6) command-edit.test.ts:95:1 › edit (fixup) input - range selection ──────────────────────────────
Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements:
1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block')
2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' })
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByText('Nearest Code Block')
111 | // Check the correct range item is auto-selected
112 | const rangeItem = page.getByText('Nearest Code Block')
> 113 | expect(rangeItem).toBeVisible()
| ^
114 |
115 | // Open the range input and check it has the correct item selected
116 | await rangeItem.click()
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:113:23
|
command-edit.test.ts:95:1 › edit (fixup) input - range selection:
vscode/test/e2e/command-edit.test.ts#L113
7) command-edit.test.ts:95:1 › edit (fixup) input - range selection ──────────────────────────────
Error: expect.toBeVisible: Error: strict mode violation: getByText('Nearest Code Block') resolved to 2 elements:
1) <span class="monaco-highlighted-label">…</span> aka getByLabel('$(file-code) Nearest Code').getByText('Nearest Code Block')
2) <span class="monaco-highlighted-label">…</span> aka getByLabel('check file-code Nearest Code').locator('a').filter({ hasText: 'Nearest Code Block' })
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByText('Nearest Code Block')
111 | // Check the correct range item is auto-selected
112 | const rangeItem = page.getByText('Nearest Code Block')
> 113 | expect(rangeItem).toBeVisible()
| ^
114 |
115 | // Open the range input and check it has the correct item selected
116 | await rangeItem.click()
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:113:23
|
🎭 Playwright Run Summary
19 failed
chat-atFile.test.ts:35:3 › @-mention file in chat ──────────────────────────────────────────────
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ─────────────────────────────────────────
chat-context.test.ts:11:5 › chat followup context ──────────────────────────────────────────────
chat-input.test.ts:194:3 › chat model selector ─────────────────────────────────────────────────
chat-input.test.ts:194:3 › chat model selector ─────────────────────────────────────────────────
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ──────────────────────────
chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ──────────────────
chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ──────────────────
chat-rateLimit.test.ts:52:3 › shows standard rate limit message for pro users ──────────────────
command-commit.test.ts:24:3 › use terminal output as context ───────────────────────────────────
command-edit.test.ts:95:1 › edit (fixup) input - range selection ───────────────────────────────
command-edit.test.ts:95:1 › edit (fixup) input - range selection ───────────────────────────────
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error
enterprise-cody-ignore.test.ts:43:7 › using actively invoked commands and autocomplete shows a error
initial-context.test.ts:83:82 › chat try-again actions ─────────────────────────────────────────
uninstall.test.ts:7:5 › uninstall extension ────────────────────────────────────────────────────
uninstall.test.ts:7:5 › uninstall extension ────────────────────────────────────────────────────
uninstall.test.ts:7:5 › uninstall extension ────────────────────────────────────────────────────
uninstall.test.ts:7:5 › uninstall extension ────────────────────────────────────────────────────
12 skipped
157 passed (40.1m)
|
🎭 Playwright Run Summary
7 failed
chat-atFile.test.ts:35:3 › @-mention file in chat ──────────────────────────────────────────────
chat-atFile.test.ts:35:3 › @-mention file in chat ──────────────────────────────────────────────
chat-keyboard-shortcuts.test.ts:35:5 › re-opening chat adds selection ──────────────────────────
command-core.test.ts:106:3 › Explain Command from Prompts Tab ──────────────────────────────────
command-core.test.ts:106:3 › Explain Command from Prompts Tab ──────────────────────────────────
command-edit.test.ts:95:1 › edit (fixup) input - range selection ───────────────────────────────
command-edit.test.ts:95:1 › edit (fixup) input - range selection ───────────────────────────────
30 skipped
433 passed (1.5h)
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
playwright-recordings ubuntu
|
5.22 MB |
|
playwright-recordings windows
|
860 KB |
|