test(dashboard): wait for page load before asserting in annotate tests#40739
Closed
SebTardif wants to merge 1 commit intomicrosoft:mainfrom
Closed
test(dashboard): wait for page load before asserting in annotate tests#40739SebTardif wants to merge 1 commit intomicrosoft:mainfrom
SebTardif wants to merge 1 commit intomicrosoft:mainfrom
Conversation
On Windows CI, the dashboard page navigation can take longer than the
default 5000ms expect timeout. Every annotate test grabs the dashboard
page via browser.contexts()[0].pages()[0] and immediately asserts or
clicks without waiting for the page to finish loading.
Add waitForLoadState('load') after obtaining the dashboard page in all
13 test sites. This ensures the page HTML and scripts have loaded before
any assertions run.
Member
|
That should not be necessary. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
waitForLoadState('load')after obtaining the dashboard page in all 13 annotate test sites. The tests grab the page viabrowser.contexts()[0].pages()[0]and immediately assert or click without waiting for the page to finish loading.Problem
The annotate tests are the dominant source of MCP CI failures on Windows. Every failing test shows the same error in CI logs:
The page hasn't finished navigating when the assertion starts. On Linux/macOS, the dashboard loads within the 5000ms default expect timeout. On Windows CI, it doesn't.
CI failure statistics (last 14 MCP workflow runs)
Data from runs 25543907347 through 25579035284 (May 8, 2026):
navigation to finishin logsThe annotate failures are spread across all Windows browser projects (chrome, chromium, msedge, webkit, firefox) and hit nearly every annotate test:
should abort MCP annotation when last screenshot is removedshould capture multiple screenshots in one annotationshould cancel browser_annotate when the MCP request is abortedshould enter annotate mode on fresh dashboard.tsx mountuser-initiated annotate downloads zip with feedback.mdshould abort annotation when last screenshot is removedshould cancel browser_annotate when the MCP client disconnectsshould annotate via direct browser_annotate MCP callshould annotate when context has no fixed viewportRoot cause
Every annotate test follows this pattern:
connectToDashboardconnects to the Playwright server but does not wait for any page to load. The test grabs the first page and immediately starts asserting. Playwright'sexpect().toBeVisible()retries within its 5000ms timeout, but if the page navigation itself hasn't completed (the dashboard is a React SPA that needs to load HTML, JS bundles, and render), the locator search fails because the DOM is empty.The
McpError: Connection closederrors reported alongside these failures are a consequence, not the cause: when the assertion times out and the test fails, the framework kills the MCP server during cleanup, which closes the connection on the pendingbrowser_annotatecall.Fix
Add
await dashboard.waitForLoadState('load')after everybrowser.contexts()[0].pages()[0]call (13 sites). This ensures the page HTML and scripts have loaded before any assertions or interactions begin.Related: #40729 (bumps timeout for one specific assertion), #40348 (fixes a different annotate race), #40296 (attempted IPC fix, closed).