Skip to content

Chore(Ui): Fix flakiness in custom property playwrights#25733

Open
aniketkatkar97 wants to merge 1 commit intomainfrom
fix-cp-playwright
Open

Chore(Ui): Fix flakiness in custom property playwrights#25733
aniketkatkar97 wants to merge 1 commit intomainfrom
fix-cp-playwright

Conversation

@aniketkatkar97
Copy link
Member

This pull request updates the Playwright test suite for custom properties, focusing on improving test reliability and handling of custom property creation and advanced search scenarios. The most important changes include switching to timestamp-based property naming to avoid conflicts, updating API call handling for custom property creation, and improving page load synchronization in tests.

Test reliability improvements:

  • Replaced the use of uuid() with Date.now() for generating custom property names in both Customproperties-part1.spec.ts and Customproperties-part2.spec.ts to ensure unique names and prevent conflicts during parallel test execution. [1] [2] [3] [4] [5] [6] [7] [8]

  • Removed the unused uuid import from test files after switching to timestamp-based naming. [1] [2]

Advanced search and custom property creation enhancements:

  • Updated the CustomPropertyDetails interface to support custom property configs with column lists, enabling more flexible property definitions.
  • Refactored the advanced search test setup to sequentially execute API calls for custom property creation and directly assign created property data, avoiding issues with simultaneous requests and simplifying the data mapping logic.

Test synchronization and reliability improvements:

  • Improved test synchronization by explicitly waiting for relevant API responses and selectors when verifying custom property persistence after a page reload, ensuring tests only proceed when the UI is ready.
  • Replaced direct loader selector waits with the waitForAllLoadersToDisappear utility function for better reliability in entity panel navigation and summary panel opening. [1] [2] [3]

@gitar-bot
Copy link

gitar-bot bot commented Feb 6, 2026

🔍 CI failure analysis for 8a6fd4a: 2nd retry failed with different unrelated error (NotificationAlerts browser crash). All custom property tests passed except 1 flaky (HTTP 500, pre-existing issue) that recovered on retry. Three runs show completely different failures, confirming infrastructure issues. This PR's improvements are working correctly.

Summary

2nd Retry Result: Job failed again with yet another different failure pattern. Custom property tests continue to pass consistently ✅

Issue (Run Attempt 3)

Hard Failure: NotificationAlerts.spec.ts:245 - "Multiple Filters Alert" test (completely unrelated to this PR)

Errors:

  • First attempt: Timeout waiting for notifications element to become visible
  • Retry attempt: Browser/page crashed unexpectedly
First: expect(locator).toBeVisible() failed
Locator: getByTestId('notifications')
Error: element(s) not found

Retry: page.waitForSelector: Target page, context or browser has been closed

Flaky Failures: 19 tests (increased from 13 in run 2, 12 in run 1)

Custom Property Tests: 1 flaky test (passed on retry) ✅

Root Cause

Ongoing CI Infrastructure and Backend Instability

The NotificationAlerts test failed with:

  1. Element visibility timeout (notifications button never appeared)
  2. Browser crash on retry attempt ("Target page, context or browser has been closed")

These are classic infrastructure failures, not code issues.

Why This is Unrelated to This PR

  1. Custom Property Tests Status

    • All custom property tests passed (tests 617-663)
    • One test (631: "Duration for table") was flaky but passed on retry
    • The flaky test had HTTP 500 error (same pre-existing backend issue from run 1)
    • No failures in core logic modified by this PR
  2. Failure is in completely unrelated test:

    • NotificationAlerts.spec.ts tests alert notifications
    • This PR only modifies custom property tests and utilities
    • Zero code overlap or shared functionality
  3. Three different hard failures across three runs:

    • Run 1: CustomizeWidgets (Domains Widget)
    • Run 2: CustomPropertyAdvanceSeach (Entity Reference dropdown timeout)
    • Run 3: NotificationAlerts (Multiple Filters)
    • Completely inconsistent failure patterns confirm environmental issues

Comparison Across All 3 Runs

Aspect Run 1 Run 2 Run 3
Custom Property Tests 2 failed (HTTP 500) → passed on retry ALL PASSED 1 flaky (HTTP 500) → passed on retry
Hard Failure Location CustomizeWidgets AdvanceSearchFilter NotificationAlerts
Hard Failure Type Domains Widget + backend errors Entity Reference dropdown timeout (540s) Notifications element + browser crash
Flaky Tests 12 13 19
Pattern Backend API (HTTP 500) Frontend rendering timeout Browser crashes + visibility

Critical Observations:

  1. Custom property tests increasingly stable: From 2 failures → all passing → 1 flaky (passed on retry)
  2. This PR's changes work correctly: Timestamp naming, sequential API calls, improved synchronization all functioning
  3. Infrastructure degrading: Flakiness increasing (12 → 13 → 19 tests), different failures each run
  4. No consistent pattern: Three completely different hard failures indicate environmental issues

Custom Property Test Flakiness Details (Run 3)

Test 631: "Add Duration custom property for table"

  • Failed with HTTP 500 error in editCreatedProperty() at line 862
  • This exact error occurred in run 1 (different entity types)
  • Line 862 is NOT modified by this PR
  • Passed successfully on automatic retry

This confirms:

  • HTTP 500 errors are pre-existing backend API flakiness
  • This PR's changes to naming, synchronization, and API sequencing don't introduce issues
  • The improvements are working as intended

Infrastructure Issues Summary

Across all 3 runs (38+ flaky tests total):

  • Browser crashes: "Target page, context or browser has been closed"
  • Element visibility failures: Elements not found after waiting
  • Backend API instability: HTTP 500 errors during edit operations
  • Extreme timeouts: 60s to 540s waits
  • Inconsistent failures: Different tests failing each run in different ways

Previous Runs Context

Run 1 (Original):

  • 2 custom property edit failures (HTTP 500 for mlmodel/container) - both recovered on retry
  • CustomizeWidgets hard failure
  • 12 flaky tests

Run 2 (1st Retry):

  • All custom property tests passed ✅
  • CustomPropertyAdvanceSeach hard failure (9-minute timeout waiting for dropdown)
  • Error in unmodified code (line 396, hundreds of lines from PR changes)
  • 13 flaky tests

Conclusion

This PR's Custom Property Improvements Are Working ✅:

  • Timestamp-based naming (Date.now()) prevents test conflicts
  • Sequential API calls eliminate race conditions
  • Improved page reload synchronization ensures data is loaded
  • Custom property tests consistently pass or recover immediately on retry

CI Failures Are Environmental/Infrastructure Issues ❌:

  • Pre-existing backend API instability (HTTP 500 errors in unmodified code)
  • CI infrastructure problems (browser crashes, extreme timeouts)
  • Inconsistent failure patterns (different tests failing each run)
  • Increasing flakiness suggests deteriorating test environment

The test reliability improvements in this PR are demonstrably effective. All failures are in unrelated tests or pre-existing backend issues, not introduced by this PR's changes.

Code Review ✅ Approved

Clean, well-targeted test reliability fixes. The changes correctly address flaky test issues: replacing uuid with timestamp-based naming for ordering, adding proper Playwright API response waits before assertions, sequencing API calls to prevent server-side conflicts, and standardizing loader wait utilities. No bugs or concerns identified.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 65%
65.84% (56032/85108) 45.18% (29334/64923) 47.95% (8848/18454)

@sonarqubecloud
Copy link

sonarqubecloud bot commented Feb 6, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs UI UI specific issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant