Skip to content

Commit e166e69

Browse files
wenytang-msCopilot
wenytang-ms
and
Copilot
authored
test: migrate e2e tests from Playwright to autotest YAML framework (#995)
* fix: support Unicode identifiers in Java class name validation (#789) Change isJavaIdentifier() regex from ASCII-only /^([a-zA-Z_$][a-zA-Z\d_$]*)$/ to Unicode-aware regex using ES2018 property escapes per JLS §3.8. This allows non-ASCII identifiers (e.g., Japanese ほげ, Chinese 中文类名, accented Ñoño) to be accepted in the New Java File and Rename wizards. Fixes #789 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: fix flaky simple project name assertion The .project file defined name '1.helloworld' but JDT LS may override it with the folder name 'simple'. Align .project name with folder name to eliminate the inconsistency. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test: use name-based lookups instead of hardcoded indices JDT LS may return additional source roots (e.g. target/generated-sources) that shift node indices. Replace hardcoded array indices with find()-by-name lookups to make tests resilient to JDT LS version differences. - Maven: find containers by name instead of [2]/[3] - Maven: check specific nodes hidden instead of exact count - Gradle: find nodes by name, increase timeout to 120s - Multi-module: add assertion guard before getChildren() Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: upgrade GitHub Actions to v4, fix multi-module test LS readiness - Upgrade actions/checkout@v2 -> @v4, actions/setup-java@v1 -> @v4, actions/setup-node@v2 -> @v4 across all 3 CI workflows - setup-java@v4 with distribution: temurin fixes macOS ARM64 JAVA_HOME - Multi-module test: add languageServerApiManager.ready() to suiteSetup so Maven module names are fully resolved before assertions - Accept folder name or .project name variants for level1 submodule Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: migrate e2e tests from Playwright to autotest YAML framework Replace the Playwright-based e2e test suite with autotest YAML test plans: - Add test/e2e-plans/java-dep-project-explorer.yaml Covers: focus Java Projects, link/unlink with editor, tree expansion - Add test/e2e-plans/java-dep-file-operations.yaml Covers: create class, create package, rename file, delete file - Update linuxUI.yml and windowsUI.yml CI workflows to: Build VSIX → clone/build autotest → run-all test plans Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: use npm install for autotest, fix rename flow for Linux CI - CI: replace git clone + build with 'npm install -g github:...' - Fix rename test: use open-file + link-with-editor instead of manual tree expansion (avoids tree hierarchy change after package creation) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: use npm registry for autotest install Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add wait before rename input, increase stability on Linux Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add LLM env vars to CI, fix rename flow stability - CI: add AZURE_OPENAI_* env vars, remove --no-llm - Rename test: click to select before context menu, add waitBefore - Rename test: add waitBefore to fillQuickInput for input box timing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: use fillAnyInput for rename (handles both inline and quick input) On Linux, context menu Rename may trigger VSCode's built-in inline tree rename instead of the extension's showInputBox. fillAnyInput detects whichever input appears and fills it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: rename uses fillAnyInput + confirmDialog, fix tab title - Rename: use fillAnyInput (handles both inline and quick input) - Rename: use confirmDialog instead of tryClickDialogButton OK - Fix verify tab title: 'AppRenamed' not 'AppRenamed.java' Local verification: 44/44 passed (31/31 + 13/13) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent e02fd0f commit e166e69

4 files changed

Lines changed: 281 additions & 6 deletions

File tree

‎.github/workflows/linuxUI.yml‎

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,10 +42,20 @@ jobs:
4242
run: npm run build-server
4343

4444
- name: Build VSIX file
45-
run: vsce package
45+
run: vsce package -o vscode-java-dependency.vsix
4646

47-
- name: E2E Test (Playwright)
48-
run: DISPLAY=:99 npm run test-e2e
47+
- name: Setup autotest
48+
run: npm install -g @vscjava/vscode-autotest
49+
50+
- name: E2E Test (autotest)
51+
env:
52+
AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
53+
AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
54+
AZURE_OPENAI_DEPLOYMENT: ${{ secrets.AZURE_OPENAI_DEPLOYMENT }}
55+
run: |
56+
DISPLAY=:99 autotest run-all test/e2e-plans \
57+
--vsix $(pwd)/vscode-java-dependency.vsix \
58+
--output test-results
4959
5060
- name: Upload test results
5161
if: ${{ always() }}

‎.github/workflows/windowsUI.yml‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,10 +42,18 @@ jobs:
4242
run: npm run build-server
4343

4444
- name: Build VSIX file
45-
run: vsce package
45+
run: vsce package -o vscode-java-dependency.vsix
4646

47-
- name: E2E Test (Playwright)
48-
run: npm run test-e2e
47+
- name: Setup autotest
48+
run: npm install -g @vscjava/vscode-autotest
49+
50+
- name: E2E Test (autotest)
51+
env:
52+
AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
53+
AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
54+
AZURE_OPENAI_DEPLOYMENT: ${{ secrets.AZURE_OPENAI_DEPLOYMENT }}
55+
run: |
56+
autotest run-all test/e2e-plans --vsix "$((Get-Location).Path)\vscode-java-dependency.vsix" --output test-results
4957
5058
- name: Upload test results
5159
if: ${{ always() }}

‎test/e2e-plans/java-dep-file-operations.yaml‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
# Test Plan: Java Dependency — File Operations
2+
#
3+
# Covers fileOperations.test.ts scenarios:
4+
# - create new Java class
5+
# - create new package
6+
# - rename Java file
7+
# - delete Java file
8+
#
9+
# Note: The workspace is auto-copied to a temp directory by autotest,
10+
# so rename/delete operations don't pollute the source test fixtures.
11+
#
12+
# Usage:
13+
# npx autotest run test/e2e-plans/java-dep-file-operations.yaml --vsix <path-to-vsix>
14+
15+
name: "Java Dependency — File Operations"
16+
description: |
17+
Tests file/resource operations in the Java Projects view:
18+
create class, create package, rename file, delete file.
19+
Replaces test/e2e/tests/fileOperations.test.ts.
20+
21+
setup:
22+
extension: "redhat.java"
23+
vscodeVersion: "stable"
24+
workspace: "../maven"
25+
timeout: 180
26+
settings:
27+
java.configuration.checkProjectSettingsExclusions: false
28+
29+
steps:
30+
# ── Setup: wait for LS, focus Java Projects ──
31+
- id: "ls-ready"
32+
action: "waitForLanguageServer"
33+
verify: "Java Language Server is ready"
34+
timeout: 180
35+
36+
- id: "focus-java-projects"
37+
action: "run command Java Projects: Focus on Java Projects View"
38+
verify: "Java Projects view is focused"
39+
40+
- id: "wait-tree-load"
41+
action: "wait 3 seconds"
42+
43+
# ── Test 1: create new Java class ──
44+
- id: "click-project-node"
45+
action: "click my-app tree item"
46+
47+
- id: "trigger-new-resource"
48+
action: "clickTreeItemAction my-app New..."
49+
verify: "New resource quick pick opened"
50+
51+
- id: "select-java-class"
52+
action: "selectOptionByIndex 0"
53+
verify: "Java Class selected"
54+
55+
- id: "select-source-folder"
56+
action: "select src/main/java option"
57+
verify: "Source folder selected"
58+
59+
- id: "enter-class-name"
60+
action: "fillQuickInput App2"
61+
verify: "Class name entered"
62+
63+
- id: "verify-new-class-tab"
64+
action: "wait 2 seconds"
65+
verify: "Editor tab App2.java should appear"
66+
verifyEditorTab:
67+
title: "App2.java"
68+
timeout: 15
69+
70+
# ── Test 2: create new package ──
71+
- id: "focus-java-projects-2"
72+
action: "run command Java Projects: Focus on Java Projects View"
73+
74+
- id: "click-project-node-2"
75+
action: "click my-app tree item"
76+
77+
- id: "trigger-new-resource-2"
78+
action: "clickTreeItemAction my-app New..."
79+
80+
- id: "select-package"
81+
action: "select Package option"
82+
verify: "Package selected"
83+
84+
- id: "select-source-folder-2"
85+
action: "select src/main/java option"
86+
87+
- id: "enter-package-name"
88+
action: "fillQuickInput com.mycompany.newpkg"
89+
verify: "Package name entered"
90+
91+
- id: "wait-package-creation"
92+
action: "wait 3 seconds"
93+
verify: "Package directory created"
94+
95+
# ── Test 3: rename Java file ──
96+
# Open the target file, let link-with-editor reveal it in the tree,
97+
# then click to select it and trigger rename via context menu.
98+
- id: "open-rename-target"
99+
action: "open file AppToRename.java"
100+
waitBefore: 3
101+
102+
- id: "focus-java-projects-3"
103+
action: "run command Java Projects: Focus on Java Projects View"
104+
waitBefore: 3
105+
106+
# Click the tree item first to select it
107+
- id: "select-rename-target"
108+
action: "click AppToRename tree item"
109+
waitBefore: 2
110+
111+
# Use context menu to trigger extension's rename (shows showInputBox)
112+
- id: "rename-context-menu"
113+
action: "contextMenu AppToRename Rename"
114+
verify: "Rename input box opened"
115+
116+
- id: "enter-new-name"
117+
action: "fillAnyInput AppRenamed"
118+
verify: "New name entered and confirmed"
119+
waitBefore: 2
120+
121+
# Handle optional rename confirmation dialog — may not appear on all platforms
122+
# (Electron native dialog is auto-dismissed by monkey-patch)
123+
- id: "handle-rename-dialog"
124+
action: "confirmDialog"
125+
verify: "Rename confirmation handled (if present)"
126+
127+
# Handle optional Refactor Preview panel
128+
- id: "handle-refactor-preview"
129+
action: "tryClickButton Apply"
130+
verify: "Refactor preview applied (if present)"
131+
132+
- id: "wait-rename"
133+
action: "wait 3 seconds"
134+
135+
- id: "verify-renamed-tab"
136+
action: "wait 1 seconds"
137+
verify: "Editor tab AppRenamed should appear"
138+
verifyEditorTab:
139+
title: "AppRenamed"
140+
timeout: 15
141+
142+
# ── Test 4: delete Java file ──
143+
# Instead of manually expanding the tree (which requires viewport space),
144+
# open the file and let link-with-editor reveal it in the tree.
145+
- id: "open-delete-target"
146+
action: "open file AppToDelete.java"
147+
waitBefore: 5
148+
149+
- id: "focus-java-projects-4"
150+
action: "run command Java Projects: Focus on Java Projects View"
151+
waitBefore: 2
152+
153+
- id: "delete-context-menu"
154+
action: "contextMenu AppToDelete Delete"
155+
verify: "Delete confirmation triggered"
156+
157+
# VSCode shows a platform-specific confirmation dialog for delete
158+
- id: "confirm-delete"
159+
action: "confirmDialog"
160+
161+
- id: "wait-delete"
162+
action: "wait 3 seconds"
163+
164+
- id: "verify-deleted"
165+
action: "wait 1 seconds"
166+
verify: "AppToDelete tree item should disappear"
167+
verifyTreeItem:
168+
name: "AppToDelete"
169+
visible: false
170+
timeout: 15

‎test/e2e-plans/java-dep-project-explorer.yaml‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Test Plan: Java Dependency — Project Explorer
2+
#
3+
# Covers projectExplorer.test.ts scenarios:
4+
# - javaProjectExplorer.focus shows Java Projects section
5+
# - linkWithFolderExplorer reveals active file in tree
6+
# - unlinkWithFolderExplorer stops auto-reveal
7+
#
8+
# Usage:
9+
# npx autotest run test/e2e-plans/java-dep-project-explorer.yaml --vsix <path-to-vsix>
10+
11+
name: "Java Dependency — Project Explorer"
12+
description: |
13+
Tests the Java Projects explorer view: focus, link/unlink with editor.
14+
Replaces test/e2e/tests/projectExplorer.test.ts.
15+
16+
setup:
17+
extension: "redhat.java"
18+
vscodeVersion: "stable"
19+
workspace: "../maven"
20+
timeout: 180
21+
settings:
22+
java.configuration.checkProjectSettingsExclusions: false
23+
24+
steps:
25+
# ── Wait for LS ready ──
26+
- id: "ls-ready"
27+
action: "waitForLanguageServer"
28+
verify: "Java Language Server is ready"
29+
timeout: 180
30+
31+
# ── Test 1: javaProjectExplorer.focus ──
32+
- id: "focus-java-projects"
33+
action: "run command Java Projects: Focus on Java Projects View"
34+
verify: "Java Projects view is focused"
35+
36+
- id: "wait-tree-load"
37+
action: "wait 3 seconds"
38+
39+
- id: "verify-project-node"
40+
action: "wait 1 seconds"
41+
verify: "my-app project node is visible in the tree"
42+
verifyTreeItem:
43+
name: "my-app"
44+
timeout: 15
45+
46+
# ── Test 2: linkWithFolderExplorer ──
47+
- id: "expand-project"
48+
action: "expand my-app tree item"
49+
50+
- id: "expand-src"
51+
action: "expand src/main/java tree item"
52+
waitBefore: 2
53+
54+
- id: "verify-package"
55+
action: "wait 1 seconds"
56+
verify: "Package com.mycompany.app is visible"
57+
verifyTreeItem:
58+
name: "com.mycompany.app"
59+
timeout: 15
60+
61+
- id: "expand-package"
62+
action: "expand com.mycompany.app tree item"
63+
waitBefore: 2
64+
65+
- id: "verify-app-class"
66+
action: "wait 1 seconds"
67+
verify: "App class node is visible (exact match, not App.java or AppToDelete)"
68+
verifyTreeItem:
69+
name: "App"
70+
exact: true
71+
timeout: 15
72+
73+
# ── Test 3: unlinkWithFolderExplorer ──
74+
- id: "unlink-editor"
75+
action: "run command Java: Unlink with Editor"
76+
verify: "Editor unlinked from tree"
77+
78+
- id: "open-rename-file"
79+
action: "open file AppToRename.java"
80+
81+
- id: "wait-after-open"
82+
action: "wait 3 seconds"
83+
verify: "Tree should not auto-expand to AppToRename"
84+
85+
- id: "relink-editor"
86+
action: "run command Java: Link with Editor"
87+
verify: "Editor re-linked with tree"

0 commit comments

Comments
 (0)