Playground: fix referenceImage fields, re-enable 7 tests#1719
Merged
bkaradzic-microsoft merged 1 commit intoMay 28, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes Playground visual-test catalog entries so they point to existing canonical reference PNGs, while re-enabling the subset that now validates and keeping known failures excluded with clearer tracking reasons.
Changes:
- Updates 35
referenceImagevalues in Playgroundconfig.json. - Re-enables 7 tests by removing stale automatic-test exclusions.
- Refreshes 3 reference PNGs to RGBA-encoded versions expected by the native validation path.
Reviewed changes
Copilot reviewed 1 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
Apps/Playground/Scripts/config.json |
Corrects reference image names and updates exclusion reasons for affected Playground tests. |
Apps/Playground/ReferenceImages/Asset-Container-Instantiate-to-Scene.png |
Refreshes reference PNG encoding for validation compatibility. |
Apps/Playground/ReferenceImages/Asset-Container-Instantiate-to-Scene-2.png |
Refreshes reference PNG encoding for validation compatibility. |
Apps/Playground/ReferenceImages/wgsl-in-shadermaterial.png |
Refreshes reference PNG encoding for validation compatibility. |
Update 35 entries' `referenceImage` field in `Apps/Playground/Scripts/config.json` to the canonical filenames on disk (hyphen-slugified names, full `.png` extension, exact case). Most were field-only mismatches against PNGs that were already present in `Apps/Playground/ReferenceImages/`; the others mapped to PNGs that exist on disk under a different spelling. The renderer does an exact-string XHR GET for the file, so any field mismatch short-circuits the test at the `MISSING_REFERENCE_IMAGE:` pre-flight check in `validation_native.js`. Re-enable 7 tests that pass with the corrected references (drop `excludeFromAutomaticTesting` + stale `reason`): idx 115 -- OpenPBR Base Weight pixelDiff=1254 idx 116 -- OpenPBR Base Diffuse Roughness Analytical Lights pixelDiff=1346 idx 360 -- Asset Container Instantiate to Scene pixelDiff= 810 idx 361 -- Asset Container Instantiate to Scene 2 pixelDiff= 270 idx 713 -- Depth Renderer and RTT - debug markers pixelDiff= 974 idx 717 -- Unbalanced push pop - debug markers pixelDiff= 178 idx 718 -- Pending pop - debug markers pixelDiff= 121 (all sub-threshold under default threshold=25, errorRatio=2.5; same pattern as PR BabylonJS#1693) Rewrite `reason` for the remaining 28 tests with self-contained technical descriptions matching observed failures (no more misleading "No matching reference image found" or "Requires WebGL2-only feature"): Cubemap load not implemented on Babylon Native (2) CDF renderer / IBL prefiltering fails with 'FLOAT' undefined (6) Pixel diff: ~99% of pixels differ vs reference (1) Loading remote https:// assets is not supported (1) Chakra parse failure on modern JS syntax (14) FrameGraph render pipeline hangs past 120s (2) NativeEngine missing createStorageBuffer for WGSL ShaderMaterial (1) ReflectionProbe: 'TEXTURE_CUBE_MAP' undefined on NativeEngine (1) Refresh 3 stale PNGs (`Asset-Container-Instantiate-to-Scene.png`, `Asset-Container-Instantiate-to-Scene-2.png`, `wgsl-in-shadermaterial.png`) with the upstream Babylon.js RGBA-encoded versions. BN renders RGBA, so 3-byte/px RGB references throw `Reference data length ... must match render data length` in `compare()`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
a30b873 to
e40aadb
Compare
CedricGuillemet
approved these changes
May 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Update 35 entries'
referenceImagefield inApps/Playground/Scripts/config.jsonto the canonical filenames on disk (hyphen-slugified names, full.pngextension, exact case). Most were field-only mismatches against PNGs that were already present inApps/Playground/ReferenceImages/; the others mapped to PNGs that exist on disk under a different spelling. The renderer does an exact-string XHR GET for the file, so any field mismatch short-circuits the test at theMISSING_REFERENCE_IMAGE:pre-flight check invalidation_native.js.What's in this PR
1.
Apps/Playground/Scripts/config.jsonreferenceImageupdated to the canonical PNG name.excludeFromAutomaticTesting+ stalereasondropped (same pattern as Re-enable 3 stale-excluded Playground tests (345, 346, 533). #1693).reasonrewritten with a self-contained technical description (instead of misleading text like "No matching reference image found" or "Requires WebGL2-only feature").2. Three PNGs refreshed
Asset-Container-Instantiate-to-Scene.png,Asset-Container-Instantiate-to-Scene-2.png,wgsl-in-shadermaterial.pngwere previously committed RGB-encoded; replaced with their upstream Babylon.js RGBA-encoded counterparts. BN renders RGBA, so 3-byte/px RGB references throwReference data length ... must match render data lengthincompare().The other 32 PNGs already in the repo matched byte-for-byte; only the
referenceImagefield needed to change.Verification
Each re-enabled test was run by default (no
--include-excluded) on Win32 V8 D3D11 Debug:All sub-threshold under default
threshold=25,errorRatio=2.5; max 1346 of 240000 pixels. No custom margin needed.Each of the 28 failing entries was confirmed to skip by default (
skipped=1) and reach the documented JS/native error under--include-excluded.Failing-test buckets
These categories are individually tracked outside this PR; each
reasonstring maps 1:1 to a known issue.