[draft] Combined: bgfx MSAA+mipmaps fix + Canvas FB-pool survivability (CI smoke)#1715
Draft
bkaradzic-microsoft wants to merge 5 commits into
Draft
Conversation
The Canvas polyfill creates a fresh bgfx framebuffer per JS Canvas instance and per text-rendering operation. Under V8's GC pacing, finalizers (which release the framebuffers) only run in batches, so a long playground sweep can outpace the default 128-slot bgfx pool. The first allocation past the cliff returned BGFX_INVALID_HANDLE, the bare assert() at Canvas.cpp tripped (Debug) or the invalid handle silently bled into Graphics::FrameBuffer/bgfx-internal arrays (Release, later AV at bgfx_p.h:5526). The whole run aborted regardless of which test happened to be unlucky. Three changes: - Dependencies/CMakeLists.txt now propagates BGFX_CONFIG_MAX_FRAME_BUFFERS to the bgfx target as a compile definition, defaulting to 512 (4x the bgfx default) and honoring any -D override the way CI workflows already pass it. Previously the CI -D was a no-op. - Canvas.cpp UpdateRenderTarget() now throws std::runtime_error when createFrameBuffer returns kInvalidHandle and frees the two textures it just allocated. Context::Flush wraps the call and rethrows as Napi::Error so the offending test fails cleanly without taking the rest of the sweep down. - FrameBufferPool::Add gets the same treatment for symmetry. The pool bump alone is expected to remove the cascade in practice; the throw paths are defense-in-depth so future regressions surface as a catchable JS error instead of a silent process kill. Refs BabylonJS/BabylonNative cascade-victim discussion (113 playground tests previously excluded with reasons like "Suspected to corrupt state causing later D3D11 cascade crash").
6d2a348 to
b467942
Compare
Re-enables 112 tests previously excluded with "Suspected to corrupt state causing later Win32 D3D11 cascade crash" reasons, now that PR BabylonJS#1573 (Canvas framebuffer pool survivability fix) makes the in-suite cascade failure mode survivable. For 21 of the 112 cascade-victims whose original reason also mentioned Linux/OpenGL or D3D11 issues, re-add "excludedGraphicsApis: ["OpenGL"]" or keep "excludeFromAutomaticTesting: true" so they stay excluded on the affected backend(s): OpenGL-only exclusion (15 tests): 47 Fresnel, 54 Scissor test w/ 0.9 HW scaling, 101 Gaussian Splatting viewports, 137 Volumetric Light Scattering PP w/ Morph Targets, 159 Node material 0, 161 Node material 2, 163 Node material 4, 165 Node material 6, 166 Node material 7, 189 Fresnel, 240 Depth of field, 258 Local cubemaps, 312 Scissor test w/ 0.9 HW scaling, 378 SerializeScene + ImportMesh w/ MorphTargetManager, 385 Command encoder order in WebGPU 2. Excluded on both D3D11 and OpenGL (6 tests): 358 Roundtrip babylon file (skeletal/morph), 397 simple-sphere-in-4- mirrors, 398 procedural-texture-with-node-material, 400 simple- custom-shader, 409 lens-flare, 410 change-texture-of-material. Also fixes Dependencies/CMakeLists.txt:62 to guard the BGFX_CONFIG_MAX_ FRAME_BUFFERS bump with "if(NOT X)" instead of "if(NOT DEFINED X)" since bgfx.cmake declares the variable as "" CACHE STRING (defined but empty). Without this fix, all non-Win32 CI builds compiled with an empty -D flag and broke bgfx_p.h template instantiation.
b467942 to
9dac136
Compare
bkaradzic-microsoft
added a commit
to BabylonJS/bgfx
that referenced
this pull request
May 22, 2026
The SortKey struct in src/bgfx_p.h declared its members with no inline
initializer and relied on every caller to invoke reset() before reading
m_hasAlphaRef, m_blend, etc. via encodeDraw(). On at least one code path
exercised by BabylonNative's Playground sweep, encodeDraw(SortSequence)
runs before reset() / setState(), causing UndefinedBehaviorSanitizer to
fire with:
bgfx_p.h:1339:42: runtime error: load of value 190, which is not a
valid value for type 'bool'
at the m_hasAlphaRef read. Add in-class default initializers to all six
members so a default-constructed SortKey is in a valid (zeroed) state.
The reset() method continues to work unchanged.
Reported on the application side as BabylonJS/BabylonNative#1715
(MacOS_Sanitizers job).
920b18e to
a720c85
Compare
…xes (BabylonJS#1714). bgfx.cmake (BabylonJS) branch: bn-bump-bgfx-msaa-mips-and-ubsan @ 4a53db5 - bgfx submodule -> 89dcbaea (b617ec8f MSAA+mips + cherry-picked upstream bgfx PR #3688 UBSan: SortKey reset() in EncoderImpl ctor + BitMask shift-by-width guard) - bx and bimg unchanged from BabylonJS/bgfx.cmake master
a720c85 to
1c1942e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Draft PR for CI smoke-test only. Combines the two weekend feature branches into one tree to exercise all platform builds at once. Do not review or land from here — splits below.
This combined branch contains two independent fixes:
MSAA + mipmaps fails on D3D11/D3D12/Vulkan (bgfx) #1714 — bgfx MSAA + mipmaps fix (
bn-fix-msaa-mipsonBabylonJS/bgfx, picked up via bgfx.cmake bump). SetsMipLevels=1on the MSAA target texture in all four backends (D3D11, D3D12, Vulkan, Metal); the resolve target still getsm_numMips. Without this, MSAA-enabled render targets that also request mipmaps fail validation (E_INVALIDARGon D3D11/12, VUID on Vulkan, MTL validation on Metal). Three-level pin chain:BabylonJS/bgfxbranchbn-fix-msaa-mips@b617ec8f4BabylonJS/bgfx.cmakebranchbn-bump-bgfx-msaa-mips@2bd6f6214(advances bgfx submodule)CMakeLists.txtGIT_TAGfor bgfx.cmake to2bd6f6214Canvas framebuffer-pool survivability fix (cascade root cause). Many playground tests were marked
excludeFromAutomaticTestingwith reasons like "Suspected to cause Win32 D3D11 cascade crash". Empirically: the cascade is V8 GC pacing outpacing bgfx's default 128-slot framebuffer pool; whichever test ran when the pool finally exhausted got blamed, varying from run to run, so dozens of innocent tests accumulated the exclusion.This branch:
BGFX_CONFIG_MAX_FRAME_BUFFERSfrom 128 → 512 inDependencies/CMakeLists.txtand propagates CMake-Doverrides to bgfx's compile defines (the existing CI-D BGFX_CONFIG_MAX_FRAME_BUFFERS=256was previously a silent no-op).assert(handle.idx != bgfx::kInvalidHandle)sites withstd::runtime_errorthrows + texture cleanup, so pool exhaustion no longer abort()s the process. The CanvasContext::FlushJS binding now catches and rethrows asNapi::Errorso just the offending test fails instead of killing the whole sweep.config.jsonthat were marked excluded as cascade victims. All 112 verified to pass when run sequentially as one batch on Win32 D3D11 Release with the FB pool bump.CI expectations
This is the first time these two changes have been exercised on the non-Win32-D3D11 backends. Things to watch for:
renderer_*.cpp.--include-excluded. The 112 re-enabled tests' newvalidatedlines should appear in the sweep output. A test that genuinely fails on a non-Win32-D3D11 backend (real rendering bug, not cascade) will show up as a new pixel-diff or JS error in that backend's CI log.