feat: Allow variable fields on shadow blocks#9695
Draft
B-head wants to merge 8 commits intoRaspberryPiFoundation:mainfrom
Draft
feat: Allow variable fields on shadow blocks#9695B-head wants to merge 8 commits intoRaspberryPiFoundation:mainfrom
B-head wants to merge 8 commits intoRaspberryPiFoundation:mainfrom
Conversation
The generators and compile webdriver scripts built `file://` URLs by concatenating `__dirname` directly. On Windows `__dirname` contains backslashes, which yields a malformed URL like `file://S:\source\blockly\.../tests/compile/index.html` that Chrome refuses to load — making the `generators` and `advanced_compile_in_browser` test steps fail. Route both URLs through the existing `posixPath()` helper in `scripts/helpers.js`, matching the pattern already used in `tests/mocha/webdriver.js`. The helper is a no-op on POSIX so this keeps existing Linux/macOS behaviour unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
`eslint-plugin-prettier` inherits Prettier's `endOfLine: 'lf'` default, so any source file checked out with CRLF on a Windows machine that has `core.autocrlf=true` fails lint with `Delete \`␍\`` errors. Setting `* text=auto eol=lf` makes git always check out detected text files with LF regardless of the user's local autocrlf setting, removing the class of failures at its source. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The renamings test step ran the `.mjs` file directly via execSync, relying on its `#!/usr/bin/env node` shebang. Windows `cmd.exe` does not honor shebangs and `.mjs` has no default file association, so the step failed with "is not recognized as an internal or external command". Prefix the invocation with `node` so it works on every platform. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
`tests/node/node_modules/blockly-test` is a git symlink (mode 120000) pointing at `../../../dist`. With `core.symlinks` disabled (the default on Windows) git checks it out as a 13-byte text file containing the link target literally, so `import 'blockly-test'` in `tests/node/run_node_test.mjs` fails with `ERR_MODULE_NOT_FOUND` and the `node` test step never runs. Add an `ensureBlocklyTestLink` helper that runs immediately before the node test step. It detects the broken state via `lstatSync`, replaces the file with a junction (which works on Windows without admin rights and is treated as a regular symlink on POSIX), then best-effort marks the path `skip-worktree` so the resulting "deleted symlink" diff stays out of `git status`. Unexpected failures of the skip-worktree step are surfaced as a warning rather than silently swallowed. The helper is idempotent and a no-op on Linux/macOS, where the git checkout already produces a usable symlink. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Drop the throw in FieldVariable.setSourceBlock that prevented a FieldVariable from attaching to a shadow block. This is the minimum precondition for using variables_get as a shadow under any value input; without it the field cannot even be constructed in a shadow context. This commit intentionally does NOT touch the variable deletion cascade, respawn machinery, or any other related code path. Those concerns are exercised by the accompanying test suite (tests/mocha/shadow_variable_fields_test.js) and will be addressed by a follow-up redesign. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add "Shadow Variables" and "Shadow Variables (typed)" toolbox options to
tests/playground.html that mirror the block types defined inline in
tests/mocha/shadow_variable_fields_test.js (shadow_var_parent,
shadow_var_wrapper, shadow_var_two_field) plus a small statement-style
scaffold (shadow_var_host), so an operator can reach every scenario the
spec exercises from the playground UI:
- shadow_var_parent + shadow variables_get
basic / Case 1 / Case 2 / rename / delete cascade / save-load
- shadow_var_parent + shadow shadow_var_wrapper + shadow variables_get
nested shadow
- shadow_var_parent + shadow shadow_var_two_field
mixed Case 1 / Case 2
- shadow_var_host + shadow_var_parent + shadow variables_get
shadow_var_parent attached to a real statement parent
The typed variant routes the same shapes through variables_get_dynamic
and the dynamic Variables category, exercising the FieldVariable shadow
patch on the type-enforcing onchange path.
The block types are defined inline via Blockly.defineBlocksWithJsonArray
in the existing module script of playground.html; no new files are
introduced.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add tests/mocha/shadow_variable_fields_test.js (and register it in tests/mocha/index.html) as an executable specification for the "shadow variables_get" feature. The suite encodes the quality requirements that were distilled from the design discussion driving this work: RaspberryPiFoundation#1 Public API side-effect freedom: attaching a shadow variables_get does not fire VarCreate or change getAllVariables(). RaspberryPiFoundation#2 Symmetric behavior: shadow and non-shadow variables_get on the same id share the same VariableModel; rename and code generation flow through the same path; serialization round-trips preserve the field id. RaspberryPiFoundation#3 Variable delete cascade processes shadow uses (no special skipping by isShadow()). RaspberryPiFoundation#4 Undo restores both the variable and the shadow display, with the shadow block id preserved across delete/undo cycles. RaspberryPiFoundation#5 Case 2 (parent template references the deleted variable, reached by save / load) does not crash, throw, or leave the variable map inconsistent. RaspberryPiFoundation#6 deleteVariable does not throw for inputs the patch enables. Coverage areas (29 active tests + 4 acceptance criteria as test.skip): - basic shadow attachment and identity sharing with non-shadow - no spurious VarCreate / getAllVariables churn at attachment time - Case 1 cascade reset (live shadow value differs from template) - Case 2 cascade behavior (parent template references the deleted variable, reached via save / load round-trip) - symmetry with non-shadow variables_get for rename / cascade / code generation / save-load - rename propagation and undo - load auto-creates a missing variable (parity with non-shadow) - deletion of an unrelated variable leaves the shadow alone - chained deleteVariable calls without intermediate undo - BlockChange event recording for cascade resets - nested deleteVariable from a change listener - workspace.clear() with shadow variables_get blocks - non-default variable type round-trip - mixed Case 1 / Case 2 fields in one shadow - nested shadow blocks (shadow inside shadow inside parent) The "acceptance criteria" suite collects four `test.skip` tests that encode the contract the implementation must meet. They appear as pending in mocha output and serve as the explicit acceptance gate for the redesign work that will follow this spec branch. Many of the active tests fail today (most fail with "shadow child must be attached: expected null to exist", because the default delete cascade disposes shadow uses without preserving them). The failing tests are the red side of the spec; making them green is the redesign session's charter. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
VariableMap.deleteVariable now keeps shadow variable_get blocks attached
at the same parent input by replaying each shadow's parent connection
shadowState through getOrCreateVariablePackage. The helper either
returns an existing live variable (template references a different,
still-live variable — Case 1) or re-creates a same-id same-name
variable (template references the variable being deleted — Case 2),
and setValue rebinds the field to the result. The two cases run on
opposite sides of the variable map removal so the resulting event group
unwinds cleanly:
(a) Case 1 setValue replay (BlockChange events)
(b) non-shadow uses dispose, with workspace.suppressShadowRespawn set
so Connection.disconnectInternal does not auto-spawn a fresh
shadow from a stale shadowState that still references the
doomed variable
(c) variable map removal + VarDelete fire
(d) Case 2 setValue replay (re-creates the variable, fires VarCreate
for the fresh VariableModel)
Undo unwinds in reverse:
- Case 2 VarCreate is undone (the regenerated VariableModel goes
away — the surviving field still holds a reference to it but the
block id is preserved)
- VarDelete is undone (the original variable comes back)
- non-shadow dispose events undo (their BlockCreate events restore
the disposed blocks)
- Case 1 BlockChange events undo (the field validator now finds the
restored variable on the workspace, so setValue accepts the old id)
The cascade also dedupes uses by block id (a single block with two
matching fields would otherwise appear twice in the raw uses list)
and falls back to the non-shadow dispose path for shadows that cannot
be classified (no reachable parent, no stored shadowState, no matching
field).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Allows
field_variableon shadow blocks. Specifically, this PR permitsvariable_getto be placed as a shadow block, and adjusts the connection / serialization / variable cascade paths so that variable fields on shadow blocks stay in sync with the workspace's variable map.Motivation
In a personal project, I'm defining blocks that operate on objects and that perform higher-order list operations (map / filter / reduce style blocks). Having to specify a variable explicitly every time I use one of these blocks is cumbersome. I'd like to provide blocks that come with a default variable reference preset, so users can just drop them in. Today I work around this by inserting a non-shadow variable getter, but it would be more natural UX-wise if the preset were a shadow that the user could simply replace with another variable.
For context, my own use case involves combining this with
@mit-app-inventor/blockly-block-lexical-variables. However, the changes in this PR work with stock Blockly'sfield_variablein the same way, so this is not a plugin-specific feature. That said, I have not yet verified whether this PR alone is sufficient to make the lexical-variables combination work end-to-end as intended; additional adjustments may be required.Why a draft?
I judged that ad-hoc patching was not the right way to fix this. The code in this PR is also pure AI-generated code that I have not yet reviewed in detail myself.
The patches in this PR work around several existing issues in the codebase, which makes them fragile and likely to break under future changes. The right fix would be a more fundamental design change, but most of those changes would be breaking. Rather than push that decision unilaterally, I'd like to report the issues I uncovered while building this experimental implementation and ask for guidance on how to proceed.
For transparency: the issue analysis below was put together through a dialogue with Claude Code. I have verified the substance of the findings myself, but some of the wording is left as-is from the AI.
What this PR implements
Just allowing variable fields on shadow blocks isn't enough on its own. To make a variable field on a shadow block behave equivalently to a non-shadow
variable_get, the following two consistency requirements had to be addressed:Workspace.deleteVariable, the shadow's reference must also be cleaned up so that the workspace stays consistent.To make these work, the PR routes around the existing event system / serialization / connection paths in several places. The structural reasons why those workarounds were necessary are documented in the Known issues section below.
Known issues
1. Shadow lifecycle is not a first-class undo/redo operation
Symptom: Creating and destroying a shadow block does fire
BlockCreate/BlockDeleteevents, but events_block_create.ts:48-50 and events_block_delete.ts:54-57 forcerecordUndo = falsewheneverisShadow()is true, so these events never end up on the undo stack. Field edits inside a shadow are similarly not recorded. In effect, the shadow lifecycle relies entirely on "side effects of the parent's events plus implicit respawning along the connect/disconnect path."Impact on this PR: Because shadow blocks are now allowed to carry variable fields, there is no way to undo a variable-id change inside a shadow, and the cascade for variable deletion has no choice but to ride on the parent block's events.
Suggested upstream fix: Drop the forced
recordUndo = falseon shadow events and promote the shadow lifecycle to a first-class undoable operation. However, this is tightly coupled to the meaning ofConnectionState.shadow(see section 2 below), so it can't really be fixed in isolation. I think of it as an RFC-level item for a major version (v14).2.
ConnectionState.shadowconflates template and live snapshotSymptom: serialization/blocks.ts:322
saveConnectioncallsgetShadowState(true), which serializes the live shadow block's current values. On load, that serialized state is written back intoshadowState_viasetShadowStateInternalin connection.ts, and then re-serialized again byserializeShadowat L622/L628/L632. After even a single save/load round trip, the original template is lost and the most recently saved live state becomes the new template. As a result, "the shadow template right after dropping from the flyout" and "the shadow template after a save/load cycle" no longer mean the same thing.Impact on this PR: During a cascade,
parent.shadowStatemay still be a template, or it may already be a live snapshot, depending on the calling context. This forced us to introduce a Case 1 (template != live) / Case 2 (template == live) split and anisSelfTemplateflag.Suggested upstream fix: Split the
ConnectionStateschema intoshadowTemplateandshadowOverride. This is a serialization-output change and therefore clearly breaking. It really should be solved together with section 1.3.
Connection.disconnectInternalimplicitly respawns shadow blocksSymptom: Around connection.ts:311, when a non-shadow child is disconnected, the parent silently runs
respawnShadow_(). This is a side effect that's invisible to API consumers, and it triggers a chain likedisconnect → respawn → getOrCreateVariablePackage → recreates a variable that was about to be deleted.Workaround in this PR: Introduced a
Workspace.suppressShadowRespawnflag and wrapped the entire cascade dispose loop with it to suppress respawning.Suggested upstream fix: Add a
{respawnShadow?: boolean}option todisconnect()so the side effect becomes opt-in at the call site. Changing the default would befeat!:; rewriting only internal callers while keeping the surface API compatible could potentially go in as a non-breakingfeat:. If section 1 is being addressed, the cleanest path is to replace the implicit respawn with an event-driven mechanism instead.4.
getOrCreateVariablePackage's lookup-or-create semanticsSymptom: variables.ts:691-702 returns an existing variable or creates one if missing. This is convenient when loading variables, but in order-sensitive contexts like a cascade, the result depends on whether you call it before or after the deletion has happened. In this PR, we had to split the cascade into two passes to make this work.
Suggested upstream fix: Add an explicit "always create" API such as
createVariableSameId(workspace, id, name, type). Since this leaves the existinggetOrCreateVariablePackageuntouched, it's an additive change and could land as afeat:in v13. The conditions are that it should still fire aVarCreateevent like the existingcreateVariable, and that its behavior when a variable with the same id already exists should be explicitly defined (throwing seems safest).5.
getVariableUsesByIdreturns the same block multiple timesSymptom: variables.ts:852-867 returns the same block multiple times if a single block contains multiple fields referencing the same variable. This PR works around it with a local
Set<string>dedupe insidedeleteVariable, butBlockly.Variables.deleteVariableat variables.ts:879 usesuses.lengthdirectly as the count shown in theDELETE_VARIABLE_CONFIRMATIONdialog. That makes it a user-visible bug that double-counts the same block. Theuses.length > 1threshold check is also off, so there are cases where a confirmation dialog appears even though only a single block uses the variable.Suggested upstream fix: Make
getVariableUsesByIditself dedupe by block id. The internal callers (VariableMap.deleteVariableat variable_map.ts:308 is already defended by anisDeadOrDying()guard; the deprecated wrappers just forward) are unaffected, and the dialog bug above gets fixed as a bonus. This should be doable as afix:in v13. Since it's independent of this PR, it could also be split out as a separate PR.