Add HTML block checker to enforce native Gutenberg blocks#3016
Add HTML block checker to enforce native Gutenberg blocks#3016
Conversation
Introduces a deterministic check_html_blocks tool that parses block content via wp.blocks.parse() in a real browser, identifies core/html blocks wrapping structural HTML (divs, sections, headings, etc.), and flags them with replacement suggestions. The check is also integrated into validate_blocks so both validations run in a single call. Updates the system prompt with detailed block pattern references and stricter guidelines.
Keeps trunk's refactoring (separate remote/local content guidelines) while applying the HTML block checker's enhanced block patterns and stricter guidelines to LOCAL_CONTENT_GUIDELINES.
📊 Performance Test ResultsComparing 7bcbc11 vs trunk app-size
site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
apps/cli/ai/system-prompt.ts
Outdated
|
|
||
| 1. **Get site details**: Use site_info to get the site path, URL, and credentials. | ||
| 2. **Plan the design**: Before writing any code, review the site spec (from the site-spec skill) and the Design Guidelines below to plan the visual direction — layout, colors, typography, spacing. | ||
| 2. **Plan the design and block structure**: Before writing any code, review the site spec (from the site-spec skill) and the Design Guidelines below to plan the visual direction — layout, colors, typography, spacing. Also plan how each section maps to Gutenberg blocks: which sections use \`core/group\`, where to use \`core/columns\`, which text is \`core/heading\` vs \`core/paragraph\`, etc. Refer to the Block Content Guidelines for the correct markup patterns. Do NOT default to \`core/html\` — compose in native blocks from the start. |
There was a problem hiding this comment.
What I don't like much is the coupling of the system prompt to the core blocks. Aren't the validation changes sufficient.
I'm also wondering why you used a single tool for both kind of validation (block validation and html blocks) instead of using separate tools. I'm not opinionated here as long as the solution works though.
There was a problem hiding this comment.
That's definitely a concern I have too, shared here also. The underlying problem seems to be that the agent tends to use html blocks all the time. Also in this comment I explain the issue I found with using only the tool.
I am going to experiment using external skills as an alternative or a more generic hint that doesn't couple that much with block implementations
There was a problem hiding this comment.
I'm also wondering why you used a single tool for both kind of validation (block validation and html blocks) instead of using separate tools. I'm not opinionated here as long as the solution works though.
This was updated because during testing the AI agent reliably called validate_blocks on every piece of block content but consistently forgot to call check_html_blocks as a separate step
apps/cli/ai/system-prompt.ts
Outdated
| - \`<form>\` elements and interactive inputs | ||
| - Animation/interaction markup with no block equivalent (marquee, cursor) | ||
| - A single \`<script>\` block at the bottom of the page for JS | ||
| **CRITICAL — Think in blocks, not HTML.** When writing page/post content or template parts, you MUST compose content using Gutenberg block markup from the start. Do NOT write raw HTML sections and wrap them in \`<!-- wp:html -->\`. Instead, use the block patterns below to build every section. |
There was a problem hiding this comment.
Did you find that the changes to the system prompt here are necessary? Can't we just rely on the tool for validation and the agent should be capable to fix things properly based on the errors thrown by the tool (like we do for regular block validation).
There was a problem hiding this comment.
That was my initial approach and didn't work well, the agent was in a loop that took around 45 minutes where:
- Initially created mainly html blocks
- Used the tool to replace the html blocks by other core blocks
- Refined the design and replaced the core blocks by html blocks
- Validated again using the tool and replacing back to core blocks
The final result included a mixture of Groups and html blocks nested. I recorded a video with that first approach, you can skip to the relevant parts and see the loop 3ae9e-pb, for example on the minute 7:20 the agent had validated al the html blocks to use core blocks but it continues until minute 46 to complete the site.
Then I changed the system prompt to emphasize as much as possible using blocks, that improved the usage of blocks
There was a problem hiding this comment.
I see, but doesn't this mean that the validation tool is now unnecessary (so basically your skill suggestion)?
There was a problem hiding this comment.
Yes, it is a bit redundant now, but I found that the agent still uses it and updates some blocks. I am going to continue testing, and if I don't find that the tool helps, I will remove it
|
One idea @mtias brings up all the time as well, is that maybe instead of asking our system prompt to generate the design in one step, we should split it into two steps: First generate the design in raw HTML + CSS, Second convert the HTML into blocks. |
I like that approach, as I had success in converting one of the testing sites from html to core blocks in a different session. I am going to iterate over that 👍 |
|
Yeah, two clean stages is worth a shot because there are various flows in which design > html+css is more straightforward for the model than design > blocks.
|
…sion Replace the single-pass workflow that mixed design and block concerns with two explicit phases. Phase 1 focuses on visual design with free HTML/CSS, Phase 2 converts content to native Gutenberg blocks with validation.
Remove check_html_blocks tool and its integration in validate_blocks. Phase 2 now relies on the agent self-identifying and converting HTML blocks based on the block content guidelines, matching the natural workflow observed in testing sessions. validate_blocks is kept for block markup validation only.
Related issues
How AI was used in this PR
Claude was used to implement the HTML block checker tool, write tests, and update the system prompt with block pattern references. All code was reviewed and iterated on by the author.
Proposed Changes
html-block-checker.ts: a deterministic tool that parses block content viawp.blocks.parse()in a real browser, identifiescore/htmlblocks wrapping structural HTML (divs, sections, headings, paragraphs, lists, etc.), and flags them with native Gutenberg block replacement suggestions.validate_blocksso both block markup validation and HTML block misuse detection run in a single call viaPromise.all. This was necessary because during testing the AI agent reliably calledvalidate_blockson every piece of block content but consistently forgot to callcheck_html_blocksas a separate step — it would validate markup, get a pass, and move on without ever checking for HTML block misuse. By merging both checks intovalidate_blocks, the HTML block analysis is guaranteed to run every time block content is validated, without relying on the AI to remember a second tool call.check_html_blockstool for independent use.data-*attributes to bypass the checkercore/htmlis acceptable (SVGs, forms, animation markup, scripts)Testing Instructions
npm run cli:buildandnode apps/cli/dist/cli/main.mjs aicore/htmlwrappers. For that, you can open<site-domain>/wp-admin/site-editor.php, click on the page and check theDocument overviewPre-merge Checklist